[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:11:59.153744  6937 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.198.126:36505
I20260709 13:11:59.155243  6937 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:11:59.156194  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:59.172751  6946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:59.172744  6944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:59.173849  6937 server_base.cc:1061] running on GCE node
W20260709 13:11:59.172912  6943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:00.355036  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:00.355284  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:00.355499  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602720355473 us; error 0 us; skew 500 ppm
I20260709 13:12:00.356336  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:00.362747  6937 webserver.cc:533] Webserver started at http://127.6.198.126:32805/ using document root <none> and password file <none>
I20260709 13:12:00.363610  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:00.363857  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:00.364308  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:00.369921  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/master-0-root/instance:
uuid: "dc559a6efa91405d8c8d255ceb7f8498"
format_stamp: "Formatted at 2026-07-09 13:12:00 on dist-test-slave-tqzw"
I20260709 13:12:00.377418  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:00.382458  6952 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:00.383621  6937 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20260709 13:12:00.383986  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/master-0-root
uuid: "dc559a6efa91405d8c8d255ceb7f8498"
format_stamp: "Formatted at 2026-07-09 13:12:00 on dist-test-slave-tqzw"
I20260709 13:12:00.384325  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:00.442514  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:00.444041  6937 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:12:00.444514  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:00.515723  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.126:36505
I20260709 13:12:00.515793  7003 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.126:36505 every 8 connection(s)
I20260709 13:12:00.523591  7004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:00.544021  7004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498: Bootstrap starting.
I20260709 13:12:00.549551  7004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:00.551241  7004 log.cc:826] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498: Log is configured to *not* fsync() on all Append() calls
I20260709 13:12:00.555480  7004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498: No bootstrap required, opened a new log
I20260709 13:12:00.574868  7004 raft_consensus.cc:359] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc559a6efa91405d8c8d255ceb7f8498" member_type: VOTER }
I20260709 13:12:00.575377  7004 raft_consensus.cc:385] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:00.575577  7004 raft_consensus.cc:740] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc559a6efa91405d8c8d255ceb7f8498, State: Initialized, Role: FOLLOWER
I20260709 13:12:00.576363  7004 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc559a6efa91405d8c8d255ceb7f8498" member_type: VOTER }
I20260709 13:12:00.576804  7004 raft_consensus.cc:399] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:12:00.577039  7004 raft_consensus.cc:493] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:12:00.577310  7004 raft_consensus.cc:3060] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:00.581200  7004 raft_consensus.cc:515] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc559a6efa91405d8c8d255ceb7f8498" member_type: VOTER }
I20260709 13:12:00.581862  7004 leader_election.cc:304] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc559a6efa91405d8c8d255ceb7f8498; no voters: 
I20260709 13:12:00.583494  7004 leader_election.cc:290] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:12:00.583887  7007 raft_consensus.cc:2804] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:00.586445  7007 raft_consensus.cc:697] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [term 1 LEADER]: Becoming Leader. State: Replica: dc559a6efa91405d8c8d255ceb7f8498, State: Running, Role: LEADER
I20260709 13:12:00.587255  7007 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc559a6efa91405d8c8d255ceb7f8498" member_type: VOTER }
I20260709 13:12:00.587756  7004 sys_catalog.cc:565] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:12:00.599368  7008 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc559a6efa91405d8c8d255ceb7f8498" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc559a6efa91405d8c8d255ceb7f8498" member_type: VOTER } }
I20260709 13:12:00.600167  7008 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:00.600584  7009 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc559a6efa91405d8c8d255ceb7f8498. Latest consensus state: current_term: 1 leader_uuid: "dc559a6efa91405d8c8d255ceb7f8498" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc559a6efa91405d8c8d255ceb7f8498" member_type: VOTER } }
I20260709 13:12:00.601053  7009 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:00.603276  7017 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:12:00.614005  7017 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:12:00.616055  6937 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:12:00.631031  7017 catalog_manager.cc:1383] Generated new cluster ID: 4bbf54c878464c118bf566e55805d70d
I20260709 13:12:00.631331  7017 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:12:00.643479  7017 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:12:00.645512  7017 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:12:00.670048  7017 catalog_manager.cc:6070] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498: Generated new TSK 0
I20260709 13:12:00.671233  7017 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:12:00.682307  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:00.690697  7026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:00.692145  7027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:00.697578  6937 server_base.cc:1061] running on GCE node
W20260709 13:12:00.698112  7029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:00.699012  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:00.699293  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:00.699517  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602720699495 us; error 0 us; skew 500 ppm
I20260709 13:12:00.700310  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:00.704514  6937 webserver.cc:533] Webserver started at http://127.6.198.65:45817/ using document root <none> and password file <none>
I20260709 13:12:00.705076  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:00.705322  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:00.705641  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:00.706811  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-0-root/instance:
uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
format_stamp: "Formatted at 2026-07-09 13:12:00 on dist-test-slave-tqzw"
I20260709 13:12:00.712492  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:12:00.716557  7034 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:00.717423  6937 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:12:00.717777  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-0-root
uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
format_stamp: "Formatted at 2026-07-09 13:12:00 on dist-test-slave-tqzw"
I20260709 13:12:00.718114  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:00.736050  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:00.737375  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:00.740170  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:00.744122  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:00.744385  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:00.744649  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:00.744845  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:00.847311  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.65:45219
I20260709 13:12:00.847368  7096 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.65:45219 every 8 connection(s)
I20260709 13:12:00.854918  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:00.867408  7101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:00.870877  7102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:00.876752  6937 server_base.cc:1061] running on GCE node
W20260709 13:12:00.878332  7104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:00.879527  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:00.879840  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:00.880064  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602720880047 us; error 0 us; skew 500 ppm
I20260709 13:12:00.880721  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:00.883775  6937 webserver.cc:533] Webserver started at http://127.6.198.66:39395/ using document root <none> and password file <none>
I20260709 13:12:00.884277  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:00.884546  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:00.884863  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:00.886166  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-1-root/instance:
uuid: "5aefd9529a6f4747b12913dd7875a6be"
format_stamp: "Formatted at 2026-07-09 13:12:00 on dist-test-slave-tqzw"
I20260709 13:12:00.888669  7097 heartbeater.cc:344] Connected to a master server at 127.6.198.126:36505
I20260709 13:12:00.889137  7097 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:00.890354  7097 heartbeater.cc:507] Master 127.6.198.126:36505 requested a full tablet report, sending...
I20260709 13:12:00.892616  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:12:00.893584  6969 ts_manager.cc:194] Registered new tserver with Master: 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219)
I20260709 13:12:00.895879  6969 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52478
I20260709 13:12:00.897730  7109 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:00.898672  6937 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:12:00.899032  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-1-root
uuid: "5aefd9529a6f4747b12913dd7875a6be"
format_stamp: "Formatted at 2026-07-09 13:12:00 on dist-test-slave-tqzw"
I20260709 13:12:00.899389  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:00.921381  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:00.923043  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:00.925508  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:00.929031  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:00.929286  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:00.929557  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:00.929769  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:01.022751  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.66:33387
I20260709 13:12:01.022920  7171 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.66:33387 every 8 connection(s)
I20260709 13:12:01.031128  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:01.040975  7175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:01.048689  7172 heartbeater.cc:344] Connected to a master server at 127.6.198.126:36505
I20260709 13:12:01.049170  7172 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:01.050424  7172 heartbeater.cc:507] Master 127.6.198.126:36505 requested a full tablet report, sending...
I20260709 13:12:01.053416  6969 ts_manager.cc:194] Registered new tserver with Master: 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
I20260709 13:12:01.055758  6969 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52490
W20260709 13:12:01.047079  7176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:01.060075  6937 server_base.cc:1061] running on GCE node
W20260709 13:12:01.061805  7178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:01.062841  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:01.063102  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:01.063333  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602721063315 us; error 0 us; skew 500 ppm
I20260709 13:12:01.064036  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:01.066686  6937 webserver.cc:533] Webserver started at http://127.6.198.67:45541/ using document root <none> and password file <none>
I20260709 13:12:01.067296  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:01.067524  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:01.067829  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:01.069262  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-2-root/instance:
uuid: "658bfc3ebb1b4380a823078d38152ad5"
format_stamp: "Formatted at 2026-07-09 13:12:01 on dist-test-slave-tqzw"
I20260709 13:12:01.074817  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:12:01.078727  7183 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:01.079550  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:12:01.079862  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-2-root
uuid: "658bfc3ebb1b4380a823078d38152ad5"
format_stamp: "Formatted at 2026-07-09 13:12:01 on dist-test-slave-tqzw"
I20260709 13:12:01.080212  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602718985547-6937-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:01.117640  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:01.119205  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:01.120870  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:01.123533  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:01.123788  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:01.124056  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:01.124269  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:01.210285  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.67:38755
I20260709 13:12:01.210403  7245 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.67:38755 every 8 connection(s)
I20260709 13:12:01.226724  7246 heartbeater.cc:344] Connected to a master server at 127.6.198.126:36505
I20260709 13:12:01.227100  7246 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:01.227850  7246 heartbeater.cc:507] Master 127.6.198.126:36505 requested a full tablet report, sending...
I20260709 13:12:01.229836  6969 ts_manager.cc:194] Registered new tserver with Master: 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:01.230638  6937 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014777362s
I20260709 13:12:01.231518  6969 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52492
I20260709 13:12:01.287591  6969 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52498:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:12:01.291119  6969 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:12:01.373482  7137 tablet_service.cc:1511] Processing CreateTablet for tablet 15eb2f9468c244b38238d84d56343214 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:01.374213  7136 tablet_service.cc:1511] Processing CreateTablet for tablet e001b76d430e47e9a4bc37528a3b0509 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:01.375156  7137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15eb2f9468c244b38238d84d56343214. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.375923  7136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e001b76d430e47e9a4bc37528a3b0509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.391513  7135 tablet_service.cc:1511] Processing CreateTablet for tablet f1d6d86807c34fe0b76ef5ea2d2e25db (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:01.392869  7135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d6d86807c34fe0b76ef5ea2d2e25db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.398190  7134 tablet_service.cc:1511] Processing CreateTablet for tablet dce58ff3844546a5a97a84b771250c34 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:01.403065  7134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dce58ff3844546a5a97a84b771250c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.413419  7133 tablet_service.cc:1511] Processing CreateTablet for tablet 9ff96dd48bc94ab19d8146a755399808 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:01.416366  7211 tablet_service.cc:1511] Processing CreateTablet for tablet 15eb2f9468c244b38238d84d56343214 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:01.418098  7133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ff96dd48bc94ab19d8146a755399808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.418529  7264 tablet_bootstrap.cc:492] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be: Bootstrap starting.
I20260709 13:12:01.419898  7208 tablet_service.cc:1511] Processing CreateTablet for tablet dce58ff3844546a5a97a84b771250c34 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:01.420481  7209 tablet_service.cc:1511] Processing CreateTablet for tablet f1d6d86807c34fe0b76ef5ea2d2e25db (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:01.415936  7210 tablet_service.cc:1511] Processing CreateTablet for tablet e001b76d430e47e9a4bc37528a3b0509 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:01.423697  7207 tablet_service.cc:1511] Processing CreateTablet for tablet 9ff96dd48bc94ab19d8146a755399808 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:01.424665  7211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15eb2f9468c244b38238d84d56343214. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.425668  7210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e001b76d430e47e9a4bc37528a3b0509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.426604  7209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d6d86807c34fe0b76ef5ea2d2e25db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.428447  7207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ff96dd48bc94ab19d8146a755399808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.430511  7208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dce58ff3844546a5a97a84b771250c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.436743  7062 tablet_service.cc:1511] Processing CreateTablet for tablet 15eb2f9468c244b38238d84d56343214 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:01.442207  7059 tablet_service.cc:1511] Processing CreateTablet for tablet dce58ff3844546a5a97a84b771250c34 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:01.448180  7062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15eb2f9468c244b38238d84d56343214. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.441804  7061 tablet_service.cc:1511] Processing CreateTablet for tablet e001b76d430e47e9a4bc37528a3b0509 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:01.452112  7206 tablet_service.cc:1511] Processing CreateTablet for tablet 16cb2fdc74e64e8cbda0c21b3e5081ab (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:01.449980  7058 tablet_service.cc:1511] Processing CreateTablet for tablet 9ff96dd48bc94ab19d8146a755399808 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:01.452693  7060 tablet_service.cc:1511] Processing CreateTablet for tablet f1d6d86807c34fe0b76ef5ea2d2e25db (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:01.457104  7059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dce58ff3844546a5a97a84b771250c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.458804  7206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16cb2fdc74e64e8cbda0c21b3e5081ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.455672  7137 tablet_service.cc:1511] Processing CreateTablet for tablet 16cb2fdc74e64e8cbda0c21b3e5081ab (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:01.461225  7137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16cb2fdc74e64e8cbda0c21b3e5081ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.458686  7057 tablet_service.cc:1511] Processing CreateTablet for tablet 16cb2fdc74e64e8cbda0c21b3e5081ab (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:01.465188  7058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ff96dd48bc94ab19d8146a755399808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.483404  7060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1d6d86807c34fe0b76ef5ea2d2e25db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.485427  7057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16cb2fdc74e64e8cbda0c21b3e5081ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.487406  7061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e001b76d430e47e9a4bc37528a3b0509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.492100  7136 tablet_service.cc:1511] Processing CreateTablet for tablet b18aa7629be0450988a1b7bcaacba182 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:01.502876  7056 tablet_service.cc:1511] Processing CreateTablet for tablet b18aa7629be0450988a1b7bcaacba182 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:01.506387  7056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b18aa7629be0450988a1b7bcaacba182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.508282  7136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b18aa7629be0450988a1b7bcaacba182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.544646  7205 tablet_service.cc:1511] Processing CreateTablet for tablet b18aa7629be0450988a1b7bcaacba182 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:01.546938  7205 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b18aa7629be0450988a1b7bcaacba182. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.603708  7055 tablet_service.cc:1511] Processing CreateTablet for tablet 6294e276896f402b8d8ea2a0af5dbbe4 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:01.608353  7055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6294e276896f402b8d8ea2a0af5dbbe4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:12:01.611899  7016 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:01.619331  7264 tablet_bootstrap.cc:654] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:01.640098  7133 tablet_service.cc:1511] Processing CreateTablet for tablet 6294e276896f402b8d8ea2a0af5dbbe4 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:01.641417  7133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6294e276896f402b8d8ea2a0af5dbbe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.681519  7204 tablet_service.cc:1511] Processing CreateTablet for tablet 6294e276896f402b8d8ea2a0af5dbbe4 (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:01.683548  7204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6294e276896f402b8d8ea2a0af5dbbe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.682309  7264 tablet_bootstrap.cc:492] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be: No bootstrap required, opened a new log
I20260709 13:12:01.698846  7264 ts_tablet_manager.cc:1403] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be: Time spent bootstrapping tablet: real 0.289s	user 0.018s	sys 0.028s
I20260709 13:12:01.705554  7266 tablet_bootstrap.cc:492] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea: Bootstrap starting.
I20260709 13:12:01.716452  7267 tablet_bootstrap.cc:492] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5: Bootstrap starting.
I20260709 13:12:01.728242  7134 tablet_service.cc:1511] Processing CreateTablet for tablet cba29b3b993144c8b6e088785509ad5b (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:01.728507  7056 tablet_service.cc:1511] Processing CreateTablet for tablet cba29b3b993144c8b6e088785509ad5b (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:01.729507  7134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cba29b3b993144c8b6e088785509ad5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.737752  7264 raft_consensus.cc:359] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.738595  7264 raft_consensus.cc:385] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:01.729516  7056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cba29b3b993144c8b6e088785509ad5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.738934  7264 raft_consensus.cc:740] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Initialized, Role: FOLLOWER
I20260709 13:12:01.740001  7264 consensus_queue.cc:260] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.747866  7172 heartbeater.cc:499] Master 127.6.198.126:36505 was elected leader, sending a full tablet report...
I20260709 13:12:01.752122  7211 tablet_service.cc:1511] Processing CreateTablet for tablet cba29b3b993144c8b6e088785509ad5b (DEFAULT_TABLE table=test-workload [id=dfff0a6aa0dc41cab14a784a47c7e533]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:01.754158  7264 ts_tablet_manager.cc:1434] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.003s
I20260709 13:12:01.757328  7211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cba29b3b993144c8b6e088785509ad5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:01.771744  7267 tablet_bootstrap.cc:654] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:01.772099  7264 tablet_bootstrap.cc:492] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be: Bootstrap starting.
I20260709 13:12:01.777881  7266 tablet_bootstrap.cc:654] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:01.781306  7267 tablet_bootstrap.cc:492] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5: No bootstrap required, opened a new log
I20260709 13:12:01.783766  7267 ts_tablet_manager.cc:1403] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5: Time spent bootstrapping tablet: real 0.075s	user 0.007s	sys 0.007s
W20260709 13:12:01.783969  7173 tablet.cc:2406] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:12:01.787518  7267 raft_consensus.cc:359] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.788451  7267 raft_consensus.cc:385] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:01.788745  7267 raft_consensus.cc:740] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Initialized, Role: FOLLOWER
I20260709 13:12:01.789649  7267 consensus_queue.cc:260] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.792593  7246 heartbeater.cc:499] Master 127.6.198.126:36505 was elected leader, sending a full tablet report...
I20260709 13:12:01.795264  7267 ts_tablet_manager.cc:1434] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:12:01.796120  7267 tablet_bootstrap.cc:492] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5: Bootstrap starting.
I20260709 13:12:01.799916  7264 tablet_bootstrap.cc:654] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:01.800258  7266 tablet_bootstrap.cc:492] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea: No bootstrap required, opened a new log
I20260709 13:12:01.800753  7266 ts_tablet_manager.cc:1403] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent bootstrapping tablet: real 0.124s	user 0.011s	sys 0.009s
I20260709 13:12:01.802788  7267 tablet_bootstrap.cc:654] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:01.806715  7266 raft_consensus.cc:359] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.807596  7266 raft_consensus.cc:385] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:01.807952  7266 raft_consensus.cc:740] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Initialized, Role: FOLLOWER
I20260709 13:12:01.808890  7266 consensus_queue.cc:260] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.816840  7264 tablet_bootstrap.cc:492] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be: No bootstrap required, opened a new log
I20260709 13:12:01.817324  7264 ts_tablet_manager.cc:1403] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be: Time spent bootstrapping tablet: real 0.059s	user 0.010s	sys 0.004s
I20260709 13:12:01.819852  7097 heartbeater.cc:499] Master 127.6.198.126:36505 was elected leader, sending a full tablet report...
I20260709 13:12:01.820271  7266 ts_tablet_manager.cc:1434] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent starting tablet: real 0.019s	user 0.000s	sys 0.013s
I20260709 13:12:01.820523  7264 raft_consensus.cc:359] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.821271  7266 tablet_bootstrap.cc:492] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea: Bootstrap starting.
I20260709 13:12:01.821354  7264 raft_consensus.cc:385] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:01.821697  7264 raft_consensus.cc:740] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Initialized, Role: FOLLOWER
I20260709 13:12:01.822492  7264 consensus_queue.cc:260] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.824818  7264 ts_tablet_manager.cc:1434] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:12:01.825709  7264 tablet_bootstrap.cc:492] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be: Bootstrap starting.
I20260709 13:12:01.827706  7266 tablet_bootstrap.cc:654] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:01.839607  7266 tablet_bootstrap.cc:492] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea: No bootstrap required, opened a new log
I20260709 13:12:01.840054  7266 ts_tablet_manager.cc:1403] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.008s
I20260709 13:12:01.840626  7267 tablet_bootstrap.cc:492] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5: No bootstrap required, opened a new log
I20260709 13:12:01.841104  7267 ts_tablet_manager.cc:1403] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.019s
I20260709 13:12:01.843084  7266 raft_consensus.cc:359] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.843813  7266 raft_consensus.cc:385] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:01.844496  7266 raft_consensus.cc:740] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Initialized, Role: FOLLOWER
I20260709 13:12:01.844909  7267 raft_consensus.cc:359] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.845603  7267 raft_consensus.cc:385] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:01.845914  7267 raft_consensus.cc:740] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Initialized, Role: FOLLOWER
I20260709 13:12:01.845175  7266 consensus_queue.cc:260] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.846880  7267 consensus_queue.cc:260] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.848917  7267 ts_tablet_manager.cc:1434] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:12:01.849749  7267 tablet_bootstrap.cc:492] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5: Bootstrap starting.
I20260709 13:12:01.855808  7266 ts_tablet_manager.cc:1434] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 13:12:01.856714  7266 tablet_bootstrap.cc:492] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea: Bootstrap starting.
I20260709 13:12:01.859848  7264 tablet_bootstrap.cc:654] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:01.860255  7267 tablet_bootstrap.cc:654] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:01.866847  7266 tablet_bootstrap.cc:654] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:01.869376  7267 tablet_bootstrap.cc:492] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5: No bootstrap required, opened a new log
I20260709 13:12:01.869875  7267 ts_tablet_manager.cc:1403] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.008s
I20260709 13:12:01.870241  7264 tablet_bootstrap.cc:492] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be: No bootstrap required, opened a new log
I20260709 13:12:01.870719  7264 ts_tablet_manager.cc:1403] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be: Time spent bootstrapping tablet: real 0.045s	user 0.009s	sys 0.003s
I20260709 13:12:01.872818  7267 raft_consensus.cc:359] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.873968  7267 raft_consensus.cc:385] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:01.873368  7264 raft_consensus.cc:359] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.874362  7267 raft_consensus.cc:740] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Initialized, Role: FOLLOWER
I20260709 13:12:01.874652  7264 raft_consensus.cc:385] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:01.874985  7264 raft_consensus.cc:740] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Initialized, Role: FOLLOWER
I20260709 13:12:01.875368  7267 consensus_queue.cc:260] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.875720  7264 consensus_queue.cc:260] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.878268  7264 ts_tablet_manager.cc:1434] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:01.879134  7264 tablet_bootstrap.cc:492] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be: Bootstrap starting.
I20260709 13:12:01.885557  7264 tablet_bootstrap.cc:654] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:01.888772  7266 tablet_bootstrap.cc:492] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea: No bootstrap required, opened a new log
I20260709 13:12:01.889281  7266 ts_tablet_manager.cc:1403] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent bootstrapping tablet: real 0.033s	user 0.024s	sys 0.002s
I20260709 13:12:01.890947  7267 ts_tablet_manager.cc:1434] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.003s
I20260709 13:12:01.891959  7267 tablet_bootstrap.cc:492] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5: Bootstrap starting.
I20260709 13:12:01.892115  7266 raft_consensus.cc:359] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.892841  7266 raft_consensus.cc:385] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:01.893139  7266 raft_consensus.cc:740] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Initialized, Role: FOLLOWER
I20260709 13:12:01.900085  7266 consensus_queue.cc:260] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.902364  7266 ts_tablet_manager.cc:1434] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:12:01.903304  7266 tablet_bootstrap.cc:492] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea: Bootstrap starting.
I20260709 13:12:01.904327  7267 tablet_bootstrap.cc:654] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:01.910089  7271 raft_consensus.cc:493] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.910647  7271 raft_consensus.cc:515] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.914366  7268 raft_consensus.cc:493] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.914866  7268 raft_consensus.cc:515] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.916400  7267 tablet_bootstrap.cc:492] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5: No bootstrap required, opened a new log
I20260709 13:12:01.916867  7267 ts_tablet_manager.cc:1403] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.003s
I20260709 13:12:01.917878  7268 leader_election.cc:290] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:01.920037  7267 raft_consensus.cc:359] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.921092  7267 raft_consensus.cc:385] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:01.921489  7267 raft_consensus.cc:740] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Initialized, Role: FOLLOWER
I20260709 13:12:01.922418  7267 consensus_queue.cc:260] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.924762  7271 leader_election.cc:290] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
I20260709 13:12:01.929324  7268 raft_consensus.cc:493] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.929884  7268 raft_consensus.cc:515] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.932224  7268 leader_election.cc:290] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:01.937397  7264 tablet_bootstrap.cc:492] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be: No bootstrap required, opened a new log
I20260709 13:12:01.937855  7264 ts_tablet_manager.cc:1403] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be: Time spent bootstrapping tablet: real 0.059s	user 0.007s	sys 0.011s
I20260709 13:12:01.940593  7264 raft_consensus.cc:359] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.941282  7264 raft_consensus.cc:385] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:01.941571  7264 raft_consensus.cc:740] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Initialized, Role: FOLLOWER
I20260709 13:12:01.942266  7264 consensus_queue.cc:260] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.944479  7264 ts_tablet_manager.cc:1434] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:12:01.945147  7267 ts_tablet_manager.cc:1434] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5: Time spent starting tablet: real 0.028s	user 0.014s	sys 0.012s
I20260709 13:12:01.945377  7264 tablet_bootstrap.cc:492] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be: Bootstrap starting.
I20260709 13:12:01.946110  7267 tablet_bootstrap.cc:492] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5: Bootstrap starting.
I20260709 13:12:01.953344  7267 tablet_bootstrap.cc:654] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:01.968473  7264 tablet_bootstrap.cc:654] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:01.954319  7266 tablet_bootstrap.cc:654] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:01.981810  7267 tablet_bootstrap.cc:492] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5: No bootstrap required, opened a new log
I20260709 13:12:01.982265  7267 ts_tablet_manager.cc:1403] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5: Time spent bootstrapping tablet: real 0.036s	user 0.004s	sys 0.008s
I20260709 13:12:01.985352  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16cb2fdc74e64e8cbda0c21b3e5081ab" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be" is_pre_election: true
I20260709 13:12:01.985525  7267 raft_consensus.cc:359] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.986608  7267 raft_consensus.cc:385] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:01.987032  7267 raft_consensus.cc:740] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Initialized, Role: FOLLOWER
W20260709 13:12:01.987258  7184 leader_election.cc:343] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:01.988024  7267 consensus_queue.cc:260] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.989569  7271 raft_consensus.cc:493] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.989785  7287 raft_consensus.cc:493] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.990047  7271 raft_consensus.cc:515] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.990381  7071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" is_pre_election: true
I20260709 13:12:01.991441  7072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15eb2f9468c244b38238d84d56343214" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" is_pre_election: true
W20260709 13:12:01.992004  7112 leader_election.cc:343] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:01.990260  7287 raft_consensus.cc:515] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:01.993148  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be" is_pre_election: true
I20260709 13:12:01.992125  7271 leader_election.cc:290] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
I20260709 13:12:01.993803  7147 raft_consensus.cc:2468] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 0.
I20260709 13:12:01.993916  7287 leader_election.cc:290] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
W20260709 13:12:01.994372  7112 leader_election.cc:343] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:01.996695  7267 ts_tablet_manager.cc:1434] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:12:01.997773  7267 tablet_bootstrap.cc:492] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5: Bootstrap starting.
I20260709 13:12:01.998150  7266 tablet_bootstrap.cc:492] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea: No bootstrap required, opened a new log
I20260709 13:12:01.998672  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be" is_pre_election: true
I20260709 13:12:01.999277  7147 raft_consensus.cc:2468] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 0.
I20260709 13:12:01.998696  7266 ts_tablet_manager.cc:1403] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent bootstrapping tablet: real 0.096s	user 0.007s	sys 0.005s
I20260709 13:12:01.998203  7184 leader_election.cc:304] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:02.003177  7287 raft_consensus.cc:2804] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:02.003306  7184 leader_election.cc:304] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:02.003716  7287 raft_consensus.cc:493] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.004159  7287 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.004261  7266 raft_consensus.cc:359] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.005035  7266 raft_consensus.cc:385] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.005370  7266 raft_consensus.cc:740] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.006060  7266 consensus_queue.cc:260] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.007113  7271 raft_consensus.cc:2804] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:02.007467  7271 raft_consensus.cc:493] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.007771  7271 raft_consensus.cc:3060] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.008389  7266 ts_tablet_manager.cc:1434] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:12:02.009266  7266 tablet_bootstrap.cc:492] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea: Bootstrap starting.
I20260709 13:12:02.023316  7271 raft_consensus.cc:515] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.025715  7072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16cb2fdc74e64e8cbda0c21b3e5081ab" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" is_pre_election: true
I20260709 13:12:02.026381  7072 raft_consensus.cc:2468] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 0.
I20260709 13:12:02.026799  7071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" is_pre_election: true
I20260709 13:12:02.027647  7266 tablet_bootstrap.cc:654] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:02.028471  7186 leader_election.cc:304] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5; no voters: 5aefd9529a6f4747b12913dd7875a6be
W20260709 13:12:02.029644  7186 leader_election.cc:343] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:02.031684  7069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:02.027477  7070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" is_pre_election: true
I20260709 13:12:02.032481  7264 tablet_bootstrap.cc:492] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be: No bootstrap required, opened a new log
I20260709 13:12:02.032918  7264 ts_tablet_manager.cc:1403] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be: Time spent bootstrapping tablet: real 0.088s	user 0.021s	sys 0.028s
W20260709 13:12:02.033713  7186 leader_election.cc:343] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:02.034444  7287 raft_consensus.cc:515] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
W20260709 13:12:02.035364  7186 leader_election.cc:343] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:02.035300  7221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15eb2f9468c244b38238d84d56343214" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658bfc3ebb1b4380a823078d38152ad5" is_pre_election: true
I20260709 13:12:02.035646  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658bfc3ebb1b4380a823078d38152ad5" is_pre_election: true
I20260709 13:12:02.036120  7221 raft_consensus.cc:2468] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 0.
I20260709 13:12:02.038870  7264 raft_consensus.cc:359] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
W20260709 13:12:02.039220  7111 leader_election.cc:343] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:02.039619  7264 raft_consensus.cc:385] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.039697  7111 leader_election.cc:304] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be; no voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5
I20260709 13:12:02.039958  7264 raft_consensus.cc:740] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.040539  7268 raft_consensus.cc:2749] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:02.041097  7111 leader_election.cc:304] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5; no voters: 5b9678bd83b64815bb6c7123b7d6e0ea
I20260709 13:12:02.040764  7264 consensus_queue.cc:260] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.041850  7268 raft_consensus.cc:2804] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:02.042218  7292 raft_consensus.cc:2804] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:02.042332  7268 raft_consensus.cc:493] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.042857  7292 raft_consensus.cc:493] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.042922  7268 raft_consensus.cc:3060] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.043396  7292 raft_consensus.cc:3060] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.044133  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:02.044685  7069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:02.044678  7147 raft_consensus.cc:3060] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.050676  7292 raft_consensus.cc:515] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
W20260709 13:12:02.051575  7186 leader_election.cc:343] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:02.053998  7146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16cb2fdc74e64e8cbda0c21b3e5081ab" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:02.054569  7069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16cb2fdc74e64e8cbda0c21b3e5081ab" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:02.055078  7069 raft_consensus.cc:3060] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:02.055455  7184 leader_election.cc:343] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:02.056041  7292 leader_election.cc:290] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
I20260709 13:12:02.053108  7147 raft_consensus.cc:2468] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 1.
I20260709 13:12:02.057507  7184 leader_election.cc:304] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5; no voters: 5b9678bd83b64815bb6c7123b7d6e0ea
I20260709 13:12:02.043058  7271 leader_election.cc:290] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
I20260709 13:12:02.058974  7271 raft_consensus.cc:2804] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:02.062554  7271 raft_consensus.cc:697] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Becoming Leader. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Running, Role: LEADER
I20260709 13:12:02.060554  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:02.063158  7147 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.066670  7266 tablet_bootstrap.cc:492] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea: No bootstrap required, opened a new log
I20260709 13:12:02.067103  7266 ts_tablet_manager.cc:1403] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent bootstrapping tablet: real 0.058s	user 0.025s	sys 0.011s
I20260709 13:12:02.061398  7268 raft_consensus.cc:515] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.062057  7287 leader_election.cc:290] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
I20260709 13:12:02.063403  7271 consensus_queue.cc:237] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.061609  7069 raft_consensus.cc:2468] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 1.
I20260709 13:12:02.065326  7264 ts_tablet_manager.cc:1434] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.003s
I20260709 13:12:02.071797  7264 tablet_bootstrap.cc:492] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be: Bootstrap starting.
I20260709 13:12:02.071833  7186 leader_election.cc:304] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5; no voters: 5aefd9529a6f4747b12913dd7875a6be
I20260709 13:12:02.070880  7267 tablet_bootstrap.cc:654] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:02.074576  7287 raft_consensus.cc:2804] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:02.075088  7287 raft_consensus.cc:697] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Becoming Leader. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Running, Role: LEADER
I20260709 13:12:02.075913  7287 consensus_queue.cc:237] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.078974  7264 tablet_bootstrap.cc:654] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:02.069937  7266 raft_consensus.cc:359] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.079370  7266 raft_consensus.cc:385] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.081514  7069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15eb2f9468c244b38238d84d56343214" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:02.081816  7266 raft_consensus.cc:740] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.082845  7221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15eb2f9468c244b38238d84d56343214" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:02.083544  7221 raft_consensus.cc:3060] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.083601  7147 raft_consensus.cc:2468] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 1.
I20260709 13:12:02.085119  7184 leader_election.cc:304] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5; no voters: 5b9678bd83b64815bb6c7123b7d6e0ea
I20260709 13:12:02.085351  7266 consensus_queue.cc:260] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.082149  7069 raft_consensus.cc:3060] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.086177  7292 raft_consensus.cc:2804] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:02.086665  7268 leader_election.cc:290] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:02.091759  7268 raft_consensus.cc:493] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.092404  7268 raft_consensus.cc:515] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.095171  7070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18aa7629be0450988a1b7bcaacba182" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" is_pre_election: true
I20260709 13:12:02.095757  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18aa7629be0450988a1b7bcaacba182" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658bfc3ebb1b4380a823078d38152ad5" is_pre_election: true
I20260709 13:12:02.095952  7070 raft_consensus.cc:2468] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 0.
I20260709 13:12:02.096438  7220 raft_consensus.cc:2468] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 0.
I20260709 13:12:02.097513  7111 leader_election.cc:304] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:02.098570  7294 raft_consensus.cc:2804] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:02.098948  7294 raft_consensus.cc:493] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.099280  7294 raft_consensus.cc:3060] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.105891  7266 ts_tablet_manager.cc:1434] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.004s
I20260709 13:12:02.106120  7294 raft_consensus.cc:515] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.106906  7266 tablet_bootstrap.cc:492] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea: Bootstrap starting.
I20260709 13:12:02.108269  7294 leader_election.cc:290] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:02.109186  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18aa7629be0450988a1b7bcaacba182" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:02.109117  7070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18aa7629be0450988a1b7bcaacba182" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:02.109843  7220 raft_consensus.cc:3060] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.110752  7268 leader_election.cc:290] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:02.113032  6968 catalog_manager.cc:5697] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "658bfc3ebb1b4380a823078d38152ad5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:02.121665  7069 raft_consensus.cc:2468] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 1.
I20260709 13:12:02.123135  7112 leader_election.cc:304] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 5b9678bd83b64815bb6c7123b7d6e0ea; no voters: 
I20260709 13:12:02.124506  7268 raft_consensus.cc:2804] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:02.125084  7268 raft_consensus.cc:697] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Becoming Leader. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:02.126189  7268 consensus_queue.cc:237] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.128880  7220 raft_consensus.cc:2468] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 1.
I20260709 13:12:02.130234  7111 leader_election.cc:304] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:02.131321  7294 raft_consensus.cc:2804] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:02.131529  7221 raft_consensus.cc:2468] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 1.
I20260709 13:12:02.131685  7274 raft_consensus.cc:493] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.132305  7274 raft_consensus.cc:515] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.117803  7266 tablet_bootstrap.cc:654] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:02.120443  7070 raft_consensus.cc:3060] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.136631  7274 leader_election.cc:290] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:02.146162  7294 raft_consensus.cc:697] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Becoming Leader. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:02.152665  7294 consensus_queue.cc:237] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.189564  7267 tablet_bootstrap.cc:492] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5: No bootstrap required, opened a new log
I20260709 13:12:02.187342  6967 catalog_manager.cc:5697] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be reported cstate change: term changed from 0 to 1, leader changed from <none> to 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66). New cstate: current_term: 1 leader_uuid: "5aefd9529a6f4747b12913dd7875a6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:02.190124  7292 raft_consensus.cc:697] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Becoming Leader. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Running, Role: LEADER
I20260709 13:12:02.197840  7264 tablet_bootstrap.cc:492] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be: No bootstrap required, opened a new log
I20260709 13:12:02.198869  7267 ts_tablet_manager.cc:1403] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5: Time spent bootstrapping tablet: real 0.201s	user 0.012s	sys 0.015s
I20260709 13:12:02.211108  7292 consensus_queue.cc:237] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.206187  7070 raft_consensus.cc:2468] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 1.
I20260709 13:12:02.210973  7264 ts_tablet_manager.cc:1403] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be: Time spent bootstrapping tablet: real 0.139s	user 0.027s	sys 0.038s
I20260709 13:12:02.222524  7266 tablet_bootstrap.cc:492] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea: No bootstrap required, opened a new log
I20260709 13:12:02.223008  7266 ts_tablet_manager.cc:1403] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent bootstrapping tablet: real 0.116s	user 0.011s	sys 0.012s
I20260709 13:12:02.219861  7267 raft_consensus.cc:359] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.224846  7267 raft_consensus.cc:385] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.227419  7266 raft_consensus.cc:359] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.228363  7266 raft_consensus.cc:385] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.228722  7266 raft_consensus.cc:740] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.225222  7267 raft_consensus.cc:740] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.237617  7266 consensus_queue.cc:260] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.241854  7267 consensus_queue.cc:260] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.244449  7266 ts_tablet_manager.cc:1434] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent starting tablet: real 0.021s	user 0.001s	sys 0.006s
I20260709 13:12:02.245386  7266 tablet_bootstrap.cc:492] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea: Bootstrap starting.
I20260709 13:12:02.241066  7264 raft_consensus.cc:359] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.249193  7264 raft_consensus.cc:385] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.249509  7264 raft_consensus.cc:740] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.249869  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6294e276896f402b8d8ea2a0af5dbbe4" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be" is_pre_election: true
I20260709 13:12:02.252559  7267 ts_tablet_manager.cc:1434] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 13:12:02.253445  7267 tablet_bootstrap.cc:492] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5: Bootstrap starting.
W20260709 13:12:02.254676  7035 leader_election.cc:343] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:02.250365  7264 consensus_queue.cc:260] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.258821  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6294e276896f402b8d8ea2a0af5dbbe4" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658bfc3ebb1b4380a823078d38152ad5" is_pre_election: true
I20260709 13:12:02.259406  7220 raft_consensus.cc:2468] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 0.
I20260709 13:12:02.259536  7264 ts_tablet_manager.cc:1434] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.001s
I20260709 13:12:02.260809  7264 tablet_bootstrap.cc:492] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be: Bootstrap starting.
I20260709 13:12:02.262089  7036 leader_election.cc:304] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5; no voters: 5aefd9529a6f4747b12913dd7875a6be
I20260709 13:12:02.263598  7274 raft_consensus.cc:2804] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:02.263983  7274 raft_consensus.cc:493] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.264315  7274 raft_consensus.cc:3060] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.274192  7274 raft_consensus.cc:515] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.277494  7274 leader_election.cc:290] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 1 election: Requested vote from peers 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:02.278563  7294 raft_consensus.cc:493] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.279245  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6294e276896f402b8d8ea2a0af5dbbe4" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:02.279026  7294 raft_consensus.cc:515] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.279755  7220 raft_consensus.cc:3060] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.280826  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6294e276896f402b8d8ea2a0af5dbbe4" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:02.281894  7294 leader_election.cc:290] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:02.283062  7266 tablet_bootstrap.cc:654] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:02.284446  7035 leader_election.cc:343] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:02.284577  7070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" is_pre_election: true
I20260709 13:12:02.285071  7221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658bfc3ebb1b4380a823078d38152ad5" is_pre_election: true
I20260709 13:12:02.285338  7070 raft_consensus.cc:2468] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 0.
W20260709 13:12:02.286450  7111 leader_election.cc:343] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:02.294934  7112 leader_election.cc:304] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 5b9678bd83b64815bb6c7123b7d6e0ea; no voters: 658bfc3ebb1b4380a823078d38152ad5
I20260709 13:12:02.295850  7294 raft_consensus.cc:2804] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:02.296231  7294 raft_consensus.cc:493] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.296557  7294 raft_consensus.cc:3060] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.296707  7274 raft_consensus.cc:493] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.298105  7274 raft_consensus.cc:515] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.302446  7221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658bfc3ebb1b4380a823078d38152ad5" is_pre_election: true
I20260709 13:12:02.302996  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be" is_pre_election: true
I20260709 13:12:02.303665  7147 raft_consensus.cc:2468] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 0.
W20260709 13:12:02.304730  7036 leader_election.cc:343] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:02.301108  7274 leader_election.cc:290] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:02.305488  7035 leader_election.cc:304] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 5b9678bd83b64815bb6c7123b7d6e0ea; no voters: 658bfc3ebb1b4380a823078d38152ad5
I20260709 13:12:02.307977  7294 raft_consensus.cc:515] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.310562  7274 raft_consensus.cc:2804] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:02.311031  7274 raft_consensus.cc:493] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.311456  7274 raft_consensus.cc:3060] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.312124  7070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:02.316673  7070 raft_consensus.cc:3060] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.317111  7264 tablet_bootstrap.cc:654] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:02.296522  7220 raft_consensus.cc:2468] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 1.
I20260709 13:12:02.324090  6967 catalog_manager.cc:5697] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be reported cstate change: term changed from 0 to 1, leader changed from <none> to 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66). New cstate: current_term: 1 leader_uuid: "5aefd9529a6f4747b12913dd7875a6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:02.347937  7221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
W20260709 13:12:02.349637  7111 leader_election.cc:343] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:02.368247  7267 tablet_bootstrap.cc:654] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:02.356045  7274 raft_consensus.cc:515] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.370079  7070 raft_consensus.cc:2468] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 1.
I20260709 13:12:02.350975  7294 leader_election.cc:290] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:02.372792  7112 leader_election.cc:304] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 5b9678bd83b64815bb6c7123b7d6e0ea; no voters: 658bfc3ebb1b4380a823078d38152ad5
I20260709 13:12:02.374593  7274 leader_election.cc:290] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 1 election: Requested vote from peers 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:02.375633  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:02.376281  7147 raft_consensus.cc:3060] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.377298  7036 leader_election.cc:304] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5; no voters: 5aefd9529a6f4747b12913dd7875a6be
I20260709 13:12:02.378283  7274 raft_consensus.cc:2804] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:02.379802  6968 catalog_manager.cc:5697] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "658bfc3ebb1b4380a823078d38152ad5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:02.382033  7267 tablet_bootstrap.cc:492] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5: No bootstrap required, opened a new log
I20260709 13:12:02.382494  7267 ts_tablet_manager.cc:1403] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5: Time spent bootstrapping tablet: real 0.129s	user 0.013s	sys 0.011s
I20260709 13:12:02.382792  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:02.383822  7294 raft_consensus.cc:2804] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:02.384317  7294 raft_consensus.cc:697] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Becoming Leader. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
W20260709 13:12:02.384699  7036 leader_election.cc:343] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:02.385092  7294 consensus_queue.cc:237] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.385392  7267 raft_consensus.cc:359] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.386061  7267 raft_consensus.cc:385] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.386430  7274 raft_consensus.cc:697] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 LEADER]: Becoming Leader. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Running, Role: LEADER
I20260709 13:12:02.387261  7274 consensus_queue.cc:237] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.391958  7266 tablet_bootstrap.cc:492] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea: No bootstrap required, opened a new log
I20260709 13:12:02.392426  7266 ts_tablet_manager.cc:1403] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent bootstrapping tablet: real 0.147s	user 0.036s	sys 0.036s
I20260709 13:12:02.393610  7147 raft_consensus.cc:2468] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 1.
I20260709 13:12:02.395175  7035 leader_election.cc:304] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 5b9678bd83b64815bb6c7123b7d6e0ea; no voters: 658bfc3ebb1b4380a823078d38152ad5
I20260709 13:12:02.395314  7266 raft_consensus.cc:359] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.396067  7266 raft_consensus.cc:385] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.396344  7266 raft_consensus.cc:740] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.394817  6968 catalog_manager.cc:5697] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "658bfc3ebb1b4380a823078d38152ad5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:02.397068  7266 consensus_queue.cc:260] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.404104  7314 raft_consensus.cc:2804] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:02.404824  7314 raft_consensus.cc:697] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 LEADER]: Becoming Leader. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Running, Role: LEADER
I20260709 13:12:02.405941  7314 consensus_queue.cc:237] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.398447  7267 raft_consensus.cc:740] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.410319  7267 consensus_queue.cc:260] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.412654  7267 ts_tablet_manager.cc:1434] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.002s
I20260709 13:12:02.411885  7264 tablet_bootstrap.cc:492] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be: No bootstrap required, opened a new log
I20260709 13:12:02.413877  7267 tablet_bootstrap.cc:492] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5: Bootstrap starting.
I20260709 13:12:02.413918  7264 ts_tablet_manager.cc:1403] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be: Time spent bootstrapping tablet: real 0.153s	user 0.017s	sys 0.025s
I20260709 13:12:02.414710  6967 catalog_manager.cc:5697] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65). New cstate: current_term: 1 leader_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:02.417510  7264 raft_consensus.cc:359] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.424002  6966 catalog_manager.cc:5697] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be reported cstate change: term changed from 0 to 1, leader changed from <none> to 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66). New cstate: current_term: 1 leader_uuid: "5aefd9529a6f4747b12913dd7875a6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:02.418219  7264 raft_consensus.cc:385] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.426113  7264 raft_consensus.cc:740] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.426813  7264 consensus_queue.cc:260] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.428507  7264 ts_tablet_manager.cc:1434] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.000s
I20260709 13:12:02.428869  7267 tablet_bootstrap.cc:654] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:02.429296  7264 tablet_bootstrap.cc:492] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be: Bootstrap starting.
I20260709 13:12:02.439872  7267 tablet_bootstrap.cc:492] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5: No bootstrap required, opened a new log
I20260709 13:12:02.440323  7267 ts_tablet_manager.cc:1403] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.006s
I20260709 13:12:02.442883  7267 raft_consensus.cc:359] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.443552  7267 raft_consensus.cc:385] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.443835  7267 raft_consensus.cc:740] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.444468  7267 consensus_queue.cc:260] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.446503  7267 ts_tablet_manager.cc:1434] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:12:02.447540  7267 tablet_bootstrap.cc:492] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5: Bootstrap starting.
I20260709 13:12:02.452951  7267 tablet_bootstrap.cc:654] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:02.456133  7266 ts_tablet_manager.cc:1434] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent starting tablet: real 0.063s	user 0.011s	sys 0.022s
I20260709 13:12:02.457118  7266 tablet_bootstrap.cc:492] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea: Bootstrap starting.
I20260709 13:12:02.459265  7264 tablet_bootstrap.cc:654] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:02.463919  7266 tablet_bootstrap.cc:654] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:02.468156  7267 tablet_bootstrap.cc:492] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5: No bootstrap required, opened a new log
I20260709 13:12:02.468581  7267 ts_tablet_manager.cc:1403] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.008s
I20260709 13:12:02.470283  6966 catalog_manager.cc:5697] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65). New cstate: current_term: 1 leader_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:02.471124  7267 raft_consensus.cc:359] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.471777  7267 raft_consensus.cc:385] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.472038  7267 raft_consensus.cc:740] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.475317  7264 tablet_bootstrap.cc:492] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be: No bootstrap required, opened a new log
I20260709 13:12:02.472805  7267 consensus_queue.cc:260] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.476060  7264 ts_tablet_manager.cc:1403] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be: Time spent bootstrapping tablet: real 0.047s	user 0.020s	sys 0.000s
I20260709 13:12:02.480031  7267 ts_tablet_manager.cc:1434] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:12:02.487113  7292 consensus_queue.cc:1048] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:02.489957  7264 raft_consensus.cc:359] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.490640  7264 raft_consensus.cc:385] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.490903  7264 raft_consensus.cc:740] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.491874  7266 tablet_bootstrap.cc:492] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea: No bootstrap required, opened a new log
I20260709 13:12:02.491544  7264 consensus_queue.cc:260] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.492455  7266 ts_tablet_manager.cc:1403] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent bootstrapping tablet: real 0.036s	user 0.016s	sys 0.018s
I20260709 13:12:02.493570  7264 ts_tablet_manager.cc:1434] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.000s
I20260709 13:12:02.494446  7264 tablet_bootstrap.cc:492] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be: Bootstrap starting.
I20260709 13:12:02.495092  7266 raft_consensus.cc:359] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.495765  7266 raft_consensus.cc:385] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.496062  7266 raft_consensus.cc:740] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.496819  7266 consensus_queue.cc:260] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.499104  7266 ts_tablet_manager.cc:1434] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:12:02.499979  7266 tablet_bootstrap.cc:492] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea: Bootstrap starting.
I20260709 13:12:02.501094  7264 tablet_bootstrap.cc:654] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:02.507292  7266 tablet_bootstrap.cc:654] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:02.509305  7314 raft_consensus.cc:493] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.509755  7314 raft_consensus.cc:515] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.513473  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658bfc3ebb1b4380a823078d38152ad5" is_pre_election: true
I20260709 13:12:02.513896  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be" is_pre_election: true
I20260709 13:12:02.514951  7147 raft_consensus.cc:2393] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b9678bd83b64815bb6c7123b7d6e0ea in current term 1: Already voted for candidate 658bfc3ebb1b4380a823078d38152ad5 in this term.
I20260709 13:12:02.515728  7294 raft_consensus.cc:493] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.516290  7294 raft_consensus.cc:515] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.519980  7035 leader_election.cc:304] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b9678bd83b64815bb6c7123b7d6e0ea; no voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5
I20260709 13:12:02.520047  7071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16cb2fdc74e64e8cbda0c21b3e5081ab" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" is_pre_election: true
I20260709 13:12:02.521319  7314 leader_election.cc:290] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:02.522410  7318 raft_consensus.cc:3060] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.521847  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16cb2fdc74e64e8cbda0c21b3e5081ab" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658bfc3ebb1b4380a823078d38152ad5" is_pre_election: true
I20260709 13:12:02.523418  7266 tablet_bootstrap.cc:492] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea: No bootstrap required, opened a new log
I20260709 13:12:02.523831  7266 ts_tablet_manager.cc:1403] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.007s
I20260709 13:12:02.524237  7111 leader_election.cc:304] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be; no voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5
I20260709 13:12:02.525049  7268 raft_consensus.cc:2749] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:02.521880  7294 leader_election.cc:290] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:02.526695  7287 raft_consensus.cc:493] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.526753  7266 raft_consensus.cc:359] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.527529  7266 raft_consensus.cc:385] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.527261  7287 raft_consensus.cc:515] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.527824  7266 raft_consensus.cc:740] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.529315  7287 leader_election.cc:290] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
I20260709 13:12:02.531158  7266 consensus_queue.cc:260] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.531607  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cba29b3b993144c8b6e088785509ad5b" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be" is_pre_election: true
I20260709 13:12:02.532337  7147 raft_consensus.cc:2468] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 0.
I20260709 13:12:02.533174  7266 ts_tablet_manager.cc:1434] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 13:12:02.533740  7184 leader_election.cc:304] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:02.534698  7287 raft_consensus.cc:2804] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:02.532363  7070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cba29b3b993144c8b6e088785509ad5b" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" is_pre_election: true
I20260709 13:12:02.535105  7287 raft_consensus.cc:493] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.535310  7070 raft_consensus.cc:2468] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 0.
I20260709 13:12:02.535465  7287 raft_consensus.cc:3060] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.538760  7318 raft_consensus.cc:2749] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:02.541349  7287 raft_consensus.cc:515] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.546635  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cba29b3b993144c8b6e088785509ad5b" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:02.546824  7287 leader_election.cc:290] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
I20260709 13:12:02.547202  7147 raft_consensus.cc:3060] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.554975  7146 raft_consensus.cc:3060] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.557626  7264 tablet_bootstrap.cc:492] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be: No bootstrap required, opened a new log
I20260709 13:12:02.558076  7264 ts_tablet_manager.cc:1403] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be: Time spent bootstrapping tablet: real 0.064s	user 0.019s	sys 0.029s
I20260709 13:12:02.559341  7147 raft_consensus.cc:2468] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 1.
I20260709 13:12:02.560016  7070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cba29b3b993144c8b6e088785509ad5b" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:02.560407  7184 leader_election.cc:304] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:02.560613  7070 raft_consensus.cc:3060] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.560720  7264 raft_consensus.cc:359] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.561434  7264 raft_consensus.cc:385] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:02.561718  7264 raft_consensus.cc:740] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Initialized, Role: FOLLOWER
I20260709 13:12:02.562372  7264 consensus_queue.cc:260] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.564374  7264 ts_tablet_manager.cc:1434] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:12:02.566123  7287 raft_consensus.cc:2804] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:02.566685  7287 raft_consensus.cc:697] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Becoming Leader. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Running, Role: LEADER
I20260709 13:12:02.567363  7287 consensus_queue.cc:237] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.570931  7294 raft_consensus.cc:493] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.571353  7294 raft_consensus.cc:515] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.573135  7294 leader_election.cc:290] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:02.575435  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6294e276896f402b8d8ea2a0af5dbbe4" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "658bfc3ebb1b4380a823078d38152ad5" is_pre_election: true
I20260709 13:12:02.575768  7069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6294e276896f402b8d8ea2a0af5dbbe4" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" is_pre_election: true
I20260709 13:12:02.576175  7220 raft_consensus.cc:2393] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5aefd9529a6f4747b12913dd7875a6be in current term 1: Already voted for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in this term.
I20260709 13:12:02.577458  7112 leader_election.cc:304] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be; no voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5
I20260709 13:12:02.578510  7268 raft_consensus.cc:3060] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.579825  7294 consensus_queue.cc:1048] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:02.570452  7070 raft_consensus.cc:2468] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 1.
I20260709 13:12:02.583992  7287 raft_consensus.cc:493] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.583714  6968 catalog_manager.cc:5697] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "658bfc3ebb1b4380a823078d38152ad5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:02.584478  7287 raft_consensus.cc:515] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.587554  7294 consensus_queue.cc:1048] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:02.589282  7302 consensus_queue.cc:1048] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:02.600157  7268 raft_consensus.cc:2749] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:02.603314  7292 consensus_queue.cc:1048] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:02.607007  7069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" is_pre_election: true
I20260709 13:12:02.613075  7292 consensus_queue.cc:1048] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:02.631712  7302 consensus_queue.cc:1048] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:02.632089  7145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be" is_pre_election: true
I20260709 13:12:02.633214  7145 raft_consensus.cc:2393] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 658bfc3ebb1b4380a823078d38152ad5 in current term 1: Already voted for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in this term.
I20260709 13:12:02.634825  7184 leader_election.cc:304] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 658bfc3ebb1b4380a823078d38152ad5; no voters: 5aefd9529a6f4747b12913dd7875a6be, 5b9678bd83b64815bb6c7123b7d6e0ea
I20260709 13:12:02.627070  7287 leader_election.cc:290] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
I20260709 13:12:02.638429  7292 raft_consensus.cc:3060] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.655896  7292 raft_consensus.cc:2749] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:02.673584  7299 consensus_queue.cc:1048] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:12:02.689852  7299 raft_consensus.cc:493] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.690794  7299 raft_consensus.cc:515] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.696524  7299 leader_election.cc:290] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
I20260709 13:12:02.699635  7072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" is_pre_election: true
I20260709 13:12:02.704852  7072 raft_consensus.cc:2393] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 658bfc3ebb1b4380a823078d38152ad5 in current term 1: Already voted for candidate 5aefd9529a6f4747b12913dd7875a6be in this term.
I20260709 13:12:02.706739  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aefd9529a6f4747b12913dd7875a6be" is_pre_election: true
I20260709 13:12:02.721905  7184 leader_election.cc:304] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 658bfc3ebb1b4380a823078d38152ad5; no voters: 5aefd9529a6f4747b12913dd7875a6be, 5b9678bd83b64815bb6c7123b7d6e0ea
I20260709 13:12:02.723023  7292 raft_consensus.cc:3060] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.734776  7299 consensus_queue.cc:1048] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:02.739926  7292 raft_consensus.cc:2749] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:02.753412  7146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15eb2f9468c244b38238d84d56343214"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
mode: GRACEFUL
new_leader_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
 from {username='slave'} at 127.0.0.1:49566
I20260709 13:12:02.754065  7146 raft_consensus.cc:606] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Received request to transfer leadership to TS 5b9678bd83b64815bb6c7123b7d6e0ea
I20260709 13:12:02.757946  7072 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:02.766206  7292 consensus_queue.cc:1048] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:02.785743  7221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db"
dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
mode: GRACEFUL
new_leader_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
 from {username='slave'} at 127.0.0.1:49066
I20260709 13:12:02.786278  7221 raft_consensus.cc:606] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Received request to transfer leadership to TS 5b9678bd83b64815bb6c7123b7d6e0ea
I20260709 13:12:02.790436  7318 consensus_queue.cc:1048] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:02.791837  7146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
mode: GRACEFUL
new_leader_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
 from {username='slave'} at 127.0.0.1:49566
I20260709 13:12:02.792412  7146 raft_consensus.cc:606] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Received request to transfer leadership to TS 5b9678bd83b64815bb6c7123b7d6e0ea
I20260709 13:12:02.797900  7221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808"
dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
mode: GRACEFUL
new_leader_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:49066
I20260709 13:12:02.798394  7221 raft_consensus.cc:606] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Received request to transfer leadership to TS 5aefd9529a6f4747b12913dd7875a6be
I20260709 13:12:02.804726  7221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16cb2fdc74e64e8cbda0c21b3e5081ab"
dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
mode: GRACEFUL
new_leader_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:49066
I20260709 13:12:02.805370  7221 raft_consensus.cc:606] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Received request to transfer leadership to TS 5aefd9529a6f4747b12913dd7875a6be
I20260709 13:12:02.809978  7318 consensus_queue.cc:1048] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:02.812312  7294 consensus_queue.cc:1048] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:02.841991  7071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6294e276896f402b8d8ea2a0af5dbbe4"
dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
mode: GRACEFUL
new_leader_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:44500
I20260709 13:12:02.842619  7071 raft_consensus.cc:606] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 LEADER]: Received request to transfer leadership to TS 5aefd9529a6f4747b12913dd7875a6be
I20260709 13:12:02.847252  7338 raft_consensus.cc:993] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be: : Instructing follower 5b9678bd83b64815bb6c7123b7d6e0ea to start an election
I20260709 13:12:02.848241  7302 raft_consensus.cc:1081] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Signalling peer 5b9678bd83b64815bb6c7123b7d6e0ea to start an election
I20260709 13:12:02.850416  7071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34"
dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
 from {username='slave'} at 127.0.0.1:44492
I20260709 13:12:02.850981  7071 raft_consensus.cc:493] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:02.851349  7071 raft_consensus.cc:3060] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:02.853505  7302 consensus_queue.cc:1048] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:02.859321  7071 raft_consensus.cc:515] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.861567  7071 leader_election.cc:290] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 2 election: Requested vote from peers 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:02.862139  7072 raft_consensus.cc:1240] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Rejecting Update request from peer 5aefd9529a6f4747b12913dd7875a6be for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:02.864624  7221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:02.865185  7221 raft_consensus.cc:3060] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:02.865936  7146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:02.866564  7146 raft_consensus.cc:3055] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:02.866919  7146 raft_consensus.cc:740] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:02.867702  7146 consensus_queue.cc:260] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.868660  7146 raft_consensus.cc:3060] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:02.871809  7221 raft_consensus.cc:2468] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 2.
I20260709 13:12:02.873114  7146 raft_consensus.cc:2468] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 2.
I20260709 13:12:02.873802  7036 leader_election.cc:304] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:02.874904  7274 raft_consensus.cc:2804] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:02.875319  7274 raft_consensus.cc:697] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Becoming Leader. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Running, Role: LEADER
I20260709 13:12:02.876075  7274 consensus_queue.cc:237] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:02.882894  6969 catalog_manager.cc:5697] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea reported cstate change: term changed from 1 to 2, leader changed from 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66) to 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65). New cstate: current_term: 2 leader_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:02.892761  7274 consensus_queue.cc:1048] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:02.903191  7318 consensus_queue.cc:1048] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:02.959913  7292 consensus_queue.cc:1048] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:02.975623  7287 consensus_queue.cc:1048] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:03.070304  7287 raft_consensus.cc:993] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5: : Instructing follower 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:03.070839  7287 raft_consensus.cc:1081] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Signalling peer 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:03.072547  7146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16cb2fdc74e64e8cbda0c21b3e5081ab"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:49544
I20260709 13:12:03.073082  7146 raft_consensus.cc:493] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:03.073432  7146 raft_consensus.cc:3060] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.079777  7146 raft_consensus.cc:515] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.081664  7146 leader_election.cc:290] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 2 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:03.082304  7072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16cb2fdc74e64e8cbda0c21b3e5081ab" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:03.082722  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16cb2fdc74e64e8cbda0c21b3e5081ab" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:03.082945  7072 raft_consensus.cc:3060] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.083308  7220 raft_consensus.cc:3055] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:03.083632  7220 raft_consensus.cc:740] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Running, Role: LEADER
I20260709 13:12:03.084504  7220 consensus_queue.cc:260] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.085667  7220 raft_consensus.cc:3060] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.089387  7072 raft_consensus.cc:2468] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 2.
I20260709 13:12:03.090593  7112 leader_election.cc:304] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 5b9678bd83b64815bb6c7123b7d6e0ea; no voters: 
I20260709 13:12:03.091360  7302 raft_consensus.cc:2804] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:03.091822  7302 raft_consensus.cc:697] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Becoming Leader. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:03.091918  7220 raft_consensus.cc:2468] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 2.
I20260709 13:12:03.092599  7302 consensus_queue.cc:237] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.099063  6966 catalog_manager.cc:5697] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be reported cstate change: term changed from 1 to 2, leader changed from 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67) to 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66). New cstate: current_term: 2 leader_uuid: "5aefd9529a6f4747b12913dd7875a6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:03.161556  7299 raft_consensus.cc:993] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5: : Instructing follower 5b9678bd83b64815bb6c7123b7d6e0ea to start an election
I20260709 13:12:03.161978  7287 raft_consensus.cc:1081] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Signalling peer 5b9678bd83b64815bb6c7123b7d6e0ea to start an election
I20260709 13:12:03.164516  7072 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db"
dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
 from {username='slave'} at 127.0.0.1:44488
I20260709 13:12:03.164985  7072 raft_consensus.cc:493] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:03.165315  7072 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.169581  7072 raft_consensus.cc:515] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.171136  7072 leader_election.cc:290] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 2 election: Requested vote from peers 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:03.171947  7146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:03.172135  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:03.172492  7146 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.172688  7220 raft_consensus.cc:3055] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:03.173087  7220 raft_consensus.cc:740] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Running, Role: LEADER
I20260709 13:12:03.173748  7220 consensus_queue.cc:260] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.174625  7220 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.178534  7220 raft_consensus.cc:2468] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 2.
I20260709 13:12:03.178640  7146 raft_consensus.cc:2468] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 2.
I20260709 13:12:03.179623  7036 leader_election.cc:304] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:03.180544  7318 raft_consensus.cc:2804] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:03.180955  7318 raft_consensus.cc:697] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Becoming Leader. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Running, Role: LEADER
I20260709 13:12:03.181648  7318 consensus_queue.cc:237] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.188966  6966 catalog_manager.cc:5697] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea reported cstate change: term changed from 1 to 2, leader changed from 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67) to 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65). New cstate: current_term: 2 leader_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:03.248971  7343 raft_consensus.cc:993] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea: : Instructing follower 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:03.249442  7318 raft_consensus.cc:1081] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 LEADER]: Signalling peer 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:03.250902  7146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6294e276896f402b8d8ea2a0af5dbbe4"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:49556
I20260709 13:12:03.251358  7146 raft_consensus.cc:493] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:03.251755  7146 raft_consensus.cc:3060] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.256826  7146 raft_consensus.cc:515] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.258780  7146 leader_election.cc:290] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 2 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:03.259112  7072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6294e276896f402b8d8ea2a0af5dbbe4" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:03.259464  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6294e276896f402b8d8ea2a0af5dbbe4" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:03.259667  7072 raft_consensus.cc:3055] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:03.260016  7220 raft_consensus.cc:3060] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.260038  7072 raft_consensus.cc:740] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Running, Role: LEADER
I20260709 13:12:03.260968  7072 consensus_queue.cc:260] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.262125  7072 raft_consensus.cc:3060] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.262660  7338 raft_consensus.cc:993] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be: : Instructing follower 5b9678bd83b64815bb6c7123b7d6e0ea to start an election
I20260709 13:12:03.263063  7302 raft_consensus.cc:1081] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Signalling peer 5b9678bd83b64815bb6c7123b7d6e0ea to start an election
I20260709 13:12:03.264590  7071 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15eb2f9468c244b38238d84d56343214"
dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
 from {username='slave'} at 127.0.0.1:44492
I20260709 13:12:03.265093  7071 raft_consensus.cc:493] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:03.265452  7071 raft_consensus.cc:3060] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.267752  7220 raft_consensus.cc:2468] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 2.
I20260709 13:12:03.268399  7072 raft_consensus.cc:2468] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 2.
I20260709 13:12:03.268832  7111 leader_election.cc:304] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:03.269675  7302 raft_consensus.cc:2804] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:03.270125  7302 raft_consensus.cc:697] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Becoming Leader. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:03.270900  7302 consensus_queue.cc:237] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.272253  7071 raft_consensus.cc:515] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.274581  7287 raft_consensus.cc:993] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5: : Instructing follower 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:03.274852  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15eb2f9468c244b38238d84d56343214" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:03.275025  7287 raft_consensus.cc:1081] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Signalling peer 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:03.275375  7147 raft_consensus.cc:3055] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:03.275707  7147 raft_consensus.cc:740] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:03.276428  7147 consensus_queue.cc:260] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.277417  7147 raft_consensus.cc:3060] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.277995  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15eb2f9468c244b38238d84d56343214" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:03.278597  7220 raft_consensus.cc:3060] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.279516  6966 catalog_manager.cc:5697] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be reported cstate change: term changed from 1 to 2, leader changed from 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65) to 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66). New cstate: current_term: 2 leader_uuid: "5aefd9529a6f4747b12913dd7875a6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:03.280647  7071 leader_election.cc:290] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 2 election: Requested vote from peers 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:03.283737  7147 raft_consensus.cc:2468] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 2.
I20260709 13:12:03.284550  7146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:49544
I20260709 13:12:03.285085  7146 raft_consensus.cc:493] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:03.285430  7146 raft_consensus.cc:3060] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.286233  7220 raft_consensus.cc:2468] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 2.
I20260709 13:12:03.287429  7036 leader_election.cc:304] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:03.288240  7318 raft_consensus.cc:2804] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:03.288691  7318 raft_consensus.cc:697] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Becoming Leader. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Running, Role: LEADER
I20260709 13:12:03.289449  7318 consensus_queue.cc:237] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.291960  7146 raft_consensus.cc:515] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.294687  7071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:03.294979  7221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:03.295238  7071 raft_consensus.cc:3060] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.295543  7221 raft_consensus.cc:3055] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:03.295873  7221 raft_consensus.cc:740] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Running, Role: LEADER
I20260709 13:12:03.296244  7146 leader_election.cc:290] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 2 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:03.296604  7221 consensus_queue.cc:260] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.297818  7221 raft_consensus.cc:3060] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:03.298456  6966 catalog_manager.cc:5697] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea reported cstate change: term changed from 1 to 2, leader changed from 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66) to 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65). New cstate: current_term: 2 leader_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:03.304070  7221 raft_consensus.cc:2468] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 2.
I20260709 13:12:03.304793  7071 raft_consensus.cc:2468] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 2.
I20260709 13:12:03.305284  7111 leader_election.cc:304] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:03.306087  7302 raft_consensus.cc:2804] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:03.306555  7302 raft_consensus.cc:697] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Becoming Leader. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:03.307324  7302 consensus_queue.cc:237] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:03.314126  6966 catalog_manager.cc:5697] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be reported cstate change: term changed from 1 to 2, leader changed from 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67) to 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66). New cstate: current_term: 2 leader_uuid: "5aefd9529a6f4747b12913dd7875a6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:03.408815  7221 raft_consensus.cc:1275] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Refusing update from remote peer 5b9678bd83b64815bb6c7123b7d6e0ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.410238  7318 consensus_queue.cc:1048] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.421273  7146 raft_consensus.cc:1275] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Refusing update from remote peer 5b9678bd83b64815bb6c7123b7d6e0ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.422879  7318 consensus_queue.cc:1048] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.576237  7221 raft_consensus.cc:1275] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.577852  7302 consensus_queue.cc:1048] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.587551  7071 raft_consensus.cc:1275] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.589450  7302 consensus_queue.cc:1048] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.697655  7071 raft_consensus.cc:1275] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.699002  7354 consensus_queue.cc:1048] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.706670  7221 raft_consensus.cc:1275] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.715548  7338 consensus_queue.cc:1048] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.786144  7220 raft_consensus.cc:1275] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Refusing update from remote peer 5b9678bd83b64815bb6c7123b7d6e0ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.787468  7146 raft_consensus.cc:1275] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Refusing update from remote peer 5b9678bd83b64815bb6c7123b7d6e0ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.788183  7318 consensus_queue.cc:1048] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:03.789259  7274 consensus_queue.cc:1048] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:03.795647  7221 raft_consensus.cc:1275] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.797024  7338 consensus_queue.cc:1048] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.808411  7221 raft_consensus.cc:1275] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Refusing update from remote peer 5b9678bd83b64815bb6c7123b7d6e0ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.811188  7071 raft_consensus.cc:1275] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.812706  7302 consensus_queue.cc:1048] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.818104  7274 consensus_queue.cc:1048] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:03.830229  7147 raft_consensus.cc:1275] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Refusing update from remote peer 5b9678bd83b64815bb6c7123b7d6e0ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:03.831777  7343 consensus_queue.cc:1048] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:05.873481  6937 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:12:05.874254  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:05.892802  7147 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
mode: GRACEFUL
new_leader_uuid: "658bfc3ebb1b4380a823078d38152ad5"
 from {username='slave'} at 127.0.0.1:49576
I20260709 13:12:05.893317  7147 raft_consensus.cc:606] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Received request to transfer leadership to TS 658bfc3ebb1b4380a823078d38152ad5
I20260709 13:12:05.903494  7071 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db"
dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
mode: GRACEFUL
new_leader_uuid: "658bfc3ebb1b4380a823078d38152ad5"
 from {username='slave'} at 127.0.0.1:44504
I20260709 13:12:05.904145  7071 raft_consensus.cc:606] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Received request to transfer leadership to TS 658bfc3ebb1b4380a823078d38152ad5
I20260709 13:12:05.905378  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:12:05.905869  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:06.167940  7274 raft_consensus.cc:993] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea: : Instructing follower 658bfc3ebb1b4380a823078d38152ad5 to start an election
I20260709 13:12:06.168375  7371 raft_consensus.cc:1081] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Signalling peer 658bfc3ebb1b4380a823078d38152ad5 to start an election
I20260709 13:12:06.169695  7221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db"
dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
 from {username='slave'} at 127.0.0.1:49054
I20260709 13:12:06.170084  7221 raft_consensus.cc:493] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.170367  7221 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.174471  7221 raft_consensus.cc:515] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:06.175961  7221 leader_election.cc:290] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 3 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
I20260709 13:12:06.176626  7072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:06.176896  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:06.177197  7072 raft_consensus.cc:3055] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:06.177469  7147 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.177546  7072 raft_consensus.cc:740] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Running, Role: LEADER
I20260709 13:12:06.178412  7072 consensus_queue.cc:260] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:06.179569  7072 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.183905  7147 raft_consensus.cc:2468] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 3.
I20260709 13:12:06.185031  7184 leader_election.cc:304] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:06.185505  7072 raft_consensus.cc:2468] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 3.
I20260709 13:12:06.185709  7292 raft_consensus.cc:2804] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:06.187049  7292 raft_consensus.cc:697] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 3 LEADER]: Becoming Leader. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Running, Role: LEADER
I20260709 13:12:06.187784  7292 consensus_queue.cc:237] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:06.194058  6966 catalog_manager.cc:5697] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 reported cstate change: term changed from 2 to 3, leader changed from 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65) to 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67). New cstate: current_term: 3 leader_uuid: "658bfc3ebb1b4380a823078d38152ad5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:06.215212  7370 raft_consensus.cc:993] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be: : Instructing follower 658bfc3ebb1b4380a823078d38152ad5 to start an election
I20260709 13:12:06.215631  7302 raft_consensus.cc:1081] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Signalling peer 658bfc3ebb1b4380a823078d38152ad5 to start an election
I20260709 13:12:06.216886  7221 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808"
dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
 from {username='slave'} at 127.0.0.1:49046
I20260709 13:12:06.217337  7221 raft_consensus.cc:493] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.217640  7221 raft_consensus.cc:3060] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.222227  7221 raft_consensus.cc:515] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:06.223753  7221 leader_election.cc:290] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 3 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
I20260709 13:12:06.224421  7072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:06.224843  7147 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:06.224987  7072 raft_consensus.cc:3060] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.225433  7147 raft_consensus.cc:3055] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:06.225757  7147 raft_consensus.cc:740] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:06.226568  7147 consensus_queue.cc:260] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:06.227690  7147 raft_consensus.cc:3060] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.231336  7072 raft_consensus.cc:2468] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 3.
I20260709 13:12:06.232354  7186 leader_election.cc:304] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:06.233094  7292 raft_consensus.cc:2804] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:06.233497  7292 raft_consensus.cc:697] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 LEADER]: Becoming Leader. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Running, Role: LEADER
I20260709 13:12:06.233912  7147 raft_consensus.cc:2468] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 3.
I20260709 13:12:06.234198  7292 consensus_queue.cc:237] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:06.243799  6969 catalog_manager.cc:5697] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 reported cstate change: term changed from 2 to 3, leader changed from 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66) to 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67). New cstate: current_term: 3 leader_uuid: "658bfc3ebb1b4380a823078d38152ad5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:06.630769  7147 raft_consensus.cc:1275] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Refusing update from remote peer 658bfc3ebb1b4380a823078d38152ad5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:06.631868  7292 consensus_queue.cc:1048] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:06.639982  7072 raft_consensus.cc:1275] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Refusing update from remote peer 658bfc3ebb1b4380a823078d38152ad5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:06.641479  7292 consensus_queue.cc:1048] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:06.673372  7147 raft_consensus.cc:1275] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Refusing update from remote peer 658bfc3ebb1b4380a823078d38152ad5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:06.674619  7292 consensus_queue.cc:1048] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:06.682092  7072 raft_consensus.cc:1275] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Refusing update from remote peer 658bfc3ebb1b4380a823078d38152ad5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:06.683565  7372 consensus_queue.cc:1048] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:06.906646  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:06.915735  7072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15eb2f9468c244b38238d84d56343214"
dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
mode: GRACEFUL
new_leader_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:44500
I20260709 13:12:06.916389  7072 raft_consensus.cc:606] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Received request to transfer leadership to TS 5aefd9529a6f4747b12913dd7875a6be
I20260709 13:12:06.921396  7072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509"
dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
mode: GRACEFUL
new_leader_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:44500
I20260709 13:12:06.921825  7072 raft_consensus.cc:606] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 LEADER]: Received request to transfer leadership to TS 5aefd9529a6f4747b12913dd7875a6be
I20260709 13:12:06.926090  7221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db"
dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
mode: GRACEFUL
new_leader_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:49066
I20260709 13:12:06.926578  7221 raft_consensus.cc:606] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 3 LEADER]: Received request to transfer leadership to TS 5aefd9529a6f4747b12913dd7875a6be
I20260709 13:12:06.931108  7072 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34"
dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
mode: GRACEFUL
new_leader_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:44500
I20260709 13:12:06.931566  7072 raft_consensus.cc:606] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Received request to transfer leadership to TS 5aefd9529a6f4747b12913dd7875a6be
I20260709 13:12:06.935546  7221 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808"
dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
mode: GRACEFUL
new_leader_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:49066
I20260709 13:12:06.935946  7221 raft_consensus.cc:606] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 LEADER]: Received request to transfer leadership to TS 5aefd9529a6f4747b12913dd7875a6be
I20260709 13:12:07.071821  7387 raft_consensus.cc:993] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea: : Instructing follower 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:07.072213  7371 raft_consensus.cc:1081] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Signalling peer 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:07.073426  7147 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:49556
I20260709 13:12:07.073886  7147 raft_consensus.cc:493] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:07.074180  7147 raft_consensus.cc:3060] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:07.078744  7372 raft_consensus.cc:993] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5: : Instructing follower 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:07.079216  7381 raft_consensus.cc:1081] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 LEADER]: Signalling peer 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:07.079598  7147 raft_consensus.cc:515] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.080555  7146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:49544
I20260709 13:12:07.081061  7146 raft_consensus.cc:493] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:07.081470  7146 raft_consensus.cc:3060] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.081619  7147 leader_election.cc:290] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 3 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:07.082118  7072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:07.082690  7072 raft_consensus.cc:3055] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:07.082613  7221 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:07.083060  7072 raft_consensus.cc:740] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Running, Role: LEADER
I20260709 13:12:07.083278  7221 raft_consensus.cc:3060] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:07.083635  7072 consensus_queue.cc:260] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.084304  7072 raft_consensus.cc:3060] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:07.087910  7146 raft_consensus.cc:515] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.089649  7221 raft_consensus.cc:2468] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 3.
I20260709 13:12:07.090242  7072 raft_consensus.cc:2468] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 3.
I20260709 13:12:07.090535  7071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:07.090860  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ff96dd48bc94ab19d8146a755399808" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:07.091123  7071 raft_consensus.cc:3060] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.091306  7112 leader_election.cc:304] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 5b9678bd83b64815bb6c7123b7d6e0ea; no voters: 
I20260709 13:12:07.091385  7220 raft_consensus.cc:3055] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:07.091753  7220 raft_consensus.cc:740] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Running, Role: LEADER
I20260709 13:12:07.091952  7302 raft_consensus.cc:2804] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:07.092543  7146 leader_election.cc:290] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 4 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:07.092563  7220 consensus_queue.cc:260] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.093853  7302 raft_consensus.cc:697] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 3 LEADER]: Becoming Leader. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:07.094209  7220 raft_consensus.cc:3060] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.094676  7302 consensus_queue.cc:237] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.097328  7071 raft_consensus.cc:2468] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 4.
I20260709 13:12:07.098331  7112 leader_election.cc:304] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 5b9678bd83b64815bb6c7123b7d6e0ea; no voters: 
I20260709 13:12:07.099027  7302 raft_consensus.cc:2804] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:07.099468  7302 raft_consensus.cc:697] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 4 LEADER]: Becoming Leader. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:07.100751  7220 raft_consensus.cc:2468] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 4.
I20260709 13:12:07.100214  7302 consensus_queue.cc:237] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.102595  6969 catalog_manager.cc:5697] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be reported cstate change: term changed from 2 to 3, leader changed from 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65) to 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66). New cstate: current_term: 3 leader_uuid: "5aefd9529a6f4747b12913dd7875a6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:07.112597  7387 raft_consensus.cc:993] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea: : Instructing follower 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:07.112999  7371 raft_consensus.cc:1081] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 LEADER]: Signalling peer 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:07.114297  7146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:49556
I20260709 13:12:07.114782  7146 raft_consensus.cc:493] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:07.115070  7146 raft_consensus.cc:3060] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:07.115419  6969 catalog_manager.cc:5697] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be reported cstate change: term changed from 3 to 4, leader changed from 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67) to 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66). New cstate: current_term: 4 leader_uuid: "5aefd9529a6f4747b12913dd7875a6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:07.119452  7146 raft_consensus.cc:515] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.121850  7071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:07.122009  7146 leader_election.cc:290] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 2 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:07.122211  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:07.122419  7071 raft_consensus.cc:3055] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:07.122725  7071 raft_consensus.cc:740] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Running, Role: LEADER
I20260709 13:12:07.122776  7220 raft_consensus.cc:3060] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:07.123564  7071 consensus_queue.cc:260] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.124465  7071 raft_consensus.cc:3060] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:07.126988  7220 raft_consensus.cc:2468] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 2.
I20260709 13:12:07.127880  7111 leader_election.cc:304] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:07.128489  7302 raft_consensus.cc:2804] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:07.128638  7071 raft_consensus.cc:2468] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 2.
I20260709 13:12:07.128983  7302 raft_consensus.cc:697] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Becoming Leader. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:07.129648  7302 consensus_queue.cc:237] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.135960  7372 raft_consensus.cc:993] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5: : Instructing follower 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:07.136379  7381 raft_consensus.cc:1081] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 3 LEADER]: Signalling peer 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:07.136266  6969 catalog_manager.cc:5697] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be reported cstate change: term changed from 1 to 2, leader changed from 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65) to 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66). New cstate: current_term: 2 leader_uuid: "5aefd9529a6f4747b12913dd7875a6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:07.137899  7146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:49544
I20260709 13:12:07.138453  7146 raft_consensus.cc:493] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:07.138880  7146 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.146059  7146 raft_consensus.cc:515] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.147794  7146 leader_election.cc:290] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 4 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:07.148732  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:07.149127  7071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:07.149278  7220 raft_consensus.cc:3055] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:07.149600  7071 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.149610  7220 raft_consensus.cc:740] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Running, Role: LEADER
I20260709 13:12:07.150449  7220 consensus_queue.cc:260] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.151288  7220 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.153838  7071 raft_consensus.cc:2468] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 4.
I20260709 13:12:07.154698  7112 leader_election.cc:304] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 5b9678bd83b64815bb6c7123b7d6e0ea; no voters: 
I20260709 13:12:07.155310  7302 raft_consensus.cc:2804] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:07.155325  7220 raft_consensus.cc:2468] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 4.
I20260709 13:12:07.155848  7302 raft_consensus.cc:697] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 4 LEADER]: Becoming Leader. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:07.156602  7302 consensus_queue.cc:237] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.162107  6969 catalog_manager.cc:5697] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be reported cstate change: term changed from 3 to 4, leader changed from 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67) to 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66). New cstate: current_term: 4 leader_uuid: "5aefd9529a6f4747b12913dd7875a6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:07.162966  7387 raft_consensus.cc:993] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea: : Instructing follower 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:07.163363  7371 raft_consensus.cc:1081] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Signalling peer 5aefd9529a6f4747b12913dd7875a6be to start an election
I20260709 13:12:07.164678  7146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15eb2f9468c244b38238d84d56343214"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
 from {username='slave'} at 127.0.0.1:49556
I20260709 13:12:07.165094  7146 raft_consensus.cc:493] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:07.165385  7146 raft_consensus.cc:3060] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:07.170161  7146 raft_consensus.cc:515] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.171761  7146 leader_election.cc:290] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 3 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:07.172339  7071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15eb2f9468c244b38238d84d56343214" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:07.172734  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15eb2f9468c244b38238d84d56343214" candidate_uuid: "5aefd9529a6f4747b12913dd7875a6be" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:07.172878  7071 raft_consensus.cc:3055] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:07.173687  7220 raft_consensus.cc:3060] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:07.173658  7071 raft_consensus.cc:740] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Running, Role: LEADER
I20260709 13:12:07.174759  7071 consensus_queue.cc:260] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.175967  7071 raft_consensus.cc:3060] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:07.180006  7220 raft_consensus.cc:2468] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 3.
I20260709 13:12:07.181205  7111 leader_election.cc:304] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:07.181783  7071 raft_consensus.cc:2468] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aefd9529a6f4747b12913dd7875a6be in term 3.
I20260709 13:12:07.181842  7388 raft_consensus.cc:2804] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:07.182397  7388 raft_consensus.cc:697] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 3 LEADER]: Becoming Leader. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:07.183195  7388 consensus_queue.cc:237] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:07.188928  6969 catalog_manager.cc:5697] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be reported cstate change: term changed from 2 to 3, leader changed from 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65) to 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66). New cstate: current_term: 3 leader_uuid: "5aefd9529a6f4747b12913dd7875a6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:07.496564  7071 raft_consensus.cc:1275] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 4 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:07.497728  7388 consensus_queue.cc:1048] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:07.505067  7220 raft_consensus.cc:1275] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 4 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:07.508185  7302 consensus_queue.cc:1048] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:07.524993  7071 raft_consensus.cc:1275] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:07.525949  7396 consensus_queue.cc:1048] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:07.534581  7220 raft_consensus.cc:1275] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:07.535311  7221 raft_consensus.cc:1275] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:07.535989  7396 consensus_queue.cc:1048] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:07.536862  7388 consensus_queue.cc:1048] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:07.547904  7071 raft_consensus.cc:1275] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:07.549311  7402 consensus_queue.cc:1048] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:07.593861  7071 raft_consensus.cc:1275] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:07.595054  7402 consensus_queue.cc:1048] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:07.604871  7220 raft_consensus.cc:1275] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:07.606281  7405 consensus_queue.cc:1048] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:07.678398  7220 raft_consensus.cc:1275] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 4 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:07.679589  7405 consensus_queue.cc:1048] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:07.680526  7071 raft_consensus.cc:1275] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 4 FOLLOWER]: Refusing update from remote peer 5aefd9529a6f4747b12913dd7875a6be: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:07.681856  7405 consensus_queue.cc:1048] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:12:09.841876  7000 debug-util.cc:398] Leaking SignalData structure 0x7b0800062060 after lost signal to thread 6940
W20260709 13:12:09.842797  7000 debug-util.cc:398] Leaking SignalData structure 0x7b0800062740 after lost signal to thread 7003
W20260709 13:12:09.843546  7000 debug-util.cc:398] Leaking SignalData structure 0x7b08001ab3a0 after lost signal to thread 7096
W20260709 13:12:09.844302  7000 debug-util.cc:398] Leaking SignalData structure 0x7b08001ebd00 after lost signal to thread 7171
W20260709 13:12:09.844897  7000 debug-util.cc:398] Leaking SignalData structure 0x7b08001f9000 after lost signal to thread 7245
I20260709 13:12:09.973086  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:09.980993  7146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16cb2fdc74e64e8cbda0c21b3e5081ab"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
mode: GRACEFUL
new_leader_uuid: "658bfc3ebb1b4380a823078d38152ad5"
 from {username='slave'} at 127.0.0.1:49576
I20260709 13:12:09.981737  7146 raft_consensus.cc:606] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Received request to transfer leadership to TS 658bfc3ebb1b4380a823078d38152ad5
I20260709 13:12:09.985576  7146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b18aa7629be0450988a1b7bcaacba182"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
mode: GRACEFUL
new_leader_uuid: "658bfc3ebb1b4380a823078d38152ad5"
 from {username='slave'} at 127.0.0.1:49576
I20260709 13:12:09.986163  7146 raft_consensus.cc:606] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Received request to transfer leadership to TS 658bfc3ebb1b4380a823078d38152ad5
I20260709 13:12:09.988468  7146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
mode: GRACEFUL
new_leader_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
 from {username='slave'} at 127.0.0.1:49576
I20260709 13:12:09.988879  7146 raft_consensus.cc:606] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 3 LEADER]: Received request to transfer leadership to TS 5b9678bd83b64815bb6c7123b7d6e0ea
I20260709 13:12:09.991150  7146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
mode: GRACEFUL
new_leader_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
 from {username='slave'} at 127.0.0.1:49576
I20260709 13:12:09.991564  7146 raft_consensus.cc:606] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Received request to transfer leadership to TS 5b9678bd83b64815bb6c7123b7d6e0ea
I20260709 13:12:09.993690  7146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db"
dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
mode: GRACEFUL
new_leader_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
 from {username='slave'} at 127.0.0.1:49576
I20260709 13:12:09.994060  7146 raft_consensus.cc:606] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 4 LEADER]: Received request to transfer leadership to TS 5b9678bd83b64815bb6c7123b7d6e0ea
I20260709 13:12:09.995028  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:12:09.995429  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:10.138566  7411 raft_consensus.cc:993] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be: : Instructing follower 5b9678bd83b64815bb6c7123b7d6e0ea to start an election
I20260709 13:12:10.139163  7408 raft_consensus.cc:1081] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Signalling peer 5b9678bd83b64815bb6c7123b7d6e0ea to start an election
I20260709 13:12:10.140985  7070 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509"
dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
 from {username='slave'} at 127.0.0.1:44492
I20260709 13:12:10.141665  7070 raft_consensus.cc:493] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:10.142176  7070 raft_consensus.cc:3060] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:10.148051  7070 raft_consensus.cc:515] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.150489  7070 leader_election.cc:290] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 3 election: Requested vote from peers 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:10.150699  7146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:10.151235  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e001b76d430e47e9a4bc37528a3b0509" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:10.151995  7146 raft_consensus.cc:3055] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:10.152042  7220 raft_consensus.cc:3060] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:10.152310  7146 raft_consensus.cc:740] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:10.153148  7146 consensus_queue.cc:260] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.154055  7146 raft_consensus.cc:3060] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:10.157909  7146 raft_consensus.cc:2468] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 3.
I20260709 13:12:10.159019  7035 leader_election.cc:304] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aefd9529a6f4747b12913dd7875a6be, 5b9678bd83b64815bb6c7123b7d6e0ea; no voters: 
I20260709 13:12:10.159034  7220 raft_consensus.cc:2468] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 3.
I20260709 13:12:10.160686  7414 raft_consensus.cc:2804] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:10.161748  7414 raft_consensus.cc:697] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 LEADER]: Becoming Leader. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Running, Role: LEADER
I20260709 13:12:10.162752  7414 consensus_queue.cc:237] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.174319  6969 catalog_manager.cc:5697] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea reported cstate change: term changed from 2 to 3, leader changed from 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66) to 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65). New cstate: current_term: 3 leader_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:10.181891  7408 raft_consensus.cc:993] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be: : Instructing follower 658bfc3ebb1b4380a823078d38152ad5 to start an election
I20260709 13:12:10.182435  7411 raft_consensus.cc:1081] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Signalling peer 658bfc3ebb1b4380a823078d38152ad5 to start an election
I20260709 13:12:10.184345  7220 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16cb2fdc74e64e8cbda0c21b3e5081ab"
dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
 from {username='slave'} at 127.0.0.1:49046
I20260709 13:12:10.184880  7220 raft_consensus.cc:493] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:10.185230  7220 raft_consensus.cc:3060] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:10.192307  7220 raft_consensus.cc:515] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.194936  7070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16cb2fdc74e64e8cbda0c21b3e5081ab" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:10.195223  7146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16cb2fdc74e64e8cbda0c21b3e5081ab" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:10.195505  7070 raft_consensus.cc:3060] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:10.195824  7146 raft_consensus.cc:3055] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:10.196132  7146 raft_consensus.cc:740] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:10.196839  7146 consensus_queue.cc:260] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.197890  7220 leader_election.cc:290] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 3 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
I20260709 13:12:10.197881  7146 raft_consensus.cc:3060] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:10.201020  7070 raft_consensus.cc:2468] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 3.
I20260709 13:12:10.202131  7186 leader_election.cc:304] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:10.202924  7410 raft_consensus.cc:2804] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:10.204165  7410 raft_consensus.cc:697] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 3 LEADER]: Becoming Leader. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Running, Role: LEADER
I20260709 13:12:10.204752  7146 raft_consensus.cc:2468] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 3.
I20260709 13:12:10.204972  7410 consensus_queue.cc:237] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.210686  7408 raft_consensus.cc:993] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be: : Instructing follower 5b9678bd83b64815bb6c7123b7d6e0ea to start an election
I20260709 13:12:10.211130  7411 raft_consensus.cc:1081] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 4 LEADER]: Signalling peer 5b9678bd83b64815bb6c7123b7d6e0ea to start an election
I20260709 13:12:10.212499  7070 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db"
dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
 from {username='slave'} at 127.0.0.1:44492
I20260709 13:12:10.212929  7070 raft_consensus.cc:493] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:10.213275  7070 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:10.214941  6969 catalog_manager.cc:5697] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 reported cstate change: term changed from 2 to 3, leader changed from 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66) to 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67). New cstate: current_term: 3 leader_uuid: "658bfc3ebb1b4380a823078d38152ad5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:10.219380  7070 raft_consensus.cc:515] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.222077  7070 leader_election.cc:290] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 5 election: Requested vote from peers 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:10.222085  7146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:10.223520  7146 raft_consensus.cc:3055] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:10.223804  7146 raft_consensus.cc:740] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:10.224346  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1d6d86807c34fe0b76ef5ea2d2e25db" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:10.224826  7220 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:10.224524  7146 consensus_queue.cc:260] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.225658  7146 raft_consensus.cc:3060] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:10.228500  7220 raft_consensus.cc:2468] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 5.
I20260709 13:12:10.229409  7036 leader_election.cc:304] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:10.230059  7414 raft_consensus.cc:2804] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:10.230490  7414 raft_consensus.cc:697] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 5 LEADER]: Becoming Leader. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Running, Role: LEADER
I20260709 13:12:10.231192  7414 consensus_queue.cc:237] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.231894  7146 raft_consensus.cc:2468] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 5.
I20260709 13:12:10.237774  6969 catalog_manager.cc:5697] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea reported cstate change: term changed from 4 to 5, leader changed from 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66) to 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65). New cstate: current_term: 5 leader_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:10.290601  7408 raft_consensus.cc:993] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be: : Instructing follower 658bfc3ebb1b4380a823078d38152ad5 to start an election
I20260709 13:12:10.290998  7411 raft_consensus.cc:1081] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Signalling peer 658bfc3ebb1b4380a823078d38152ad5 to start an election
I20260709 13:12:10.292572  7220 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b18aa7629be0450988a1b7bcaacba182"
dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
 from {username='slave'} at 127.0.0.1:49046
I20260709 13:12:10.292975  7220 raft_consensus.cc:493] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:10.293241  7220 raft_consensus.cc:3060] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:10.296965  7220 raft_consensus.cc:515] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.298425  7220 leader_election.cc:290] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 2 election: Requested vote from peers 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219), 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387)
I20260709 13:12:10.299073  7070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18aa7629be0450988a1b7bcaacba182" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
I20260709 13:12:10.299454  7146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18aa7629be0450988a1b7bcaacba182" candidate_uuid: "658bfc3ebb1b4380a823078d38152ad5" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:10.299652  7070 raft_consensus.cc:3060] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:10.300009  7146 raft_consensus.cc:3055] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:10.300321  7146 raft_consensus.cc:740] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:10.301043  7146 consensus_queue.cc:260] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.302148  7146 raft_consensus.cc:3060] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:10.304368  7070 raft_consensus.cc:2468] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 2.
I20260709 13:12:10.305308  7186 leader_election.cc:304] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:10.305972  7410 raft_consensus.cc:2804] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:10.306394  7410 raft_consensus.cc:697] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 LEADER]: Becoming Leader. State: Replica: 658bfc3ebb1b4380a823078d38152ad5, State: Running, Role: LEADER
I20260709 13:12:10.307193  7410 consensus_queue.cc:237] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.308540  7146 raft_consensus.cc:2468] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658bfc3ebb1b4380a823078d38152ad5 in term 2.
I20260709 13:12:10.314509  6969 catalog_manager.cc:5697] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 reported cstate change: term changed from 1 to 2, leader changed from 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66) to 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67). New cstate: current_term: 2 leader_uuid: "658bfc3ebb1b4380a823078d38152ad5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:10.430292  7408 raft_consensus.cc:993] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be: : Instructing follower 5b9678bd83b64815bb6c7123b7d6e0ea to start an election
I20260709 13:12:10.430784  7411 raft_consensus.cc:1081] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 3 LEADER]: Signalling peer 5b9678bd83b64815bb6c7123b7d6e0ea to start an election
I20260709 13:12:10.432332  7070 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34"
dest_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea"
 from {username='slave'} at 127.0.0.1:44492
I20260709 13:12:10.432900  7070 raft_consensus.cc:493] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:10.433260  7070 raft_consensus.cc:3060] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:10.440407  7070 raft_consensus.cc:515] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.443006  7070 leader_election.cc:290] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 4 election: Requested vote from peers 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387), 658bfc3ebb1b4380a823078d38152ad5 (127.6.198.67:38755)
I20260709 13:12:10.443159  7146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5aefd9529a6f4747b12913dd7875a6be"
I20260709 13:12:10.443601  7220 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dce58ff3844546a5a97a84b771250c34" candidate_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "658bfc3ebb1b4380a823078d38152ad5"
I20260709 13:12:10.443770  7146 raft_consensus.cc:3055] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:10.444125  7220 raft_consensus.cc:3060] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:10.444151  7146 raft_consensus.cc:740] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5aefd9529a6f4747b12913dd7875a6be, State: Running, Role: LEADER
I20260709 13:12:10.445117  7146 consensus_queue.cc:260] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.446180  7146 raft_consensus.cc:3060] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:10.449904  7220 raft_consensus.cc:2468] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 4.
I20260709 13:12:10.450842  7036 leader_election.cc:304] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b9678bd83b64815bb6c7123b7d6e0ea, 658bfc3ebb1b4380a823078d38152ad5; no voters: 
I20260709 13:12:10.451606  7414 raft_consensus.cc:2804] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:10.452077  7414 raft_consensus.cc:697] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 4 LEADER]: Becoming Leader. State: Replica: 5b9678bd83b64815bb6c7123b7d6e0ea, State: Running, Role: LEADER
I20260709 13:12:10.451946  7146 raft_consensus.cc:2468] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b9678bd83b64815bb6c7123b7d6e0ea in term 4.
I20260709 13:12:10.452987  7414 consensus_queue.cc:237] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } }
I20260709 13:12:10.461115  6969 catalog_manager.cc:5697] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea reported cstate change: term changed from 3 to 4, leader changed from 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66) to 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65). New cstate: current_term: 4 leader_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:10.664264  7146 raft_consensus.cc:1275] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Refusing update from remote peer 5b9678bd83b64815bb6c7123b7d6e0ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:10.665464  7414 consensus_queue.cc:1048] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:10.679322  7220 raft_consensus.cc:1275] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Refusing update from remote peer 5b9678bd83b64815bb6c7123b7d6e0ea: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:10.680708  7414 consensus_queue.cc:1048] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:10.748406  7070 raft_consensus.cc:1275] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Refusing update from remote peer 658bfc3ebb1b4380a823078d38152ad5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:10.749594  7410 consensus_queue.cc:1048] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:10.758031  7146 raft_consensus.cc:1275] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Refusing update from remote peer 658bfc3ebb1b4380a823078d38152ad5: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:10.759549  7410 consensus_queue.cc:1048] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:10.785765  7146 raft_consensus.cc:1275] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Refusing update from remote peer 658bfc3ebb1b4380a823078d38152ad5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.786945  7432 consensus_queue.cc:1048] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:10.797236  7070 raft_consensus.cc:1275] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Refusing update from remote peer 658bfc3ebb1b4380a823078d38152ad5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.799507  7432 consensus_queue.cc:1048] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b9678bd83b64815bb6c7123b7d6e0ea" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:10.803978  7146 raft_consensus.cc:1275] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 5 FOLLOWER]: Refusing update from remote peer 5b9678bd83b64815bb6c7123b7d6e0ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:12:10.805688  7414 consensus_queue.cc:1048] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:10.816572  7220 raft_consensus.cc:1275] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 5 FOLLOWER]: Refusing update from remote peer 5b9678bd83b64815bb6c7123b7d6e0ea: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:12:10.818225  7415 consensus_queue.cc:1048] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:12:10.945804  7146 raft_consensus.cc:1275] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 4 FOLLOWER]: Refusing update from remote peer 5b9678bd83b64815bb6c7123b7d6e0ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:10.946950  7414 consensus_queue.cc:1048] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aefd9529a6f4747b12913dd7875a6be" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:10.956900  7220 raft_consensus.cc:1275] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 4 FOLLOWER]: Refusing update from remote peer 5b9678bd83b64815bb6c7123b7d6e0ea: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:10.958448  7437 consensus_queue.cc:1048] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "658bfc3ebb1b4380a823078d38152ad5" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 38755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:10.996177  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:11.001425  6937 tablet_server.cc:179] TabletServer@127.6.198.65:0 shutting down...
W20260709 13:12:11.017087  7112 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:11.020761  7112 consensus_peers.cc:597] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be -> Peer 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219): Couldn't send request to peer 5b9678bd83b64815bb6c7123b7d6e0ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:11.038874  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:11.039552  6937 tablet_replica.cc:333] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea: stopping tablet replica
I20260709 13:12:11.040203  6937 raft_consensus.cc:2243] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.040791  6937 raft_consensus.cc:2272] T 9ff96dd48bc94ab19d8146a755399808 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.043198  6937 tablet_replica.cc:333] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea: stopping tablet replica
I20260709 13:12:11.043716  6937 raft_consensus.cc:2243] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:12:11.044397  6937 raft_consensus.cc:2272] T dce58ff3844546a5a97a84b771250c34 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.046248  6937 tablet_replica.cc:333] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea: stopping tablet replica
I20260709 13:12:11.046816  6937 raft_consensus.cc:2243] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:12:11.047458  6937 raft_consensus.cc:2272] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5b9678bd83b64815bb6c7123b7d6e0ea [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.049676  6937 tablet_replica.cc:333] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea: stopping tablet replica
I20260709 13:12:11.050189  6937 raft_consensus.cc:2243] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.050670  6937 raft_consensus.cc:2272] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.052345  6937 tablet_replica.cc:333] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea: stopping tablet replica
I20260709 13:12:11.052806  6937 raft_consensus.cc:2243] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:11.053443  6937 raft_consensus.cc:2272] T e001b76d430e47e9a4bc37528a3b0509 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.055446  6937 tablet_replica.cc:333] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea: stopping tablet replica
I20260709 13:12:11.055928  6937 raft_consensus.cc:2243] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.056311  6937 raft_consensus.cc:2272] T b18aa7629be0450988a1b7bcaacba182 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.058228  6937 tablet_replica.cc:333] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea: stopping tablet replica
I20260709 13:12:11.058779  6937 raft_consensus.cc:2243] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.059372  6937 raft_consensus.cc:2272] T 15eb2f9468c244b38238d84d56343214 P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.061580  6937 tablet_replica.cc:333] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea: stopping tablet replica
I20260709 13:12:11.062068  6937 raft_consensus.cc:2243] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.062480  6937 raft_consensus.cc:2272] T cba29b3b993144c8b6e088785509ad5b P 5b9678bd83b64815bb6c7123b7d6e0ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.064008  6937 tablet_replica.cc:333] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea: stopping tablet replica
I20260709 13:12:11.064428  6937 raft_consensus.cc:2243] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.064798  6937 raft_consensus.cc:2272] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5b9678bd83b64815bb6c7123b7d6e0ea [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.162441  6937 tablet_server.cc:196] TabletServer@127.6.198.65:0 shutdown complete.
I20260709 13:12:11.182426  6937 tablet_server.cc:179] TabletServer@127.6.198.66:0 shutting down...
I20260709 13:12:11.209136  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:11.209802  6937 tablet_replica.cc:333] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be: stopping tablet replica
I20260709 13:12:11.210389  6937 raft_consensus.cc:2243] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.210831  6937 raft_consensus.cc:2272] T cba29b3b993144c8b6e088785509ad5b P 5aefd9529a6f4747b12913dd7875a6be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.212536  6937 tablet_replica.cc:333] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be: stopping tablet replica
I20260709 13:12:11.213042  6937 raft_consensus.cc:2243] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.213415  6937 raft_consensus.cc:2272] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.215445  6937 tablet_replica.cc:333] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be: stopping tablet replica
I20260709 13:12:11.215957  6937 raft_consensus.cc:2243] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.216333  6937 raft_consensus.cc:2272] T dce58ff3844546a5a97a84b771250c34 P 5aefd9529a6f4747b12913dd7875a6be [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.218405  6937 tablet_replica.cc:333] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be: stopping tablet replica
I20260709 13:12:11.219168  6937 raft_consensus.cc:2243] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:12:11.220355  6937 raft_consensus.cc:2272] T 9ff96dd48bc94ab19d8146a755399808 P 5aefd9529a6f4747b12913dd7875a6be [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.223246  6937 tablet_replica.cc:333] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be: stopping tablet replica
I20260709 13:12:11.224014  6937 raft_consensus.cc:2243] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:11.224725  6937 raft_consensus.cc:2272] T 6294e276896f402b8d8ea2a0af5dbbe4 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.226604  6937 tablet_replica.cc:333] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be: stopping tablet replica
I20260709 13:12:11.227244  6937 raft_consensus.cc:2243] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.227700  6937 raft_consensus.cc:2272] T f1d6d86807c34fe0b76ef5ea2d2e25db P 5aefd9529a6f4747b12913dd7875a6be [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.229691  6937 tablet_replica.cc:333] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be: stopping tablet replica
I20260709 13:12:11.230118  6937 raft_consensus.cc:2243] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:11.230773  6937 raft_consensus.cc:2272] T 15eb2f9468c244b38238d84d56343214 P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.232401  6937 tablet_replica.cc:333] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be: stopping tablet replica
I20260709 13:12:11.232831  6937 raft_consensus.cc:2243] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.233182  6937 raft_consensus.cc:2272] T b18aa7629be0450988a1b7bcaacba182 P 5aefd9529a6f4747b12913dd7875a6be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.235042  6937 tablet_replica.cc:333] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be: stopping tablet replica
I20260709 13:12:11.235472  6937 raft_consensus.cc:2243] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.235826  6937 raft_consensus.cc:2272] T e001b76d430e47e9a4bc37528a3b0509 P 5aefd9529a6f4747b12913dd7875a6be [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.265319  6937 tablet_server.cc:196] TabletServer@127.6.198.66:0 shutdown complete.
W20260709 13:12:11.277498  7184 consensus_peers.cc:597] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 -> Peer 5aefd9529a6f4747b12913dd7875a6be (127.6.198.66:33387): Couldn't send request to peer 5aefd9529a6f4747b12913dd7875a6be. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:33387: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:11.294271  6937 tablet_server.cc:179] TabletServer@127.6.198.67:0 shutting down...
I20260709 13:12:11.319995  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:11.320767  6937 tablet_replica.cc:333] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5: stopping tablet replica
I20260709 13:12:11.321578  6937 raft_consensus.cc:2243] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.322122  6937 raft_consensus.cc:2272] T e001b76d430e47e9a4bc37528a3b0509 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.324826  6937 tablet_replica.cc:333] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5: stopping tablet replica
I20260709 13:12:11.325575  6937 raft_consensus.cc:2243] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.326040  6937 raft_consensus.cc:2272] T dce58ff3844546a5a97a84b771250c34 P 658bfc3ebb1b4380a823078d38152ad5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.328680  6937 tablet_replica.cc:333] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5: stopping tablet replica
I20260709 13:12:11.329406  6937 raft_consensus.cc:2243] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:11.330371  6937 raft_consensus.cc:2272] T b18aa7629be0450988a1b7bcaacba182 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.332876  6937 tablet_replica.cc:333] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5: stopping tablet replica
I20260709 13:12:11.333627  6937 raft_consensus.cc:2243] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.334129  6937 raft_consensus.cc:2272] T 6294e276896f402b8d8ea2a0af5dbbe4 P 658bfc3ebb1b4380a823078d38152ad5 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.336593  6937 tablet_replica.cc:333] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5: stopping tablet replica
I20260709 13:12:11.337298  6937 raft_consensus.cc:2243] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.337826  6937 raft_consensus.cc:2272] T f1d6d86807c34fe0b76ef5ea2d2e25db P 658bfc3ebb1b4380a823078d38152ad5 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.340366  6937 tablet_replica.cc:333] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5: stopping tablet replica
I20260709 13:12:11.341044  6937 raft_consensus.cc:2243] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.341555  6937 raft_consensus.cc:2272] T 9ff96dd48bc94ab19d8146a755399808 P 658bfc3ebb1b4380a823078d38152ad5 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.343813  6937 tablet_replica.cc:333] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5: stopping tablet replica
I20260709 13:12:11.344494  6937 raft_consensus.cc:2243] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.345008  6937 raft_consensus.cc:2272] T 15eb2f9468c244b38238d84d56343214 P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.347157  6937 tablet_replica.cc:333] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5: stopping tablet replica
I20260709 13:12:11.347785  6937 raft_consensus.cc:2243] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:11.348699  6937 raft_consensus.cc:2272] T cba29b3b993144c8b6e088785509ad5b P 658bfc3ebb1b4380a823078d38152ad5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.351233  6937 tablet_replica.cc:333] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5: stopping tablet replica
W20260709 13:12:11.351465  7186 consensus_peers.cc:597] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 -> Peer 5b9678bd83b64815bb6c7123b7d6e0ea (127.6.198.65:45219): Couldn't send request to peer 5b9678bd83b64815bb6c7123b7d6e0ea. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45219: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:11.351861  6937 raft_consensus.cc:2243] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:11.352777  6937 raft_consensus.cc:2272] T 16cb2fdc74e64e8cbda0c21b3e5081ab P 658bfc3ebb1b4380a823078d38152ad5 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.384092  6937 tablet_server.cc:196] TabletServer@127.6.198.67:0 shutdown complete.
I20260709 13:12:11.403012  6937 master.cc:562] Master@127.6.198.126:36505 shutting down...
I20260709 13:12:11.530989  6937 raft_consensus.cc:2243] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:11.531685  6937 raft_consensus.cc:2272] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.532109  6937 tablet_replica.cc:333] T 00000000000000000000000000000000 P dc559a6efa91405d8c8d255ceb7f8498: stopping tablet replica
I20260709 13:12:12.026662  6937 master.cc:584] Master@127.6.198.126:36505 shutdown complete.
5aefd9529a6f4747b12913dd7875a6be  4
5b9678bd83b64815bb6c7123b7d6e0ea  4
658bfc3ebb1b4380a823078d38152ad5  1
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12917 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:12:12.068311  6937 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.198.126:45895
I20260709 13:12:12.069361  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:12.074738  7445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:12.074971  7446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:12.075858  7448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:12.079865  6937 server_base.cc:1061] running on GCE node
I20260709 13:12:12.080654  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:12.080827  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:12.080999  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602732080985 us; error 0 us; skew 500 ppm
I20260709 13:12:12.081538  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:12.084074  6937 webserver.cc:533] Webserver started at http://127.6.198.126:39783/ using document root <none> and password file <none>
I20260709 13:12:12.084524  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:12.084702  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:12.084975  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:12.086000  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/master-0-root/instance:
uuid: "acf6aea4f76643a385cf939478cb7631"
format_stamp: "Formatted at 2026-07-09 13:12:12 on dist-test-slave-tqzw"
I20260709 13:12:12.089905  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:12:12.092864  7453 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:12.093497  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:12:12.093760  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/master-0-root
uuid: "acf6aea4f76643a385cf939478cb7631"
format_stamp: "Formatted at 2026-07-09 13:12:12 on dist-test-slave-tqzw"
I20260709 13:12:12.094022  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:12.124908  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:12.126077  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:12.162863  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.126:45895
I20260709 13:12:12.162976  7504 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.126:45895 every 8 connection(s)
I20260709 13:12:12.166455  7505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.175024  7505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631: Bootstrap starting.
I20260709 13:12:12.179411  7505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:12.183120  7505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631: No bootstrap required, opened a new log
I20260709 13:12:12.184971  7505 raft_consensus.cc:359] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf6aea4f76643a385cf939478cb7631" member_type: VOTER }
I20260709 13:12:12.185308  7505 raft_consensus.cc:385] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:12.185489  7505 raft_consensus.cc:740] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf6aea4f76643a385cf939478cb7631, State: Initialized, Role: FOLLOWER
I20260709 13:12:12.186022  7505 consensus_queue.cc:260] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf6aea4f76643a385cf939478cb7631" member_type: VOTER }
I20260709 13:12:12.186475  7505 raft_consensus.cc:399] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:12:12.186667  7505 raft_consensus.cc:493] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:12:12.186864  7505 raft_consensus.cc:3060] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:12.190413  7505 raft_consensus.cc:515] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf6aea4f76643a385cf939478cb7631" member_type: VOTER }
I20260709 13:12:12.190830  7505 leader_election.cc:304] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acf6aea4f76643a385cf939478cb7631; no voters: 
I20260709 13:12:12.191823  7505 leader_election.cc:290] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:12:12.192193  7508 raft_consensus.cc:2804] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:12.193497  7508 raft_consensus.cc:697] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [term 1 LEADER]: Becoming Leader. State: Replica: acf6aea4f76643a385cf939478cb7631, State: Running, Role: LEADER
I20260709 13:12:12.194046  7508 consensus_queue.cc:237] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf6aea4f76643a385cf939478cb7631" member_type: VOTER }
I20260709 13:12:12.194662  7505 sys_catalog.cc:565] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:12:12.198489  7509 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acf6aea4f76643a385cf939478cb7631" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf6aea4f76643a385cf939478cb7631" member_type: VOTER } }
I20260709 13:12:12.198988  7510 sys_catalog.cc:455] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acf6aea4f76643a385cf939478cb7631. Latest consensus state: current_term: 1 leader_uuid: "acf6aea4f76643a385cf939478cb7631" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acf6aea4f76643a385cf939478cb7631" member_type: VOTER } }
I20260709 13:12:12.199261  7509 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:12.199584  7510 sys_catalog.cc:458] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:12.201360  7515 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:12:12.206198  7515 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:12:12.209627  6937 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:12:12.214200  7515 catalog_manager.cc:1383] Generated new cluster ID: 2f0620c29a464a7ba6e1570e85c8402a
I20260709 13:12:12.214511  7515 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:12:12.236634  7515 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:12:12.237751  7515 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:12:12.247198  7515 catalog_manager.cc:6070] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631: Generated new TSK 0
I20260709 13:12:12.247707  7515 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:12:12.275784  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:12.281302  7526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:12.282542  7527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:12.284690  7529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:12.284744  6937 server_base.cc:1061] running on GCE node
I20260709 13:12:12.285723  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:12.285930  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:12.286101  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602732286085 us; error 0 us; skew 500 ppm
I20260709 13:12:12.286669  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:12.288825  6937 webserver.cc:533] Webserver started at http://127.6.198.65:46825/ using document root <none> and password file <none>
I20260709 13:12:12.289294  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:12.289501  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:12.289753  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:12.290797  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-0-root/instance:
uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
format_stamp: "Formatted at 2026-07-09 13:12:12 on dist-test-slave-tqzw"
I20260709 13:12:12.294793  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:12:12.297920  7534 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:12.298642  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:12:12.298923  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-0-root
uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
format_stamp: "Formatted at 2026-07-09 13:12:12 on dist-test-slave-tqzw"
I20260709 13:12:12.299201  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:12.313184  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:12.314429  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:12.315758  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:12.317860  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:12.318079  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:12.318310  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:12.318560  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:12:12.357887  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.65:36351
I20260709 13:12:12.357990  7596 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.65:36351 every 8 connection(s)
I20260709 13:12:12.362684  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:12.372433  7601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:12.374285  7602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:12.381645  7597 heartbeater.cc:344] Connected to a master server at 127.6.198.126:45895
I20260709 13:12:12.382046  7597 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:12.383160  7597 heartbeater.cc:507] Master 127.6.198.126:45895 requested a full tablet report, sending...
W20260709 13:12:12.384099  7604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:12.386416  7470 ts_manager.cc:194] Registered new tserver with Master: 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351)
I20260709 13:12:12.386579  6937 server_base.cc:1061] running on GCE node
I20260709 13:12:12.387567  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:12.387768  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:12.388072  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602732388046 us; error 0 us; skew 500 ppm
I20260709 13:12:12.388918  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:12.389504  7470 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54958
I20260709 13:12:12.392225  6937 webserver.cc:533] Webserver started at http://127.6.198.66:35175/ using document root <none> and password file <none>
I20260709 13:12:12.392979  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:12.393301  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:12.393679  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:12.395519  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-1-root/instance:
uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
format_stamp: "Formatted at 2026-07-09 13:12:12 on dist-test-slave-tqzw"
I20260709 13:12:12.402558  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.002s	sys 0.004s
I20260709 13:12:12.406772  7609 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:12.407788  6937 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.004s
I20260709 13:12:12.408190  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-1-root
uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
format_stamp: "Formatted at 2026-07-09 13:12:12 on dist-test-slave-tqzw"
I20260709 13:12:12.408576  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:12.432346  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:12.433372  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:12.434728  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:12.436780  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:12.437008  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:12.437233  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:12.437397  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:12.477694  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.66:38653
I20260709 13:12:12.477921  7671 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.66:38653 every 8 connection(s)
I20260709 13:12:12.482429  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:12.488867  7675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:12.490804  7676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:12.494189  6937 server_base.cc:1061] running on GCE node
I20260709 13:12:12.494881  7672 heartbeater.cc:344] Connected to a master server at 127.6.198.126:45895
I20260709 13:12:12.495292  7672 heartbeater.cc:461] Registering TS with master...
W20260709 13:12:12.495906  7678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:12.496219  7672 heartbeater.cc:507] Master 127.6.198.126:45895 requested a full tablet report, sending...
I20260709 13:12:12.497346  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:12.497592  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:12.497783  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602732497765 us; error 0 us; skew 500 ppm
I20260709 13:12:12.498121  7470 ts_manager.cc:194] Registered new tserver with Master: 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653)
I20260709 13:12:12.498378  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:12.499778  7470 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54972
I20260709 13:12:12.501127  6937 webserver.cc:533] Webserver started at http://127.6.198.67:34869/ using document root <none> and password file <none>
I20260709 13:12:12.501777  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:12.502046  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:12.502398  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:12.503430  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-2-root/instance:
uuid: "19f3607054c14bd88d20be41d80b9b3b"
format_stamp: "Formatted at 2026-07-09 13:12:12 on dist-test-slave-tqzw"
I20260709 13:12:12.507439  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:12:12.510820  7683 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:12.511566  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:12:12.511854  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-2-root
uuid: "19f3607054c14bd88d20be41d80b9b3b"
format_stamp: "Formatted at 2026-07-09 13:12:12 on dist-test-slave-tqzw"
I20260709 13:12:12.512108  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602718985547-6937-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:12.523384  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:12.524453  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:12.525769  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:12.527822  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:12.528015  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:12.528265  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:12.528430  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:12.566709  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.67:33409
I20260709 13:12:12.566823  7745 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.67:33409 every 8 connection(s)
I20260709 13:12:12.579520  7746 heartbeater.cc:344] Connected to a master server at 127.6.198.126:45895
I20260709 13:12:12.579807  7746 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:12.580416  7746 heartbeater.cc:507] Master 127.6.198.126:45895 requested a full tablet report, sending...
I20260709 13:12:12.582170  7470 ts_manager.cc:194] Registered new tserver with Master: 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:12.582996  6937 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013203099s
I20260709 13:12:12.583441  7470 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54986
I20260709 13:12:12.605739  7470 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54992:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:12:12.608477  7470 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:12:12.676541  7709 tablet_service.cc:1511] Processing CreateTablet for tablet 62ed00cf429347c9b053918ef1f89678 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:12:12.676751  7708 tablet_service.cc:1511] Processing CreateTablet for tablet 47f8fad043dd499caf70f67c2707d4cb (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:12:12.678018  7709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62ed00cf429347c9b053918ef1f89678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.679438  7710 tablet_service.cc:1511] Processing CreateTablet for tablet fc634f51e1be45a19b4733c75507025a (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:12:12.681649  7711 tablet_service.cc:1511] Processing CreateTablet for tablet a95e66c97a08430c9b40ea701e97dc17 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:12:12.685581  7707 tablet_service.cc:1511] Processing CreateTablet for tablet 214a25d9d6b14349b684d78df67114e3 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:12:12.686841  7707 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 214a25d9d6b14349b684d78df67114e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.687434  7710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc634f51e1be45a19b4733c75507025a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.688808  7706 tablet_service.cc:1511] Processing CreateTablet for tablet 6d390c1367c84b9b9f46eabe0aed2db6 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:12:12.689493  7711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a95e66c97a08430c9b40ea701e97dc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.690063  7706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d390c1367c84b9b9f46eabe0aed2db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.690502  7636 tablet_service.cc:1511] Processing CreateTablet for tablet fc634f51e1be45a19b4733c75507025a (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:12:12.691780  7636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc634f51e1be45a19b4733c75507025a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.691504  7705 tablet_service.cc:1511] Processing CreateTablet for tablet 21b3171627a4498aab897b3a72b3f0c3 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:12:12.693660  7637 tablet_service.cc:1511] Processing CreateTablet for tablet a95e66c97a08430c9b40ea701e97dc17 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:12:12.694902  7637 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a95e66c97a08430c9b40ea701e97dc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.695464  7635 tablet_service.cc:1511] Processing CreateTablet for tablet 62ed00cf429347c9b053918ef1f89678 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:12:12.696728  7635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62ed00cf429347c9b053918ef1f89678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.696771  7631 tablet_service.cc:1511] Processing CreateTablet for tablet 21b3171627a4498aab897b3a72b3f0c3 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:12:12.698063  7631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b3171627a4498aab897b3a72b3f0c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.699394  7634 tablet_service.cc:1511] Processing CreateTablet for tablet 47f8fad043dd499caf70f67c2707d4cb (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:12:12.699868  7702 tablet_service.cc:1511] Processing CreateTablet for tablet 5b4479fdd61f4dbda42f65f064a04e70 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:12:12.703426  7632 tablet_service.cc:1511] Processing CreateTablet for tablet 6d390c1367c84b9b9f46eabe0aed2db6 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:12:12.703567  7704 tablet_service.cc:1511] Processing CreateTablet for tablet 1ce93d1307cb43ffbf2b5110b26d3a34 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:12:12.704711  7632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d390c1367c84b9b9f46eabe0aed2db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.707398  7708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47f8fad043dd499caf70f67c2707d4cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.707288  7703 tablet_service.cc:1511] Processing CreateTablet for tablet 116b0660221643538da695f3d8d4595c (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:12:12.708220  7705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b3171627a4498aab897b3a72b3f0c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.708374  7630 tablet_service.cc:1511] Processing CreateTablet for tablet 1ce93d1307cb43ffbf2b5110b26d3a34 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:12:12.708942  7702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b4479fdd61f4dbda42f65f064a04e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.708675  7633 tablet_service.cc:1511] Processing CreateTablet for tablet 214a25d9d6b14349b684d78df67114e3 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:12:12.709790  7630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ce93d1307cb43ffbf2b5110b26d3a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.711663  7629 tablet_service.cc:1511] Processing CreateTablet for tablet 116b0660221643538da695f3d8d4595c (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:12:12.713115  7629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 116b0660221643538da695f3d8d4595c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.721602  7703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 116b0660221643538da695f3d8d4595c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.722674  7633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 214a25d9d6b14349b684d78df67114e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.730772  7634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47f8fad043dd499caf70f67c2707d4cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.734619  7761 tablet_bootstrap.cc:492] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841: Bootstrap starting.
I20260709 13:12:12.739544  7628 tablet_service.cc:1511] Processing CreateTablet for tablet 5b4479fdd61f4dbda42f65f064a04e70 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:12:12.740797  7628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b4479fdd61f4dbda42f65f064a04e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.743394  7704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ce93d1307cb43ffbf2b5110b26d3a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.744364  7761 tablet_bootstrap.cc:654] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:12.760005  7762 tablet_bootstrap.cc:492] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b: Bootstrap starting.
I20260709 13:12:12.773666  7762 tablet_bootstrap.cc:654] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:12.792680  7562 tablet_service.cc:1511] Processing CreateTablet for tablet a95e66c97a08430c9b40ea701e97dc17 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:12:12.794008  7562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a95e66c97a08430c9b40ea701e97dc17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.797614  7560 tablet_service.cc:1511] Processing CreateTablet for tablet 62ed00cf429347c9b053918ef1f89678 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:12:12.801430  7555 tablet_service.cc:1511] Processing CreateTablet for tablet 1ce93d1307cb43ffbf2b5110b26d3a34 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:12:12.804179  7559 tablet_service.cc:1511] Processing CreateTablet for tablet 47f8fad043dd499caf70f67c2707d4cb (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:12:12.806553  7560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62ed00cf429347c9b053918ef1f89678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.807324  7555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ce93d1307cb43ffbf2b5110b26d3a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.807073  7558 tablet_service.cc:1511] Processing CreateTablet for tablet 214a25d9d6b14349b684d78df67114e3 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:12:12.807988  7559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47f8fad043dd499caf70f67c2707d4cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.809235  7561 tablet_service.cc:1511] Processing CreateTablet for tablet fc634f51e1be45a19b4733c75507025a (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:12:12.810518  7561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc634f51e1be45a19b4733c75507025a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.814474  7557 tablet_service.cc:1511] Processing CreateTablet for tablet 6d390c1367c84b9b9f46eabe0aed2db6 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:12:12.815729  7557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d390c1367c84b9b9f46eabe0aed2db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.802014  7556 tablet_service.cc:1511] Processing CreateTablet for tablet 21b3171627a4498aab897b3a72b3f0c3 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:12:12.819388  7556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b3171627a4498aab897b3a72b3f0c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.822752  7558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 214a25d9d6b14349b684d78df67114e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.831215  7761 tablet_bootstrap.cc:492] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841: No bootstrap required, opened a new log
I20260709 13:12:12.832875  7554 tablet_service.cc:1511] Processing CreateTablet for tablet 5b4479fdd61f4dbda42f65f064a04e70 (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:12:12.834141  7554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b4479fdd61f4dbda42f65f064a04e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.839246  7555 tablet_service.cc:1511] Processing CreateTablet for tablet 116b0660221643538da695f3d8d4595c (DEFAULT_TABLE table=test-workload [id=8f0c32e9b85c4c36bbb447f1c5def047]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:12:12.840456  7555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 116b0660221643538da695f3d8d4595c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:12.842482  7761 ts_tablet_manager.cc:1403] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent bootstrapping tablet: real 0.108s	user 0.020s	sys 0.026s
I20260709 13:12:12.845340  7761 raft_consensus.cc:359] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:12.846041  7761 raft_consensus.cc:385] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:12.846333  7761 raft_consensus.cc:740] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Initialized, Role: FOLLOWER
I20260709 13:12:12.859248  7761 consensus_queue.cc:260] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:12.882328  7762 tablet_bootstrap.cc:492] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b: No bootstrap required, opened a new log
I20260709 13:12:12.882817  7762 ts_tablet_manager.cc:1403] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b: Time spent bootstrapping tablet: real 0.123s	user 0.019s	sys 0.005s
I20260709 13:12:12.882936  7672 heartbeater.cc:499] Master 127.6.198.126:45895 was elected leader, sending a full tablet report...
I20260709 13:12:12.887321  7761 ts_tablet_manager.cc:1434] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent starting tablet: real 0.044s	user 0.009s	sys 0.007s
I20260709 13:12:12.888214  7761 tablet_bootstrap.cc:492] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841: Bootstrap starting.
I20260709 13:12:12.889314  7762 raft_consensus.cc:359] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:12.890007  7762 raft_consensus.cc:385] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:12.890302  7762 raft_consensus.cc:740] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Initialized, Role: FOLLOWER
I20260709 13:12:12.897001  7764 tablet_bootstrap.cc:492] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476: Bootstrap starting.
I20260709 13:12:12.901124  7762 consensus_queue.cc:260] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:12.903735  7761 tablet_bootstrap.cc:654] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:12.906253  7746 heartbeater.cc:499] Master 127.6.198.126:45895 was elected leader, sending a full tablet report...
I20260709 13:12:12.905491  7762 ts_tablet_manager.cc:1434] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.002s
I20260709 13:12:12.912077  7764 tablet_bootstrap.cc:654] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:12.916095  7761 tablet_bootstrap.cc:492] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841: No bootstrap required, opened a new log
I20260709 13:12:12.916582  7761 ts_tablet_manager.cc:1403] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.004s
I20260709 13:12:12.916821  7764 tablet_bootstrap.cc:492] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476: No bootstrap required, opened a new log
I20260709 13:12:12.917246  7764 ts_tablet_manager.cc:1403] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.003s
I20260709 13:12:12.919535  7761 raft_consensus.cc:359] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:12.919847  7764 raft_consensus.cc:359] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:12.920303  7761 raft_consensus.cc:385] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:12.920504  7764 raft_consensus.cc:385] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:12.920681  7761 raft_consensus.cc:740] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Initialized, Role: FOLLOWER
I20260709 13:12:12.920698  7762 tablet_bootstrap.cc:492] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b: Bootstrap starting.
I20260709 13:12:12.920830  7764 raft_consensus.cc:740] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Initialized, Role: FOLLOWER
I20260709 13:12:12.921630  7761 consensus_queue.cc:260] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:12.924259  7761 ts_tablet_manager.cc:1434] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:12:12.925122  7761 tablet_bootstrap.cc:492] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841: Bootstrap starting.
I20260709 13:12:12.926378  7764 consensus_queue.cc:260] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:12.927973  7762 tablet_bootstrap.cc:654] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:12.935402  7597 heartbeater.cc:499] Master 127.6.198.126:45895 was elected leader, sending a full tablet report...
I20260709 13:12:12.935861  7761 tablet_bootstrap.cc:654] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:12.936533  7764 ts_tablet_manager.cc:1434] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.008s
I20260709 13:12:12.937582  7764 tablet_bootstrap.cc:492] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476: Bootstrap starting.
I20260709 13:12:12.943262  7762 tablet_bootstrap.cc:492] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b: No bootstrap required, opened a new log
I20260709 13:12:12.943759  7762 ts_tablet_manager.cc:1403] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.010s
I20260709 13:12:12.944471  7764 tablet_bootstrap.cc:654] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:12.946554  7762 raft_consensus.cc:359] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:12.947278  7762 raft_consensus.cc:385] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:12.947582  7762 raft_consensus.cc:740] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Initialized, Role: FOLLOWER
I20260709 13:12:12.948534  7762 consensus_queue.cc:260] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:12.954270  7762 ts_tablet_manager.cc:1434] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:12:12.955168  7762 tablet_bootstrap.cc:492] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b: Bootstrap starting.
I20260709 13:12:12.956466  7766 raft_consensus.cc:493] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:12.957077  7766 raft_consensus.cc:515] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:12.961596  7762 tablet_bootstrap.cc:654] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:12.965857  7770 raft_consensus.cc:493] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:12.966331  7770 raft_consensus.cc:515] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:12.971136  7761 tablet_bootstrap.cc:492] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841: No bootstrap required, opened a new log
I20260709 13:12:12.971414  7770 leader_election.cc:290] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:12.971593  7761 ts_tablet_manager.cc:1403] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent bootstrapping tablet: real 0.047s	user 0.018s	sys 0.013s
I20260709 13:12:12.976823  7761 raft_consensus.cc:359] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:12.977489  7761 raft_consensus.cc:385] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:12.977787  7761 raft_consensus.cc:740] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Initialized, Role: FOLLOWER
I20260709 13:12:12.978495  7761 consensus_queue.cc:260] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:12.993450  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b3171627a4498aab897b3a72b3f0c3" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" is_pre_election: true
W20260709 13:12:12.997009  7610 leader_election.cc:343] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:12.997941  7766 leader_election.cc:290] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:12.999048  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b3171627a4498aab897b3a72b3f0c3" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b" is_pre_election: true
I20260709 13:12:12.999898  7721 raft_consensus.cc:2468] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 0.
I20260709 13:12:13.007172  7611 leader_election.cc:304] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 5e15b6cc1f864cf4ac53f1a8548cd841; no voters: 70a4ee1dd69f498d8bfc06683ed1c476
I20260709 13:12:13.008859  7766 raft_consensus.cc:2804] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:13.010013  7766 raft_consensus.cc:493] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.010797  7766 raft_consensus.cc:3060] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.020152  7766 raft_consensus.cc:515] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.023043  7764 tablet_bootstrap.cc:492] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476: No bootstrap required, opened a new log
I20260709 13:12:13.024019  7764 ts_tablet_manager.cc:1403] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent bootstrapping tablet: real 0.087s	user 0.014s	sys 0.007s
I20260709 13:12:13.027590  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b3171627a4498aab897b3a72b3f0c3" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
I20260709 13:12:13.028610  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47f8fad043dd499caf70f67c2707d4cb" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b" is_pre_election: true
W20260709 13:12:13.029388  7610 leader_election.cc:343] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:13.030277  7536 leader_election.cc:343] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:13.031420  7764 raft_consensus.cc:359] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.032027  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b3171627a4498aab897b3a72b3f0c3" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:13.032586  7766 leader_election.cc:290] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 election: Requested vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.033098  7721 raft_consensus.cc:3060] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.034574  7764 raft_consensus.cc:385] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.036391  7764 raft_consensus.cc:740] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.037142  7764 consensus_queue.cc:260] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.037493  7647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47f8fad043dd499caf70f67c2707d4cb" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" is_pre_election: true
I20260709 13:12:13.038139  7647 raft_consensus.cc:2468] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 0.
I20260709 13:12:13.039057  7762 tablet_bootstrap.cc:492] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b: No bootstrap required, opened a new log
I20260709 13:12:13.039511  7538 leader_election.cc:304] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 19f3607054c14bd88d20be41d80b9b3b
I20260709 13:12:13.039515  7762 ts_tablet_manager.cc:1403] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b: Time spent bootstrapping tablet: real 0.085s	user 0.029s	sys 0.052s
I20260709 13:12:13.040652  7764 ts_tablet_manager.cc:1434] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:12:13.041567  7764 tablet_bootstrap.cc:492] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476: Bootstrap starting.
I20260709 13:12:13.043483  7721 raft_consensus.cc:2468] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 1.
I20260709 13:12:13.043495  7770 raft_consensus.cc:2804] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:13.044450  7770 raft_consensus.cc:493] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.044787  7770 raft_consensus.cc:3060] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.044754  7611 leader_election.cc:304] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 5e15b6cc1f864cf4ac53f1a8548cd841; no voters: 70a4ee1dd69f498d8bfc06683ed1c476
I20260709 13:12:13.045256  7761 ts_tablet_manager.cc:1434] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent starting tablet: real 0.071s	user 0.023s	sys 0.033s
I20260709 13:12:13.046173  7779 raft_consensus.cc:2804] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:13.046705  7779 raft_consensus.cc:697] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Becoming Leader. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Running, Role: LEADER
I20260709 13:12:13.047686  7764 tablet_bootstrap.cc:654] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.047686  7779 consensus_queue.cc:237] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.043327  7762 raft_consensus.cc:359] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.050514  7762 raft_consensus.cc:385] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.050829  7762 raft_consensus.cc:740] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.051731  7762 consensus_queue.cc:260] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.055404  7761 tablet_bootstrap.cc:492] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841: Bootstrap starting.
I20260709 13:12:13.057623  7770 raft_consensus.cc:515] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.059963  7762 ts_tablet_manager.cc:1434] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:12:13.060859  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47f8fad043dd499caf70f67c2707d4cb" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:13.061128  7762 tablet_bootstrap.cc:492] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b: Bootstrap starting.
W20260709 13:12:13.061996  7536 leader_election.cc:343] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:13.061947  7647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47f8fad043dd499caf70f67c2707d4cb" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:13.062597  7647 raft_consensus.cc:3060] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.062817  7770 leader_election.cc:290] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Requested vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.066638  7764 tablet_bootstrap.cc:492] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476: No bootstrap required, opened a new log
I20260709 13:12:13.067155  7764 ts_tablet_manager.cc:1403] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent bootstrapping tablet: real 0.026s	user 0.004s	sys 0.009s
I20260709 13:12:13.069164  7762 tablet_bootstrap.cc:654] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.070043  7764 raft_consensus.cc:359] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:13.070783  7764 raft_consensus.cc:385] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.071128  7764 raft_consensus.cc:740] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.072002  7764 consensus_queue.cc:260] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:13.074764  7647 raft_consensus.cc:2468] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 1.
I20260709 13:12:13.075903  7538 leader_election.cc:304] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 19f3607054c14bd88d20be41d80b9b3b
I20260709 13:12:13.076975  7770 raft_consensus.cc:2804] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:13.077351  7764 ts_tablet_manager.cc:1434] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:12:13.078609  7764 tablet_bootstrap.cc:492] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476: Bootstrap starting.
I20260709 13:12:13.080046  7761 tablet_bootstrap.cc:654] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.082655  7762 tablet_bootstrap.cc:492] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b: No bootstrap required, opened a new log
I20260709 13:12:13.082835  7770 raft_consensus.cc:697] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Becoming Leader. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:13.083119  7762 ts_tablet_manager.cc:1403] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.011s
I20260709 13:12:13.084051  7770 consensus_queue.cc:237] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.086066  7762 raft_consensus.cc:359] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.086884  7762 raft_consensus.cc:385] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.087069  7764 tablet_bootstrap.cc:654] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.087196  7762 raft_consensus.cc:740] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.088105  7762 consensus_queue.cc:260] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.088004  7469 catalog_manager.cc:5697] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:13.093391  7762 ts_tablet_manager.cc:1434] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:12:13.094638  7762 tablet_bootstrap.cc:492] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b: Bootstrap starting.
I20260709 13:12:13.104979  7761 tablet_bootstrap.cc:492] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841: No bootstrap required, opened a new log
I20260709 13:12:13.105445  7761 ts_tablet_manager.cc:1403] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent bootstrapping tablet: real 0.050s	user 0.007s	sys 0.019s
I20260709 13:12:13.106732  7762 tablet_bootstrap.cc:654] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.108265  7761 raft_consensus.cc:359] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.109033  7761 raft_consensus.cc:385] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.109320  7761 raft_consensus.cc:740] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.109952  7764 tablet_bootstrap.cc:492] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476: No bootstrap required, opened a new log
I20260709 13:12:13.110148  7761 consensus_queue.cc:260] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.110410  7764 ts_tablet_manager.cc:1403] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.016s
I20260709 13:12:13.130631  7761 ts_tablet_manager.cc:1434] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.021s
I20260709 13:12:13.131608  7761 tablet_bootstrap.cc:492] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841: Bootstrap starting.
I20260709 13:12:13.131843  7764 raft_consensus.cc:359] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.132515  7764 raft_consensus.cc:385] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.132792  7764 raft_consensus.cc:740] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.137223  7761 tablet_bootstrap.cc:654] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.140075  7764 consensus_queue.cc:260] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.142532  7764 ts_tablet_manager.cc:1434] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:12:13.143427  7764 tablet_bootstrap.cc:492] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476: Bootstrap starting.
I20260709 13:12:13.149309  7761 tablet_bootstrap.cc:492] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841: No bootstrap required, opened a new log
I20260709 13:12:13.149598  7764 tablet_bootstrap.cc:654] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.149813  7761 ts_tablet_manager.cc:1403] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 13:12:13.152719  7761 raft_consensus.cc:359] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:13.153448  7761 raft_consensus.cc:385] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.153760  7761 raft_consensus.cc:740] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.153049  7468 catalog_manager.cc:5697] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:13.154606  7764 tablet_bootstrap.cc:492] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476: No bootstrap required, opened a new log
I20260709 13:12:13.155021  7764 ts_tablet_manager.cc:1403] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent bootstrapping tablet: real 0.012s	user 0.008s	sys 0.004s
I20260709 13:12:13.149319  7762 tablet_bootstrap.cc:492] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b: No bootstrap required, opened a new log
I20260709 13:12:13.155622  7762 ts_tablet_manager.cc:1403] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b: Time spent bootstrapping tablet: real 0.061s	user 0.029s	sys 0.021s
I20260709 13:12:13.154500  7761 consensus_queue.cc:260] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:13.157519  7764 raft_consensus.cc:359] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.158167  7764 raft_consensus.cc:385] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.158452  7761 ts_tablet_manager.cc:1434] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:12:13.158478  7764 raft_consensus.cc:740] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.158315  7762 raft_consensus.cc:359] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:13.159113  7762 raft_consensus.cc:385] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.159402  7762 raft_consensus.cc:740] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.159426  7764 consensus_queue.cc:260] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.160236  7762 consensus_queue.cc:260] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:13.159509  7761 tablet_bootstrap.cc:492] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841: Bootstrap starting.
I20260709 13:12:13.162667  7762 ts_tablet_manager.cc:1434] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:12:13.163616  7762 tablet_bootstrap.cc:492] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b: Bootstrap starting.
I20260709 13:12:13.166123  7764 ts_tablet_manager.cc:1434] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:12:13.167023  7764 tablet_bootstrap.cc:492] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476: Bootstrap starting.
I20260709 13:12:13.167706  7761 tablet_bootstrap.cc:654] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.169703  7762 tablet_bootstrap.cc:654] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.169816  7779 raft_consensus.cc:493] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.173130  7764 tablet_bootstrap.cc:654] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.174410  7779 raft_consensus.cc:515] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:13.176735  7779 leader_election.cc:290] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409), 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351)
I20260709 13:12:13.178195  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95e66c97a08430c9b40ea701e97dc17" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b" is_pre_election: true
I20260709 13:12:13.178820  7721 raft_consensus.cc:2468] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 0.
I20260709 13:12:13.179008  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95e66c97a08430c9b40ea701e97dc17" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" is_pre_election: true
I20260709 13:12:13.179566  7761 tablet_bootstrap.cc:492] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841: No bootstrap required, opened a new log
I20260709 13:12:13.179572  7572 raft_consensus.cc:2468] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 0.
I20260709 13:12:13.180171  7761 ts_tablet_manager.cc:1403] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.008s
I20260709 13:12:13.181188  7611 leader_election.cc:304] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 5e15b6cc1f864cf4ac53f1a8548cd841; no voters: 
I20260709 13:12:13.182745  7764 tablet_bootstrap.cc:492] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476: No bootstrap required, opened a new log
I20260709 13:12:13.183187  7764 ts_tablet_manager.cc:1403] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 13:12:13.183104  7766 raft_consensus.cc:2804] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:13.183020  7761 raft_consensus.cc:359] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.183648  7766 raft_consensus.cc:493] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.183842  7761 raft_consensus.cc:385] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.184082  7766 raft_consensus.cc:3060] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.184187  7761 raft_consensus.cc:740] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.185307  7761 consensus_queue.cc:260] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.186129  7764 raft_consensus.cc:359] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.186884  7764 raft_consensus.cc:385] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.187255  7764 raft_consensus.cc:740] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.187934  7764 consensus_queue.cc:260] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.189987  7764 ts_tablet_manager.cc:1434] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:12:13.190840  7764 tablet_bootstrap.cc:492] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476: Bootstrap starting.
I20260709 13:12:13.191154  7766 raft_consensus.cc:515] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:13.192746  7779 raft_consensus.cc:493] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.196784  7764 tablet_bootstrap.cc:654] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.196569  7779 raft_consensus.cc:515] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.198696  7779 leader_election.cc:290] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.193078  7766 leader_election.cc:290] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 election: Requested vote from peers 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409), 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351)
I20260709 13:12:13.194607  7762 tablet_bootstrap.cc:492] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b: No bootstrap required, opened a new log
I20260709 13:12:13.199218  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95e66c97a08430c9b40ea701e97dc17" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
I20260709 13:12:13.195515  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95e66c97a08430c9b40ea701e97dc17" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:13.200606  7572 raft_consensus.cc:3060] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.200770  7721 raft_consensus.cc:3060] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.203473  7720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116b0660221643538da695f3d8d4595c" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b" is_pre_election: true
W20260709 13:12:13.204777  7518 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:13.206497  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116b0660221643538da695f3d8d4595c" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" is_pre_election: true
I20260709 13:12:13.208173  7770 raft_consensus.cc:493] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.208925  7762 ts_tablet_manager.cc:1403] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.008s
I20260709 13:12:13.208745  7770 raft_consensus.cc:515] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.212901  7572 raft_consensus.cc:2468] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 1.
I20260709 13:12:13.215416  7762 raft_consensus.cc:359] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.216759  7770 leader_election.cc:290] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.216907  7762 raft_consensus.cc:385] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.217433  7762 raft_consensus.cc:740] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.217792  7720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc634f51e1be45a19b4733c75507025a" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b" is_pre_election: true
I20260709 13:12:13.218585  7720 raft_consensus.cc:2468] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 0.
I20260709 13:12:13.218454  7762 consensus_queue.cc:260] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.220116  7536 leader_election.cc:304] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
W20260709 13:12:13.221654  7611 leader_election.cc:343] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:13.221810  7762 ts_tablet_manager.cc:1434] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 13:12:13.223953  7761 ts_tablet_manager.cc:1434] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent starting tablet: real 0.043s	user 0.022s	sys 0.018s
W20260709 13:12:13.224432  7610 leader_election.cc:343] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:13.224958  7761 tablet_bootstrap.cc:492] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841: Bootstrap starting.
I20260709 13:12:13.223978  7762 tablet_bootstrap.cc:492] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b: Bootstrap starting.
I20260709 13:12:13.231294  7762 tablet_bootstrap.cc:654] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.235559  7647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc634f51e1be45a19b4733c75507025a" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" is_pre_election: true
I20260709 13:12:13.236366  7647 raft_consensus.cc:2468] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 0.
I20260709 13:12:13.237775  7761 tablet_bootstrap.cc:654] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.224982  7610 leader_election.cc:304] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e15b6cc1f864cf4ac53f1a8548cd841; no voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476
I20260709 13:12:13.239957  7610 leader_election.cc:304] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:13.240970  7766 raft_consensus.cc:2749] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:13.241767  7766 raft_consensus.cc:2804] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:13.242316  7766 raft_consensus.cc:697] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Becoming Leader. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Running, Role: LEADER
I20260709 13:12:13.243374  7766 consensus_queue.cc:237] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:13.248499  7770 raft_consensus.cc:2804] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:13.249055  7770 raft_consensus.cc:493] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.249545  7770 raft_consensus.cc:3060] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.271988  7721 raft_consensus.cc:2468] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 1.
I20260709 13:12:13.291882  7470 catalog_manager.cc:5697] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:13.296761  7770 raft_consensus.cc:515] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.304561  7647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc634f51e1be45a19b4733c75507025a" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:13.305266  7647 raft_consensus.cc:3060] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.315213  7647 raft_consensus.cc:2468] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 1.
I20260709 13:12:13.317628  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc634f51e1be45a19b4733c75507025a" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:13.324290  7721 raft_consensus.cc:3060] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.325017  7761 tablet_bootstrap.cc:492] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841: No bootstrap required, opened a new log
I20260709 13:12:13.325477  7761 ts_tablet_manager.cc:1403] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent bootstrapping tablet: real 0.101s	user 0.027s	sys 0.062s
I20260709 13:12:13.325567  7762 tablet_bootstrap.cc:492] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b: No bootstrap required, opened a new log
I20260709 13:12:13.325904  7764 tablet_bootstrap.cc:492] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476: No bootstrap required, opened a new log
I20260709 13:12:13.326210  7762 ts_tablet_manager.cc:1403] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b: Time spent bootstrapping tablet: real 0.103s	user 0.022s	sys 0.066s
I20260709 13:12:13.326395  7764 ts_tablet_manager.cc:1403] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent bootstrapping tablet: real 0.136s	user 0.051s	sys 0.080s
I20260709 13:12:13.318213  7770 leader_election.cc:290] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Requested vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.328269  7761 raft_consensus.cc:359] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.328985  7761 raft_consensus.cc:385] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.329290  7761 raft_consensus.cc:740] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.329111  7764 raft_consensus.cc:359] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.329372  7538 leader_election.cc:304] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:13.329795  7764 raft_consensus.cc:385] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.330106  7764 raft_consensus.cc:740] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.330363  7770 raft_consensus.cc:2804] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:13.330201  7761 consensus_queue.cc:260] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.330996  7770 raft_consensus.cc:697] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Becoming Leader. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:13.331962  7721 raft_consensus.cc:2468] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 1.
I20260709 13:12:13.331614  7762 raft_consensus.cc:359] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.331868  7770 consensus_queue.cc:237] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.332657  7762 raft_consensus.cc:385] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.333113  7764 consensus_queue.cc:260] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.333621  7762 raft_consensus.cc:740] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.334744  7762 consensus_queue.cc:260] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.335215  7764 ts_tablet_manager.cc:1434] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:12:13.335215  7761 ts_tablet_manager.cc:1434] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:12:13.336359  7761 tablet_bootstrap.cc:492] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841: Bootstrap starting.
I20260709 13:12:13.336366  7764 tablet_bootstrap.cc:492] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476: Bootstrap starting.
I20260709 13:12:13.337090  7762 ts_tablet_manager.cc:1434] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:12:13.337996  7762 tablet_bootstrap.cc:492] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b: Bootstrap starting.
I20260709 13:12:13.342754  7761 tablet_bootstrap.cc:654] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.342754  7764 tablet_bootstrap.cc:654] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.344049  7762 tablet_bootstrap.cc:654] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.360560  7767 raft_consensus.cc:493] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.361158  7767 raft_consensus.cc:515] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.370904  7767 leader_election.cc:290] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653)
I20260709 13:12:13.379556  7470 catalog_manager.cc:5697] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:13.417716  7770 raft_consensus.cc:493] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.418524  7770 raft_consensus.cc:515] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.421948  7783 raft_consensus.cc:493] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.422441  7783 raft_consensus.cc:515] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.424744  7783 leader_election.cc:290] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.425681  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce93d1307cb43ffbf2b5110b26d3a34" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" is_pre_election: true
I20260709 13:12:13.426203  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce93d1307cb43ffbf2b5110b26d3a34" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b" is_pre_election: true
I20260709 13:12:13.426566  7646 raft_consensus.cc:2468] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 0.
I20260709 13:12:13.426851  7721 raft_consensus.cc:2468] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 0.
I20260709 13:12:13.428028  7536 leader_election.cc:304] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:13.429023  7783 raft_consensus.cc:2804] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:13.429394  7783 raft_consensus.cc:493] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.429720  7783 raft_consensus.cc:3060] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.436658  7783 raft_consensus.cc:515] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.437834  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4479fdd61f4dbda42f65f064a04e70" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b" is_pre_election: true
I20260709 13:12:13.421941  7647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4479fdd61f4dbda42f65f064a04e70" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" is_pre_election: true
I20260709 13:12:13.440096  7647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce93d1307cb43ffbf2b5110b26d3a34" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
W20260709 13:12:13.440332  7538 leader_election.cc:343] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:13.440605  7647 raft_consensus.cc:3060] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:13.441071  7536 leader_election.cc:343] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:13.441463  7536 leader_election.cc:304] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 19f3607054c14bd88d20be41d80b9b3b, 5e15b6cc1f864cf4ac53f1a8548cd841
I20260709 13:12:13.445951  7767 raft_consensus.cc:493] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.446403  7767 raft_consensus.cc:515] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.447964  7647 raft_consensus.cc:2468] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 1.
I20260709 13:12:13.448266  7767 leader_election.cc:290] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653)
I20260709 13:12:13.449973  7804 raft_consensus.cc:2749] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:13.450078  7770 leader_election.cc:290] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.451918  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce93d1307cb43ffbf2b5110b26d3a34" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:13.452509  7721 raft_consensus.cc:3060] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.452988  7538 leader_election.cc:304] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:13.453846  7804 raft_consensus.cc:2804] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:13.455601  7804 raft_consensus.cc:697] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Becoming Leader. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:13.456406  7804 consensus_queue.cc:237] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.459656  7721 raft_consensus.cc:2468] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 1.
I20260709 13:12:13.473578  7764 tablet_bootstrap.cc:492] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476: No bootstrap required, opened a new log
I20260709 13:12:13.474084  7764 ts_tablet_manager.cc:1403] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent bootstrapping tablet: real 0.138s	user 0.057s	sys 0.071s
I20260709 13:12:13.451942  7783 leader_election.cc:290] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Requested vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.475679  7762 tablet_bootstrap.cc:492] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b: No bootstrap required, opened a new log
I20260709 13:12:13.476549  7764 raft_consensus.cc:359] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.476961  7762 ts_tablet_manager.cc:1403] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b: Time spent bootstrapping tablet: real 0.139s	user 0.033s	sys 0.061s
I20260709 13:12:13.477177  7764 raft_consensus.cc:385] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.477463  7764 raft_consensus.cc:740] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.477846  7761 tablet_bootstrap.cc:492] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841: No bootstrap required, opened a new log
I20260709 13:12:13.478257  7761 ts_tablet_manager.cc:1403] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent bootstrapping tablet: real 0.142s	user 0.034s	sys 0.102s
I20260709 13:12:13.478171  7764 consensus_queue.cc:260] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.479149  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ed00cf429347c9b053918ef1f89678" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" is_pre_election: true
I20260709 13:12:13.479425  7762 raft_consensus.cc:359] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.479736  7572 raft_consensus.cc:2468] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19f3607054c14bd88d20be41d80b9b3b in term 0.
I20260709 13:12:13.480026  7762 raft_consensus.cc:385] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.480078  7764 ts_tablet_manager.cc:1434] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:12:13.480398  7762 raft_consensus.cc:740] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.480592  7761 raft_consensus.cc:359] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.481194  7764 tablet_bootstrap.cc:492] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476: Bootstrap starting.
I20260709 13:12:13.475821  7470 catalog_manager.cc:5697] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:13.481081  7762 consensus_queue.cc:260] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.481647  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" is_pre_election: true
I20260709 13:12:13.483049  7762 ts_tablet_manager.cc:1434] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:12:13.484140  7684 leader_election.cc:304] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:13.484927  7767 raft_consensus.cc:2804] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:13.485147  7762 tablet_bootstrap.cc:492] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b: Bootstrap starting.
I20260709 13:12:13.485275  7767 raft_consensus.cc:493] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.485600  7767 raft_consensus.cc:3060] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.486438  7764 tablet_bootstrap.cc:654] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:13.490720  7684 leader_election.cc:343] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:13.493381  7767 raft_consensus.cc:515] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.481211  7761 raft_consensus.cc:385] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.495826  7761 raft_consensus.cc:740] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.496846  7761 consensus_queue.cc:260] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.492055  7762 tablet_bootstrap.cc:654] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.505062  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ed00cf429347c9b053918ef1f89678" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
I20260709 13:12:13.505617  7572 raft_consensus.cc:3060] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.507243  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" is_pre_election: true
W20260709 13:12:13.508654  7687 leader_election.cc:343] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:13.509068  7687 leader_election.cc:304] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b; no voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476
I20260709 13:12:13.503770  7761 ts_tablet_manager.cc:1434] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.000s
I20260709 13:12:13.510854  7647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ed00cf429347c9b053918ef1f89678" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" is_pre_election: true
I20260709 13:12:13.511627  7647 raft_consensus.cc:2468] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19f3607054c14bd88d20be41d80b9b3b in term 0.
I20260709 13:12:13.516378  7572 raft_consensus.cc:2468] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19f3607054c14bd88d20be41d80b9b3b in term 1.
I20260709 13:12:13.518786  7684 leader_election.cc:304] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:13.519296  7761 tablet_bootstrap.cc:492] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841: Bootstrap starting.
I20260709 13:12:13.517514  7767 leader_election.cc:290] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 1 election: Requested vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653)
I20260709 13:12:13.520414  7783 raft_consensus.cc:493] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.520424  7767 raft_consensus.cc:493] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.521003  7783 raft_consensus.cc:515] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.521229  7767 raft_consensus.cc:515] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.523761  7647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a25d9d6b14349b684d78df67114e3" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" is_pre_election: true
I20260709 13:12:13.524251  7783 leader_election.cc:290] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.524434  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a25d9d6b14349b684d78df67114e3" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b" is_pre_election: true
I20260709 13:12:13.525041  7721 raft_consensus.cc:2468] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 0.
W20260709 13:12:13.525178  7538 leader_election.cc:343] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:13.526360  7536 leader_election.cc:304] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 5e15b6cc1f864cf4ac53f1a8548cd841
I20260709 13:12:13.527372  7783 raft_consensus.cc:2804] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:13.527717  7783 raft_consensus.cc:493] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.528007  7761 tablet_bootstrap.cc:654] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.528059  7783 raft_consensus.cc:3060] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.528934  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47f8fad043dd499caf70f67c2707d4cb" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" is_pre_election: true
I20260709 13:12:13.530793  7810 raft_consensus.cc:2749] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:13.531553  7767 leader_election.cc:290] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653)
I20260709 13:12:13.532182  7764 tablet_bootstrap.cc:492] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476: No bootstrap required, opened a new log
I20260709 13:12:13.532840  7764 ts_tablet_manager.cc:1403] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent bootstrapping tablet: real 0.052s	user 0.010s	sys 0.018s
I20260709 13:12:13.532265  7810 raft_consensus.cc:2804] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:13.534637  7810 raft_consensus.cc:697] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 1 LEADER]: Becoming Leader. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Running, Role: LEADER
I20260709 13:12:13.534976  7783 raft_consensus.cc:515] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.535532  7810 consensus_queue.cc:237] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.536237  7764 raft_consensus.cc:359] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.537093  7764 raft_consensus.cc:385] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.537478  7764 raft_consensus.cc:740] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.539191  7647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a25d9d6b14349b684d78df67114e3" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:13.539875  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ed00cf429347c9b053918ef1f89678" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
W20260709 13:12:13.540428  7538 leader_election.cc:343] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:13.540589  7646 raft_consensus.cc:3060] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.542930  7762 tablet_bootstrap.cc:492] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b: No bootstrap required, opened a new log
I20260709 13:12:13.543457  7762 ts_tablet_manager.cc:1403] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b: Time spent bootstrapping tablet: real 0.058s	user 0.009s	sys 0.007s
I20260709 13:12:13.545356  7761 tablet_bootstrap.cc:492] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841: No bootstrap required, opened a new log
I20260709 13:12:13.545764  7761 ts_tablet_manager.cc:1403] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.001s
I20260709 13:12:13.546279  7804 raft_consensus.cc:493] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.546864  7783 leader_election.cc:290] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Requested vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.546748  7804 raft_consensus.cc:515] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.547420  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a25d9d6b14349b684d78df67114e3" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:13.547921  7721 raft_consensus.cc:3060] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.547680  7762 raft_consensus.cc:359] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.548290  7762 raft_consensus.cc:385] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.548237  7761 raft_consensus.cc:359] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.548657  7804 leader_election.cc:290] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.548956  7761 raft_consensus.cc:385] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.548661  7762 raft_consensus.cc:740] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.549261  7761 raft_consensus.cc:740] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.549789  7762 consensus_queue.cc:260] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.549991  7647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116b0660221643538da695f3d8d4595c" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" is_pre_election: true
I20260709 13:12:13.549997  7761 consensus_queue.cc:260] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.550724  7647 raft_consensus.cc:2468] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 0.
I20260709 13:12:13.551764  7762 ts_tablet_manager.cc:1434] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:12:13.551862  7538 leader_election.cc:304] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:13.552610  7804 raft_consensus.cc:2804] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:13.552963  7804 raft_consensus.cc:493] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.553293  7804 raft_consensus.cc:3060] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.554771  7646 raft_consensus.cc:2468] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19f3607054c14bd88d20be41d80b9b3b in term 1.
I20260709 13:12:13.555200  7721 raft_consensus.cc:2468] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 1.
I20260709 13:12:13.556327  7536 leader_election.cc:304] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 5e15b6cc1f864cf4ac53f1a8548cd841
I20260709 13:12:13.556640  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47f8fad043dd499caf70f67c2707d4cb" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" is_pre_election: true
I20260709 13:12:13.557389  7646 raft_consensus.cc:2393] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19f3607054c14bd88d20be41d80b9b3b in current term 1: Already voted for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in this term.
I20260709 13:12:13.553303  7720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116b0660221643538da695f3d8d4595c" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b" is_pre_election: true
I20260709 13:12:13.559592  7687 leader_election.cc:304] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b; no voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476
I20260709 13:12:13.559823  7783 raft_consensus.cc:2804] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:13.560139  7761 ts_tablet_manager.cc:1434] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:12:13.560422  7783 raft_consensus.cc:697] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Becoming Leader. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:13.560433  7809 raft_consensus.cc:3060] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.561491  7783 consensus_queue.cc:237] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.567191  7809 raft_consensus.cc:2749] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:13.567653  7761 tablet_bootstrap.cc:492] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841: Bootstrap starting.
I20260709 13:12:13.567684  7766 consensus_queue.cc:1048] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:13.568183  7764 consensus_queue.cc:260] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.570266  7764 ts_tablet_manager.cc:1434] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.001s
I20260709 13:12:13.571152  7764 tablet_bootstrap.cc:492] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476: Bootstrap starting.
I20260709 13:12:13.574285  7804 raft_consensus.cc:515] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.575652  7766 raft_consensus.cc:493] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.575830  7761 tablet_bootstrap.cc:654] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.576095  7766 raft_consensus.cc:515] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.578843  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4479fdd61f4dbda42f65f064a04e70" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" is_pre_election: true
I20260709 13:12:13.579490  7572 raft_consensus.cc:2468] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 0.
I20260709 13:12:13.579761  7766 leader_election.cc:290] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.580448  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4479fdd61f4dbda42f65f064a04e70" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b" is_pre_election: true
I20260709 13:12:13.580708  7610 leader_election.cc:304] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:13.581027  7721 raft_consensus.cc:2468] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 0.
I20260709 13:12:13.581799  7766 raft_consensus.cc:2804] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:13.582192  7766 raft_consensus.cc:493] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.582592  7766 raft_consensus.cc:3060] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:13.585806  7536 leader_election.cc:343] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:13.586839  7762 tablet_bootstrap.cc:492] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b: Bootstrap starting.
I20260709 13:12:13.587073  7761 tablet_bootstrap.cc:492] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841: No bootstrap required, opened a new log
I20260709 13:12:13.587483  7761 ts_tablet_manager.cc:1403] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:12:13.590005  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116b0660221643538da695f3d8d4595c" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:13.590183  7761 raft_consensus.cc:359] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.590569  7646 raft_consensus.cc:3060] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.590806  7761 raft_consensus.cc:385] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.591081  7761 raft_consensus.cc:740] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.591233  7720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116b0660221643538da695f3d8d4595c" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:13.592171  7764 tablet_bootstrap.cc:654] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.591885  7761 consensus_queue.cc:260] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
W20260709 13:12:13.592430  7536 leader_election.cc:343] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:13.593357  7766 raft_consensus.cc:515] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.595906  7766 leader_election.cc:290] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 election: Requested vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.596932  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4479fdd61f4dbda42f65f064a04e70" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
I20260709 13:12:13.597482  7572 raft_consensus.cc:3060] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.597654  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4479fdd61f4dbda42f65f064a04e70" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:13.598404  7721 raft_consensus.cc:3060] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.599257  7646 raft_consensus.cc:2468] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 1.
I20260709 13:12:13.602664  7538 leader_election.cc:304] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 19f3607054c14bd88d20be41d80b9b3b
I20260709 13:12:13.602924  7468 catalog_manager.cc:5697] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67). New cstate: current_term: 1 leader_uuid: "19f3607054c14bd88d20be41d80b9b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:13.593900  7761 ts_tablet_manager.cc:1434] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 13:12:13.598249  7770 consensus_queue.cc:1048] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:13.612125  7721 raft_consensus.cc:2468] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 1.
I20260709 13:12:13.612470  7770 raft_consensus.cc:2804] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:13.612905  7572 raft_consensus.cc:2468] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 1.
I20260709 13:12:13.604085  7804 leader_election.cc:290] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Requested vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.617961  7611 leader_election.cc:304] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 5e15b6cc1f864cf4ac53f1a8548cd841; no voters: 
I20260709 13:12:13.618445  7770 raft_consensus.cc:697] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Becoming Leader. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:13.623540  7793 raft_consensus.cc:2804] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:13.623479  7770 consensus_queue.cc:237] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.624109  7793 raft_consensus.cc:697] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Becoming Leader. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Running, Role: LEADER
I20260709 13:12:13.625080  7793 consensus_queue.cc:237] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.631022  7818 consensus_queue.cc:1048] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:12:13.633400  7470 catalog_manager.cc:5697] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:13.641515  7610 consensus_peers.cc:597] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 -> Peer 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351): Couldn't send request to peer 70a4ee1dd69f498d8bfc06683ed1c476. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.667984  7762 tablet_bootstrap.cc:654] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:13.676712  7764 tablet_bootstrap.cc:492] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476: No bootstrap required, opened a new log
I20260709 13:12:13.677068  7764 ts_tablet_manager.cc:1403] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent bootstrapping tablet: real 0.106s	user 0.007s	sys 0.008s
I20260709 13:12:13.694916  7820 consensus_queue.cc:1048] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:12:13.696007  7764 raft_consensus.cc:359] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.696895  7764 raft_consensus.cc:385] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.697252  7764 raft_consensus.cc:740] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.698143  7764 consensus_queue.cc:260] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.700763  7764 ts_tablet_manager.cc:1434] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:12:13.707033  7470 catalog_manager.cc:5697] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:13.717044  7762 tablet_bootstrap.cc:492] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b: No bootstrap required, opened a new log
I20260709 13:12:13.718757  7762 ts_tablet_manager.cc:1403] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b: Time spent bootstrapping tablet: real 0.166s	user 0.020s	sys 0.020s
I20260709 13:12:13.721794  7762 raft_consensus.cc:359] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.722453  7762 raft_consensus.cc:385] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:13.721362  7468 catalog_manager.cc:5697] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:13.722777  7762 raft_consensus.cc:740] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Initialized, Role: FOLLOWER
I20260709 13:12:13.723642  7762 consensus_queue.cc:260] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.732168  7804 consensus_queue.cc:1048] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:13.732589  7820 raft_consensus.cc:493] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.733052  7820 raft_consensus.cc:515] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.735060  7820 leader_election.cc:290] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.734889  7762 ts_tablet_manager.cc:1434] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.003s
I20260709 13:12:13.737541  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a25d9d6b14349b684d78df67114e3" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b" is_pre_election: true
I20260709 13:12:13.738266  7721 raft_consensus.cc:2393] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in current term 1: Already voted for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in this term.
I20260709 13:12:13.740432  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a25d9d6b14349b684d78df67114e3" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" is_pre_election: true
I20260709 13:12:13.742193  7610 leader_election.cc:304] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e15b6cc1f864cf4ac53f1a8548cd841; no voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476
I20260709 13:12:13.743238  7820 consensus_queue.cc:1048] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:13.744453  7820 raft_consensus.cc:3060] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.747675  7783 consensus_queue.cc:1048] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:13.760888  7820 raft_consensus.cc:2749] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:13.764961  7783 raft_consensus.cc:493] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.765427  7783 raft_consensus.cc:515] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.768052  7647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" is_pre_election: true
I20260709 13:12:13.768555  7647 raft_consensus.cc:2468] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 0.
I20260709 13:12:13.769526  7538 leader_election.cc:304] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:13.770478  7816 raft_consensus.cc:2804] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:13.770541  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b" is_pre_election: true
I20260709 13:12:13.770885  7816 raft_consensus.cc:493] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:13.771124  7721 raft_consensus.cc:2468] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 0.
I20260709 13:12:13.771231  7816 raft_consensus.cc:3060] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.772329  7783 leader_election.cc:290] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.778923  7816 raft_consensus.cc:515] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.780843  7816 leader_election.cc:290] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Requested vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.781430  7647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:13.781687  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:13.781888  7647 raft_consensus.cc:3060] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.782196  7721 raft_consensus.cc:3060] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.786186  7721 raft_consensus.cc:2468] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 1.
I20260709 13:12:13.786201  7647 raft_consensus.cc:2468] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 1.
I20260709 13:12:13.787143  7536 leader_election.cc:304] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:13.787822  7816 raft_consensus.cc:2804] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:13.788197  7816 raft_consensus.cc:697] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Becoming Leader. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:13.788897  7816 consensus_queue.cc:237] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.796136  7469 catalog_manager.cc:5697] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:13.871556  7816 raft_consensus.cc:493] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.872037  7816 raft_consensus.cc:515] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.873613  7816 leader_election.cc:290] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:13.874513  7647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b3171627a4498aab897b3a72b3f0c3" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" is_pre_election: true
I20260709 13:12:13.874607  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b3171627a4498aab897b3a72b3f0c3" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19f3607054c14bd88d20be41d80b9b3b" is_pre_election: true
I20260709 13:12:13.875989  7538 leader_election.cc:304] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 19f3607054c14bd88d20be41d80b9b3b, 5e15b6cc1f864cf4ac53f1a8548cd841
I20260709 13:12:13.876526  7816 raft_consensus.cc:2749] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:13.918754  7810 raft_consensus.cc:493] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:13.919204  7810 raft_consensus.cc:515] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:13.920910  7810 leader_election.cc:290] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653)
I20260709 13:12:13.921624  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116b0660221643538da695f3d8d4595c" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" is_pre_election: true
I20260709 13:12:13.921838  7647 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116b0660221643538da695f3d8d4595c" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" is_pre_election: true
I20260709 13:12:13.922610  7647 raft_consensus.cc:2393] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 19f3607054c14bd88d20be41d80b9b3b in current term 1: Already voted for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in this term.
I20260709 13:12:13.923404  7687 leader_election.cc:304] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b; no voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476
I20260709 13:12:13.923985  7810 raft_consensus.cc:3060] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:13.928449  7816 consensus_queue.cc:1048] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:13.929406  7810 raft_consensus.cc:2749] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:13.938485  7770 consensus_queue.cc:1048] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:13.940590  7810 consensus_queue.cc:1048] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:13.954712  7810 consensus_queue.cc:1048] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:13.988631  7647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a95e66c97a08430c9b40ea701e97dc17"
dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
mode: GRACEFUL
new_leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
 from {username='slave'} at 127.0.0.1:54720
I20260709 13:12:13.989106  7647 raft_consensus.cc:606] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Received request to transfer leadership to TS 70a4ee1dd69f498d8bfc06683ed1c476
I20260709 13:12:14.002889  7721 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62ed00cf429347c9b053918ef1f89678"
dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
mode: GRACEFUL
new_leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
 from {username='slave'} at 127.0.0.1:39292
I20260709 13:12:14.003319  7721 raft_consensus.cc:606] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 1 LEADER]: Received request to transfer leadership to TS 70a4ee1dd69f498d8bfc06683ed1c476
I20260709 13:12:14.019218  7572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
mode: GRACEFUL
new_leader_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
 from {username='slave'} at 127.0.0.1:50258
I20260709 13:12:14.019654  7572 raft_consensus.cc:606] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Received request to transfer leadership to TS 5e15b6cc1f864cf4ac53f1a8548cd841
I20260709 13:12:14.025787  7572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ce93d1307cb43ffbf2b5110b26d3a34"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
mode: GRACEFUL
new_leader_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
 from {username='slave'} at 127.0.0.1:50258
I20260709 13:12:14.026165  7572 raft_consensus.cc:606] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Received request to transfer leadership to TS 5e15b6cc1f864cf4ac53f1a8548cd841
I20260709 13:12:14.030089  7572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "116b0660221643538da695f3d8d4595c"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
mode: GRACEFUL
new_leader_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
 from {username='slave'} at 127.0.0.1:50258
I20260709 13:12:14.030544  7572 raft_consensus.cc:606] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Received request to transfer leadership to TS 5e15b6cc1f864cf4ac53f1a8548cd841
I20260709 13:12:14.034546  7647 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b4479fdd61f4dbda42f65f064a04e70"
dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
mode: GRACEFUL
new_leader_uuid: "19f3607054c14bd88d20be41d80b9b3b"
 from {username='slave'} at 127.0.0.1:54720
I20260709 13:12:14.034925  7647 raft_consensus.cc:606] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Received request to transfer leadership to TS 19f3607054c14bd88d20be41d80b9b3b
I20260709 13:12:14.047097  7770 consensus_queue.cc:1048] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:14.056514  7818 consensus_queue.cc:1048] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:14.060448  7818 consensus_queue.cc:1048] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:14.067234  7816 raft_consensus.cc:993] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476: : Instructing follower 5e15b6cc1f864cf4ac53f1a8548cd841 to start an election
I20260709 13:12:14.067636  7816 raft_consensus.cc:1081] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Signalling peer 5e15b6cc1f864cf4ac53f1a8548cd841 to start an election
I20260709 13:12:14.073170  7646 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "116b0660221643538da695f3d8d4595c"
dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
 from {username='slave'} at 127.0.0.1:54702
I20260709 13:12:14.073694  7646 raft_consensus.cc:493] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:14.074066  7646 raft_consensus.cc:3060] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.075742  7818 consensus_queue.cc:1048] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:14.083264  7646 raft_consensus.cc:515] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.085294  7646 leader_election.cc:290] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 2 election: Requested vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:14.085942  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116b0660221643538da695f3d8d4595c" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
I20260709 13:12:14.086295  7721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "116b0660221643538da695f3d8d4595c" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:14.086975  7721 raft_consensus.cc:3060] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.086536  7572 raft_consensus.cc:3055] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:14.087662  7572 raft_consensus.cc:740] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:14.088438  7572 consensus_queue.cc:260] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.089601  7572 raft_consensus.cc:3060] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.094581  7721 raft_consensus.cc:2468] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 2.
I20260709 13:12:14.095697  7611 leader_election.cc:304] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 5e15b6cc1f864cf4ac53f1a8548cd841; no voters: 
I20260709 13:12:14.095928  7572 raft_consensus.cc:2468] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 2.
I20260709 13:12:14.096527  7820 raft_consensus.cc:2804] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:14.096980  7820 raft_consensus.cc:697] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 LEADER]: Becoming Leader. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Running, Role: LEADER
I20260709 13:12:14.097743  7820 consensus_queue.cc:237] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.103530  7468 catalog_manager.cc:5697] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 reported cstate change: term changed from 1 to 2, leader changed from 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65) to 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:14.190670  7572 raft_consensus.cc:3060] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:14.212634  7818 consensus_queue.cc:1048] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:14.212793  7820 consensus_queue.cc:1048] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:14.223018  7779 raft_consensus.cc:993] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841: : Instructing follower 19f3607054c14bd88d20be41d80b9b3b to start an election
I20260709 13:12:14.223006  7818 consensus_queue.cc:1048] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:14.223659  7779 raft_consensus.cc:1081] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Signalling peer 19f3607054c14bd88d20be41d80b9b3b to start an election
I20260709 13:12:14.226832  7721 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b4479fdd61f4dbda42f65f064a04e70"
dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
 from {username='slave'} at 127.0.0.1:39274
I20260709 13:12:14.227332  7721 raft_consensus.cc:493] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:14.227667  7793 consensus_queue.cc:1048] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:14.227698  7721 raft_consensus.cc:3060] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.242925  7818 raft_consensus.cc:993] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476: : Instructing follower 5e15b6cc1f864cf4ac53f1a8548cd841 to start an election
I20260709 13:12:14.243362  7818 raft_consensus.cc:1081] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Signalling peer 5e15b6cc1f864cf4ac53f1a8548cd841 to start an election
I20260709 13:12:14.245025  7647 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6"
dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
 from {username='slave'} at 127.0.0.1:54702
I20260709 13:12:14.245568  7647 raft_consensus.cc:493] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:14.245347  7721 raft_consensus.cc:515] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.245903  7647 raft_consensus.cc:3060] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.246538  7719 raft_consensus.cc:1240] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Rejecting Update request from peer 5e15b6cc1f864cf4ac53f1a8548cd841 for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:14.247393  7721 leader_election.cc:290] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 2 election: Requested vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653)
I20260709 13:12:14.248989  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4479fdd61f4dbda42f65f064a04e70" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
I20260709 13:12:14.250514  7820 consensus_queue.cc:1059] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:14.249552  7572 raft_consensus.cc:3060] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.251603  7793 raft_consensus.cc:3055] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:14.251905  7793 raft_consensus.cc:740] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Running, Role: LEADER
I20260709 13:12:14.252554  7793 consensus_queue.cc:260] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.253396  7793 raft_consensus.cc:3060] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.249405  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4479fdd61f4dbda42f65f064a04e70" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:14.254218  7647 raft_consensus.cc:515] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.256323  7647 leader_election.cc:290] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 2 election: Requested vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:14.256860  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
I20260709 13:12:14.257246  7571 raft_consensus.cc:3055] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:14.257443  7571 raft_consensus.cc:740] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:14.258046  7571 consensus_queue.cc:260] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.259088  7571 raft_consensus.cc:3060] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.259601  7719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:14.259948  7572 raft_consensus.cc:2468] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19f3607054c14bd88d20be41d80b9b3b in term 2.
I20260709 13:12:14.260103  7719 raft_consensus.cc:3060] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.261196  7684 leader_election.cc:304] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:14.262017  7809 raft_consensus.cc:2804] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:14.262530  7809 raft_consensus.cc:697] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [term 2 LEADER]: Becoming Leader. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Running, Role: LEADER
I20260709 13:12:14.263317  7809 consensus_queue.cc:237] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.266733  7719 raft_consensus.cc:2468] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 2.
I20260709 13:12:14.272035  7820 raft_consensus.cc:993] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841: : Instructing follower 70a4ee1dd69f498d8bfc06683ed1c476 to start an election
I20260709 13:12:14.271370  7470 catalog_manager.cc:5697] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b reported cstate change: term changed from 1 to 2, leader changed from 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66) to 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67). New cstate: current_term: 2 leader_uuid: "19f3607054c14bd88d20be41d80b9b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:14.272444  7820 raft_consensus.cc:1081] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Signalling peer 70a4ee1dd69f498d8bfc06683ed1c476 to start an election
I20260709 13:12:14.272498  7611 leader_election.cc:304] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 5e15b6cc1f864cf4ac53f1a8548cd841; no voters: 
I20260709 13:12:14.271883  7571 raft_consensus.cc:2468] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 2.
I20260709 13:12:14.273303  7793 raft_consensus.cc:2804] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:14.274142  7646 raft_consensus.cc:2468] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19f3607054c14bd88d20be41d80b9b3b in term 2.
I20260709 13:12:14.274396  7793 raft_consensus.cc:697] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 LEADER]: Becoming Leader. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Running, Role: LEADER
I20260709 13:12:14.275388  7571 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a95e66c97a08430c9b40ea701e97dc17"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
 from {username='slave'} at 127.0.0.1:50240
I20260709 13:12:14.275254  7793 consensus_queue.cc:237] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.275890  7571 raft_consensus.cc:493] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:14.276235  7571 raft_consensus.cc:3060] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.282176  7571 raft_consensus.cc:515] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:14.283625  7470 catalog_manager.cc:5697] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 reported cstate change: term changed from 1 to 2, leader changed from 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65) to 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:14.284566  7571 leader_election.cc:290] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 2 election: Requested vote from peers 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409), 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653)
I20260709 13:12:14.284686  7719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95e66c97a08430c9b40ea701e97dc17" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:14.285235  7719 raft_consensus.cc:3060] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.285120  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95e66c97a08430c9b40ea701e97dc17" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:14.285967  7646 raft_consensus.cc:3055] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:14.286295  7646 raft_consensus.cc:740] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Running, Role: LEADER
I20260709 13:12:14.287137  7646 consensus_queue.cc:260] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:14.288296  7646 raft_consensus.cc:3060] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.291294  7719 raft_consensus.cc:2468] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 2.
I20260709 13:12:14.292256  7536 leader_election.cc:304] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:14.292972  7804 raft_consensus.cc:2804] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:14.293474  7804 raft_consensus.cc:697] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 LEADER]: Becoming Leader. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:14.294155  7646 raft_consensus.cc:2468] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 2.
I20260709 13:12:14.294071  7804 consensus_queue.cc:237] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:14.299903  7470 catalog_manager.cc:5697] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 reported cstate change: term changed from 1 to 2, leader changed from 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66) to 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:14.448518  7783 raft_consensus.cc:993] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476: : Instructing follower 5e15b6cc1f864cf4ac53f1a8548cd841 to start an election
I20260709 13:12:14.448908  7818 raft_consensus.cc:1081] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Signalling peer 5e15b6cc1f864cf4ac53f1a8548cd841 to start an election
I20260709 13:12:14.450146  7646 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ce93d1307cb43ffbf2b5110b26d3a34"
dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
 from {username='slave'} at 127.0.0.1:54702
I20260709 13:12:14.450603  7646 raft_consensus.cc:493] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:14.450899  7646 raft_consensus.cc:3060] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.455121  7646 raft_consensus.cc:515] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.456880  7646 leader_election.cc:290] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 2 election: Requested vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:14.457588  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce93d1307cb43ffbf2b5110b26d3a34" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
I20260709 13:12:14.457832  7719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce93d1307cb43ffbf2b5110b26d3a34" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:14.458143  7571 raft_consensus.cc:3055] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:14.458330  7719 raft_consensus.cc:3060] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.458556  7571 raft_consensus.cc:740] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:14.459326  7571 consensus_queue.cc:260] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.460150  7571 raft_consensus.cc:3060] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.462780  7719 raft_consensus.cc:2468] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 2.
I20260709 13:12:14.463632  7611 leader_election.cc:304] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 5e15b6cc1f864cf4ac53f1a8548cd841; no voters: 
I20260709 13:12:14.464262  7793 raft_consensus.cc:2804] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:14.464555  7571 raft_consensus.cc:2468] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 2.
I20260709 13:12:14.464733  7793 raft_consensus.cc:697] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 LEADER]: Becoming Leader. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Running, Role: LEADER
I20260709 13:12:14.465540  7793 consensus_queue.cc:237] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.471112  7470 catalog_manager.cc:5697] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 reported cstate change: term changed from 1 to 2, leader changed from 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65) to 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:14.530298  7810 raft_consensus.cc:993] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b: : Instructing follower 70a4ee1dd69f498d8bfc06683ed1c476 to start an election
I20260709 13:12:14.530701  7809 raft_consensus.cc:1081] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 1 LEADER]: Signalling peer 70a4ee1dd69f498d8bfc06683ed1c476 to start an election
I20260709 13:12:14.531966  7571 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62ed00cf429347c9b053918ef1f89678"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
 from {username='slave'} at 127.0.0.1:50252
I20260709 13:12:14.532390  7571 raft_consensus.cc:493] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:14.532675  7571 raft_consensus.cc:3060] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.538290  7571 raft_consensus.cc:515] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.540149  7571 leader_election.cc:290] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 2 election: Requested vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:14.541071  7719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ed00cf429347c9b053918ef1f89678" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:14.541320  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ed00cf429347c9b053918ef1f89678" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:14.541647  7719 raft_consensus.cc:3055] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:14.541831  7646 raft_consensus.cc:3060] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.542011  7719 raft_consensus.cc:740] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Running, Role: LEADER
I20260709 13:12:14.542858  7719 consensus_queue.cc:260] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.543699  7719 raft_consensus.cc:3060] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:14.546422  7646 raft_consensus.cc:2468] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 2.
I20260709 13:12:14.547341  7538 leader_election.cc:304] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:14.547935  7719 raft_consensus.cc:2468] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 2.
I20260709 13:12:14.547950  7818 raft_consensus.cc:2804] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:14.548578  7818 raft_consensus.cc:697] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 LEADER]: Becoming Leader. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:14.549324  7818 consensus_queue.cc:237] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:14.554489  7470 catalog_manager.cc:5697] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 reported cstate change: term changed from 1 to 2, leader changed from 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67) to 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:14.625228  7719 raft_consensus.cc:1275] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Refusing update from remote peer 5e15b6cc1f864cf4ac53f1a8548cd841: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:14.626426  7793 consensus_queue.cc:1048] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:14.633286  7571 raft_consensus.cc:1275] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Refusing update from remote peer 5e15b6cc1f864cf4ac53f1a8548cd841: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:14.634824  7766 consensus_queue.cc:1048] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:14.682715  7571 raft_consensus.cc:1275] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Refusing update from remote peer 19f3607054c14bd88d20be41d80b9b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:14.683764  7849 consensus_queue.cc:1048] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:14.690815  7646 raft_consensus.cc:1275] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Refusing update from remote peer 19f3607054c14bd88d20be41d80b9b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:14.692224  7809 consensus_queue.cc:1048] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:14.791536  7719 raft_consensus.cc:1275] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Refusing update from remote peer 70a4ee1dd69f498d8bfc06683ed1c476: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:14.792601  7818 consensus_queue.cc:1048] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:14.800733  7646 raft_consensus.cc:1275] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Refusing update from remote peer 70a4ee1dd69f498d8bfc06683ed1c476: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:14.802587  7818 consensus_queue.cc:1048] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:14.808318  7571 raft_consensus.cc:1275] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Refusing update from remote peer 5e15b6cc1f864cf4ac53f1a8548cd841: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:14.809526  7766 consensus_queue.cc:1048] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:14.817525  7719 raft_consensus.cc:1275] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Refusing update from remote peer 5e15b6cc1f864cf4ac53f1a8548cd841: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:14.818775  7766 consensus_queue.cc:1048] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:14.961406  7571 raft_consensus.cc:1275] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Refusing update from remote peer 5e15b6cc1f864cf4ac53f1a8548cd841: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:14.962586  7766 consensus_queue.cc:1048] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:14.970141  7719 raft_consensus.cc:1275] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Refusing update from remote peer 5e15b6cc1f864cf4ac53f1a8548cd841: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:14.971729  7766 consensus_queue.cc:1048] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:15.077064  7646 raft_consensus.cc:1275] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Refusing update from remote peer 70a4ee1dd69f498d8bfc06683ed1c476: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:15.078241  7818 consensus_queue.cc:1048] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:15.090308  7719 raft_consensus.cc:1275] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Refusing update from remote peer 70a4ee1dd69f498d8bfc06683ed1c476: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:15.091758  7818 consensus_queue.cc:1048] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:16.265990  7536 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:33409, user_credentials={real_user=slave}} blocked reactor thread for 63746.8us
I20260709 13:12:17.155680  6937 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:12:17.156387  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:17.172302  7571 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "47f8fad043dd499caf70f67c2707d4cb"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
mode: GRACEFUL
new_leader_uuid: "19f3607054c14bd88d20be41d80b9b3b"
 from {username='slave'} at 127.0.0.1:50264
I20260709 13:12:17.172777  7571 raft_consensus.cc:606] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Received request to transfer leadership to TS 19f3607054c14bd88d20be41d80b9b3b
I20260709 13:12:17.175026  7571 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc634f51e1be45a19b4733c75507025a"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
mode: GRACEFUL
new_leader_uuid: "19f3607054c14bd88d20be41d80b9b3b"
 from {username='slave'} at 127.0.0.1:50264
I20260709 13:12:17.175616  7571 raft_consensus.cc:606] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Received request to transfer leadership to TS 19f3607054c14bd88d20be41d80b9b3b
I20260709 13:12:17.176587  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:12:17.177100  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:12:17.254386  7742 debug-util.cc:398] Leaking SignalData structure 0x7b080009ba60 after lost signal to thread 6940
W20260709 13:12:17.255157  7742 debug-util.cc:398] Leaking SignalData structure 0x7b08001ac9c0 after lost signal to thread 7504
W20260709 13:12:17.256464  7742 debug-util.cc:398] Leaking SignalData structure 0x7b080013a820 after lost signal to thread 7671
W20260709 13:12:17.257302  7742 debug-util.cc:398] Leaking SignalData structure 0x7b080019b160 after lost signal to thread 7745
I20260709 13:12:17.306653  7862 raft_consensus.cc:993] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476: : Instructing follower 19f3607054c14bd88d20be41d80b9b3b to start an election
I20260709 13:12:17.307153  7861 raft_consensus.cc:1081] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Signalling peer 19f3607054c14bd88d20be41d80b9b3b to start an election
I20260709 13:12:17.308516  7720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "47f8fad043dd499caf70f67c2707d4cb"
dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
 from {username='slave'} at 127.0.0.1:39286
I20260709 13:12:17.309082  7720 raft_consensus.cc:493] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:17.309474  7720 raft_consensus.cc:3060] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:17.318276  7720 raft_consensus.cc:515] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:17.321059  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47f8fad043dd499caf70f67c2707d4cb" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
I20260709 13:12:17.321281  7720 leader_election.cc:290] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 2 election: Requested vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653)
I20260709 13:12:17.321609  7571 raft_consensus.cc:3055] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:17.322288  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47f8fad043dd499caf70f67c2707d4cb" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:17.322790  7646 raft_consensus.cc:3060] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:17.325924  7571 raft_consensus.cc:740] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:17.326759  7571 consensus_queue.cc:260] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:17.327600  7646 raft_consensus.cc:2468] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19f3607054c14bd88d20be41d80b9b3b in term 2.
I20260709 13:12:17.328012  7571 raft_consensus.cc:3060] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:17.328886  7687 leader_election.cc:304] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 5e15b6cc1f864cf4ac53f1a8548cd841; no voters: 
I20260709 13:12:17.329674  7859 raft_consensus.cc:2804] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:17.333999  7859 raft_consensus.cc:697] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 2 LEADER]: Becoming Leader. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Running, Role: LEADER
I20260709 13:12:17.334904  7859 consensus_queue.cc:237] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:17.335454  7571 raft_consensus.cc:2468] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19f3607054c14bd88d20be41d80b9b3b in term 2.
I20260709 13:12:17.344431  7470 catalog_manager.cc:5697] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b reported cstate change: term changed from 1 to 2, leader changed from 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65) to 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67). New cstate: current_term: 2 leader_uuid: "19f3607054c14bd88d20be41d80b9b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:17.480540  7862 raft_consensus.cc:993] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476: : Instructing follower 19f3607054c14bd88d20be41d80b9b3b to start an election
I20260709 13:12:17.481012  7861 raft_consensus.cc:1081] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Signalling peer 19f3607054c14bd88d20be41d80b9b3b to start an election
I20260709 13:12:17.482430  7720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc634f51e1be45a19b4733c75507025a"
dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
 from {username='slave'} at 127.0.0.1:39286
I20260709 13:12:17.483000  7720 raft_consensus.cc:493] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:17.483392  7720 raft_consensus.cc:3060] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:17.490909  7720 raft_consensus.cc:515] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:17.492951  7720 leader_election.cc:290] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 2 election: Requested vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653)
I20260709 13:12:17.493532  7571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc634f51e1be45a19b4733c75507025a" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
I20260709 13:12:17.493974  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc634f51e1be45a19b4733c75507025a" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:17.494104  7571 raft_consensus.cc:3055] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:17.494571  7571 raft_consensus.cc:740] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:17.494601  7646 raft_consensus.cc:3060] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:17.495379  7571 consensus_queue.cc:260] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:17.496457  7571 raft_consensus.cc:3060] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:17.500806  7571 raft_consensus.cc:2468] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19f3607054c14bd88d20be41d80b9b3b in term 2.
I20260709 13:12:17.501446  7646 raft_consensus.cc:2468] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19f3607054c14bd88d20be41d80b9b3b in term 2.
I20260709 13:12:17.501803  7684 leader_election.cc:304] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:17.502658  7859 raft_consensus.cc:2804] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:17.503043  7859 raft_consensus.cc:697] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 2 LEADER]: Becoming Leader. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Running, Role: LEADER
I20260709 13:12:17.503742  7859 consensus_queue.cc:237] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:17.511307  7470 catalog_manager.cc:5697] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b reported cstate change: term changed from 1 to 2, leader changed from 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65) to 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67). New cstate: current_term: 2 leader_uuid: "19f3607054c14bd88d20be41d80b9b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:17.801923  7646 raft_consensus.cc:1275] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Refusing update from remote peer 19f3607054c14bd88d20be41d80b9b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:17.803052  7859 consensus_queue.cc:1048] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:17.828128  7571 raft_consensus.cc:1275] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Refusing update from remote peer 19f3607054c14bd88d20be41d80b9b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:17.830840  7866 consensus_queue.cc:1048] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:17.888684  7646 raft_consensus.cc:1275] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Refusing update from remote peer 19f3607054c14bd88d20be41d80b9b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:17.889882  7866 consensus_queue.cc:1048] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:17.899753  7571 raft_consensus.cc:1275] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Refusing update from remote peer 19f3607054c14bd88d20be41d80b9b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:17.902313  7859 consensus_queue.cc:1048] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:18.177781  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:18.189808  7720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc634f51e1be45a19b4733c75507025a"
dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
mode: GRACEFUL
new_leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
 from {username='slave'} at 127.0.0.1:39292
I20260709 13:12:18.190310  7720 raft_consensus.cc:606] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 2 LEADER]: Received request to transfer leadership to TS 70a4ee1dd69f498d8bfc06683ed1c476
I20260709 13:12:18.196970  7720 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "47f8fad043dd499caf70f67c2707d4cb"
dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
mode: GRACEFUL
new_leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
 from {username='slave'} at 127.0.0.1:39292
I20260709 13:12:18.197420  7720 raft_consensus.cc:606] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 2 LEADER]: Received request to transfer leadership to TS 70a4ee1dd69f498d8bfc06683ed1c476
I20260709 13:12:18.203992  7646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6"
dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
mode: GRACEFUL
new_leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
 from {username='slave'} at 127.0.0.1:54720
I20260709 13:12:18.204537  7646 raft_consensus.cc:606] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 LEADER]: Received request to transfer leadership to TS 70a4ee1dd69f498d8bfc06683ed1c476
I20260709 13:12:18.208860  7646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21b3171627a4498aab897b3a72b3f0c3"
dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
mode: GRACEFUL
new_leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
 from {username='slave'} at 127.0.0.1:54720
I20260709 13:12:18.209378  7646 raft_consensus.cc:606] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Received request to transfer leadership to TS 70a4ee1dd69f498d8bfc06683ed1c476
I20260709 13:12:18.213661  7646 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ce93d1307cb43ffbf2b5110b26d3a34"
dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
mode: GRACEFUL
new_leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
 from {username='slave'} at 127.0.0.1:54720
I20260709 13:12:18.214175  7646 raft_consensus.cc:606] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 LEADER]: Received request to transfer leadership to TS 70a4ee1dd69f498d8bfc06683ed1c476
I20260709 13:12:18.309985  7881 raft_consensus.cc:993] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841: : Instructing follower 70a4ee1dd69f498d8bfc06683ed1c476 to start an election
I20260709 13:12:18.310431  7860 raft_consensus.cc:1081] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 LEADER]: Signalling peer 70a4ee1dd69f498d8bfc06683ed1c476 to start an election
I20260709 13:12:18.311741  7571 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
 from {username='slave'} at 127.0.0.1:50240
I20260709 13:12:18.312184  7571 raft_consensus.cc:493] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:18.312476  7571 raft_consensus.cc:3060] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:18.316816  7571 raft_consensus.cc:515] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.318266  7571 leader_election.cc:290] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 3 election: Requested vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:18.319077  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:18.319307  7720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:18.319674  7646 raft_consensus.cc:3055] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:18.319855  7720 raft_consensus.cc:3060] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:18.320030  7646 raft_consensus.cc:740] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Running, Role: LEADER
I20260709 13:12:18.320899  7646 consensus_queue.cc:260] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.322072  7646 raft_consensus.cc:3060] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:18.324465  7720 raft_consensus.cc:2468] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 3.
I20260709 13:12:18.325448  7536 leader_election.cc:304] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:18.326087  7861 raft_consensus.cc:2804] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:18.327112  7861 raft_consensus.cc:697] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 LEADER]: Becoming Leader. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:18.327919  7861 consensus_queue.cc:237] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.328723  7646 raft_consensus.cc:2468] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 3.
I20260709 13:12:18.334688  7468 catalog_manager.cc:5697] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 reported cstate change: term changed from 2 to 3, leader changed from 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66) to 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65). New cstate: current_term: 3 leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:18.353153  7859 raft_consensus.cc:993] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b: : Instructing follower 70a4ee1dd69f498d8bfc06683ed1c476 to start an election
I20260709 13:12:18.353582  7859 raft_consensus.cc:1081] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 2 LEADER]: Signalling peer 70a4ee1dd69f498d8bfc06683ed1c476 to start an election
I20260709 13:12:18.355119  7571 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc634f51e1be45a19b4733c75507025a"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
 from {username='slave'} at 127.0.0.1:50252
I20260709 13:12:18.355647  7571 raft_consensus.cc:493] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:18.355986  7571 raft_consensus.cc:3060] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:18.360397  7571 raft_consensus.cc:515] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.361809  7571 leader_election.cc:290] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 3 election: Requested vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:18.362622  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc634f51e1be45a19b4733c75507025a" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:18.362823  7720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc634f51e1be45a19b4733c75507025a" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:18.363176  7646 raft_consensus.cc:3060] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:18.363364  7720 raft_consensus.cc:3055] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:18.363720  7720 raft_consensus.cc:740] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Running, Role: LEADER
I20260709 13:12:18.364344  7720 consensus_queue.cc:260] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.365159  7720 raft_consensus.cc:3060] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:18.369418  7646 raft_consensus.cc:2468] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 3.
I20260709 13:12:18.369441  7720 raft_consensus.cc:2468] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 3.
I20260709 13:12:18.370733  7538 leader_election.cc:304] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:18.371564  7861 raft_consensus.cc:2804] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:18.372025  7861 raft_consensus.cc:697] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 LEADER]: Becoming Leader. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:18.372807  7861 consensus_queue.cc:237] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.379249  7468 catalog_manager.cc:5697] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 reported cstate change: term changed from 2 to 3, leader changed from 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67) to 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65). New cstate: current_term: 3 leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:18.414795  7859 raft_consensus.cc:993] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b: : Instructing follower 70a4ee1dd69f498d8bfc06683ed1c476 to start an election
I20260709 13:12:18.415258  7879 raft_consensus.cc:1081] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 2 LEADER]: Signalling peer 70a4ee1dd69f498d8bfc06683ed1c476 to start an election
I20260709 13:12:18.416810  7571 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "47f8fad043dd499caf70f67c2707d4cb"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
 from {username='slave'} at 127.0.0.1:50252
I20260709 13:12:18.417239  7571 raft_consensus.cc:493] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:18.417510  7571 raft_consensus.cc:3060] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:18.421512  7571 raft_consensus.cc:515] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.423008  7571 leader_election.cc:290] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 3 election: Requested vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:18.424108  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47f8fad043dd499caf70f67c2707d4cb" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:18.424430  7720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47f8fad043dd499caf70f67c2707d4cb" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:18.424635  7646 raft_consensus.cc:3060] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:18.424939  7720 raft_consensus.cc:3055] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:18.425227  7720 raft_consensus.cc:740] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Running, Role: LEADER
I20260709 13:12:18.425964  7720 consensus_queue.cc:260] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.427089  7720 raft_consensus.cc:3060] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:18.430321  7646 raft_consensus.cc:2468] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 3.
I20260709 13:12:18.431308  7538 leader_election.cc:304] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e15b6cc1f864cf4ac53f1a8548cd841, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:18.432081  7861 raft_consensus.cc:2804] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:18.432560  7861 raft_consensus.cc:697] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 LEADER]: Becoming Leader. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:18.433492  7720 raft_consensus.cc:2468] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 3.
I20260709 13:12:18.433373  7861 consensus_queue.cc:237] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.439548  7468 catalog_manager.cc:5697] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 reported cstate change: term changed from 2 to 3, leader changed from 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67) to 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65). New cstate: current_term: 3 leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:18.453138  7881 raft_consensus.cc:993] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841: : Instructing follower 70a4ee1dd69f498d8bfc06683ed1c476 to start an election
I20260709 13:12:18.453533  7860 raft_consensus.cc:1081] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Signalling peer 70a4ee1dd69f498d8bfc06683ed1c476 to start an election
I20260709 13:12:18.454954  7571 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21b3171627a4498aab897b3a72b3f0c3"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
 from {username='slave'} at 127.0.0.1:50240
I20260709 13:12:18.455331  7571 raft_consensus.cc:493] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:18.455600  7571 raft_consensus.cc:3060] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:18.459314  7571 raft_consensus.cc:515] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.461388  7571 leader_election.cc:290] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 2 election: Requested vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:18.461436  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b3171627a4498aab897b3a72b3f0c3" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:18.462297  7646 raft_consensus.cc:3055] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:18.462656  7646 raft_consensus.cc:740] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Running, Role: LEADER
I20260709 13:12:18.462689  7720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b3171627a4498aab897b3a72b3f0c3" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:18.463292  7720 raft_consensus.cc:3060] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:18.463645  7646 consensus_queue.cc:260] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.464725  7646 raft_consensus.cc:3060] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:18.469417  7720 raft_consensus.cc:2468] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 2.
I20260709 13:12:18.470479  7646 raft_consensus.cc:2468] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 2.
I20260709 13:12:18.470458  7536 leader_election.cc:304] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:18.471282  7861 raft_consensus.cc:2804] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:18.471760  7861 raft_consensus.cc:697] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 LEADER]: Becoming Leader. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:18.472573  7861 consensus_queue.cc:237] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.479419  7468 catalog_manager.cc:5697] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 reported cstate change: term changed from 1 to 2, leader changed from 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66) to 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:18.548605  7881 raft_consensus.cc:993] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841: : Instructing follower 70a4ee1dd69f498d8bfc06683ed1c476 to start an election
I20260709 13:12:18.549032  7881 raft_consensus.cc:1081] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 LEADER]: Signalling peer 70a4ee1dd69f498d8bfc06683ed1c476 to start an election
I20260709 13:12:18.550552  7571 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ce93d1307cb43ffbf2b5110b26d3a34"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
 from {username='slave'} at 127.0.0.1:50240
I20260709 13:12:18.551005  7571 raft_consensus.cc:493] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:18.551275  7571 raft_consensus.cc:3060] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:18.555053  7571 raft_consensus.cc:515] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.556347  7571 leader_election.cc:290] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 3 election: Requested vote from peers 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:18.557233  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce93d1307cb43ffbf2b5110b26d3a34" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:18.557384  7720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ce93d1307cb43ffbf2b5110b26d3a34" candidate_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:18.557771  7646 raft_consensus.cc:3055] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:18.557960  7720 raft_consensus.cc:3060] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:18.558107  7646 raft_consensus.cc:740] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Running, Role: LEADER
I20260709 13:12:18.558895  7646 consensus_queue.cc:260] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.560000  7646 raft_consensus.cc:3060] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:18.562433  7720 raft_consensus.cc:2468] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 3.
I20260709 13:12:18.563561  7536 leader_election.cc:304] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 70a4ee1dd69f498d8bfc06683ed1c476; no voters: 
I20260709 13:12:18.564307  7882 raft_consensus.cc:2804] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:18.566071  7882 raft_consensus.cc:697] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 LEADER]: Becoming Leader. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:18.566846  7882 consensus_queue.cc:237] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:18.567624  7646 raft_consensus.cc:2468] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a4ee1dd69f498d8bfc06683ed1c476 in term 3.
I20260709 13:12:18.574123  7468 catalog_manager.cc:5697] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 reported cstate change: term changed from 2 to 3, leader changed from 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66) to 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65). New cstate: current_term: 3 leader_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:18.763051  7646 raft_consensus.cc:1275] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Refusing update from remote peer 70a4ee1dd69f498d8bfc06683ed1c476: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:18.764168  7882 consensus_queue.cc:1048] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:18.775472  7720 raft_consensus.cc:1275] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Refusing update from remote peer 70a4ee1dd69f498d8bfc06683ed1c476: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:18.777069  7888 consensus_queue.cc:1048] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:18.896355  7646 raft_consensus.cc:1275] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Refusing update from remote peer 70a4ee1dd69f498d8bfc06683ed1c476: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:18.897610  7882 consensus_queue.cc:1048] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:18.906178  7720 raft_consensus.cc:1275] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Refusing update from remote peer 70a4ee1dd69f498d8bfc06683ed1c476: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:18.908217  7895 consensus_queue.cc:1048] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:18.980285  7646 raft_consensus.cc:1275] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Refusing update from remote peer 70a4ee1dd69f498d8bfc06683ed1c476: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:18.981755  7895 consensus_queue.cc:1048] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:18.982121  7720 raft_consensus.cc:1275] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Refusing update from remote peer 70a4ee1dd69f498d8bfc06683ed1c476: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:18.983737  7895 consensus_queue.cc:1048] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:18.991866  7721 raft_consensus.cc:1275] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Refusing update from remote peer 70a4ee1dd69f498d8bfc06683ed1c476: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:18.995034  7882 consensus_queue.cc:1048] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:19.002065  7647 raft_consensus.cc:1275] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Refusing update from remote peer 70a4ee1dd69f498d8bfc06683ed1c476: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:19.003445  7861 consensus_queue.cc:1048] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:19.017360  7647 raft_consensus.cc:1275] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Refusing update from remote peer 70a4ee1dd69f498d8bfc06683ed1c476: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:19.019109  7861 consensus_queue.cc:1048] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:19.034988  7720 raft_consensus.cc:1275] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Refusing update from remote peer 70a4ee1dd69f498d8bfc06683ed1c476: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:19.036345  7895 consensus_queue.cc:1048] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:21.220752  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:21.228209  7572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "214a25d9d6b14349b684d78df67114e3"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
mode: GRACEFUL
new_leader_uuid: "19f3607054c14bd88d20be41d80b9b3b"
 from {username='slave'} at 127.0.0.1:50264
I20260709 13:12:21.229004  7572 raft_consensus.cc:606] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Received request to transfer leadership to TS 19f3607054c14bd88d20be41d80b9b3b
I20260709 13:12:21.231724  7572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62ed00cf429347c9b053918ef1f89678"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
mode: GRACEFUL
new_leader_uuid: "19f3607054c14bd88d20be41d80b9b3b"
 from {username='slave'} at 127.0.0.1:50264
I20260709 13:12:21.232200  7572 raft_consensus.cc:606] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 LEADER]: Received request to transfer leadership to TS 19f3607054c14bd88d20be41d80b9b3b
I20260709 13:12:21.234614  7572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
mode: GRACEFUL
new_leader_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
 from {username='slave'} at 127.0.0.1:50264
I20260709 13:12:21.235440  7572 raft_consensus.cc:606] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 LEADER]: Received request to transfer leadership to TS 5e15b6cc1f864cf4ac53f1a8548cd841
I20260709 13:12:21.238062  7572 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a95e66c97a08430c9b40ea701e97dc17"
dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
mode: GRACEFUL
new_leader_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
 from {username='slave'} at 127.0.0.1:50264
I20260709 13:12:21.238606  7572 raft_consensus.cc:606] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 LEADER]: Received request to transfer leadership to TS 5e15b6cc1f864cf4ac53f1a8548cd841
I20260709 13:12:21.239756  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:12:21.240259  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:21.312407  7895 raft_consensus.cc:993] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476: : Instructing follower 5e15b6cc1f864cf4ac53f1a8548cd841 to start an election
I20260709 13:12:21.312852  7895 raft_consensus.cc:1081] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 LEADER]: Signalling peer 5e15b6cc1f864cf4ac53f1a8548cd841 to start an election
I20260709 13:12:21.314404  7646 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6"
dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
 from {username='slave'} at 127.0.0.1:54702
I20260709 13:12:21.314899  7646 raft_consensus.cc:493] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:21.315192  7646 raft_consensus.cc:3060] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:21.320490  7646 raft_consensus.cc:515] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:21.322194  7646 leader_election.cc:290] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 4 election: Requested vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409)
I20260709 13:12:21.323154  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
I20260709 13:12:21.323555  7720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d390c1367c84b9b9f46eabe0aed2db6" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:21.323966  7572 raft_consensus.cc:3055] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:21.324046  7720 raft_consensus.cc:3060] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:21.324237  7572 raft_consensus.cc:740] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:21.325096  7572 consensus_queue.cc:260] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:21.326069  7572 raft_consensus.cc:3060] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:21.329877  7720 raft_consensus.cc:2468] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 4.
I20260709 13:12:21.331602  7611 leader_election.cc:304] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 5e15b6cc1f864cf4ac53f1a8548cd841; no voters: 
I20260709 13:12:21.336449  7906 raft_consensus.cc:2804] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:21.339906  7572 raft_consensus.cc:2468] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 4.
I20260709 13:12:21.341845  7906 raft_consensus.cc:697] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 4 LEADER]: Becoming Leader. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Running, Role: LEADER
I20260709 13:12:21.342937  7906 consensus_queue.cc:237] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:21.351918  7468 catalog_manager.cc:5697] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 reported cstate change: term changed from 3 to 4, leader changed from 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65) to 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66). New cstate: current_term: 4 leader_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:21.415354  7905 raft_consensus.cc:993] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476: : Instructing follower 5e15b6cc1f864cf4ac53f1a8548cd841 to start an election
I20260709 13:12:21.415808  7895 raft_consensus.cc:1081] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 LEADER]: Signalling peer 5e15b6cc1f864cf4ac53f1a8548cd841 to start an election
I20260709 13:12:21.417419  7646 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a95e66c97a08430c9b40ea701e97dc17"
dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
 from {username='slave'} at 127.0.0.1:54702
I20260709 13:12:21.418154  7646 raft_consensus.cc:493] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:21.418591  7646 raft_consensus.cc:3060] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:21.422927  7646 raft_consensus.cc:515] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:21.424453  7646 leader_election.cc:290] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 3 election: Requested vote from peers 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67:33409), 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351)
I20260709 13:12:21.425417  7720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95e66c97a08430c9b40ea701e97dc17" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
I20260709 13:12:21.425575  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a95e66c97a08430c9b40ea701e97dc17" candidate_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
I20260709 13:12:21.426023  7720 raft_consensus.cc:3060] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:21.426249  7572 raft_consensus.cc:3055] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:21.426658  7572 raft_consensus.cc:740] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:21.427399  7572 consensus_queue.cc:260] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:21.428359  7572 raft_consensus.cc:3060] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:21.430896  7720 raft_consensus.cc:2468] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 3.
I20260709 13:12:21.431860  7611 leader_election.cc:304] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 5e15b6cc1f864cf4ac53f1a8548cd841; no voters: 
I20260709 13:12:21.432657  7906 raft_consensus.cc:2804] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:21.433084  7906 raft_consensus.cc:697] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 LEADER]: Becoming Leader. State: Replica: 5e15b6cc1f864cf4ac53f1a8548cd841, State: Running, Role: LEADER
I20260709 13:12:21.433641  7572 raft_consensus.cc:2468] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e15b6cc1f864cf4ac53f1a8548cd841 in term 3.
I20260709 13:12:21.433964  7906 consensus_queue.cc:237] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } }
I20260709 13:12:21.441340  7468 catalog_manager.cc:5697] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 reported cstate change: term changed from 2 to 3, leader changed from 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65) to 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66). New cstate: current_term: 3 leader_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:21.478461  7895 raft_consensus.cc:993] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476: : Instructing follower 19f3607054c14bd88d20be41d80b9b3b to start an election
I20260709 13:12:21.479023  7905 raft_consensus.cc:1081] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 LEADER]: Signalling peer 19f3607054c14bd88d20be41d80b9b3b to start an election
I20260709 13:12:21.480650  7720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62ed00cf429347c9b053918ef1f89678"
dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
 from {username='slave'} at 127.0.0.1:39286
I20260709 13:12:21.481106  7720 raft_consensus.cc:493] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:21.481484  7720 raft_consensus.cc:3060] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:21.485769  7720 raft_consensus.cc:515] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:21.487574  7720 leader_election.cc:290] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 3 election: Requested vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653)
I20260709 13:12:21.488253  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ed00cf429347c9b053918ef1f89678" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
I20260709 13:12:21.488571  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ed00cf429347c9b053918ef1f89678" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:21.488834  7572 raft_consensus.cc:3055] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:21.489076  7646 raft_consensus.cc:3060] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:21.489220  7572 raft_consensus.cc:740] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:21.490042  7572 consensus_queue.cc:260] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:21.491118  7572 raft_consensus.cc:3060] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:21.493542  7646 raft_consensus.cc:2468] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19f3607054c14bd88d20be41d80b9b3b in term 3.
I20260709 13:12:21.494710  7687 leader_election.cc:304] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 5e15b6cc1f864cf4ac53f1a8548cd841; no voters: 
I20260709 13:12:21.495648  7879 raft_consensus.cc:2804] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:21.496035  7572 raft_consensus.cc:2468] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19f3607054c14bd88d20be41d80b9b3b in term 3.
I20260709 13:12:21.497107  7879 raft_consensus.cc:697] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 3 LEADER]: Becoming Leader. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Running, Role: LEADER
I20260709 13:12:21.498286  7879 consensus_queue.cc:237] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:21.499938  7895 raft_consensus.cc:993] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476: : Instructing follower 19f3607054c14bd88d20be41d80b9b3b to start an election
I20260709 13:12:21.500412  7905 raft_consensus.cc:1081] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Signalling peer 19f3607054c14bd88d20be41d80b9b3b to start an election
I20260709 13:12:21.502080  7720 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "214a25d9d6b14349b684d78df67114e3"
dest_uuid: "19f3607054c14bd88d20be41d80b9b3b"
 from {username='slave'} at 127.0.0.1:39286
I20260709 13:12:21.502578  7720 raft_consensus.cc:493] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:21.502986  7720 raft_consensus.cc:3060] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:21.506911  7468 catalog_manager.cc:5697] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b reported cstate change: term changed from 2 to 3, leader changed from 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65) to 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67). New cstate: current_term: 3 leader_uuid: "19f3607054c14bd88d20be41d80b9b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:21.507939  7720 raft_consensus.cc:515] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:21.509845  7720 leader_election.cc:290] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 2 election: Requested vote from peers 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351), 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653)
I20260709 13:12:21.510150  7572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a25d9d6b14349b684d78df67114e3" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "70a4ee1dd69f498d8bfc06683ed1c476"
I20260709 13:12:21.510473  7646 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214a25d9d6b14349b684d78df67114e3" candidate_uuid: "19f3607054c14bd88d20be41d80b9b3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841"
I20260709 13:12:21.510743  7572 raft_consensus.cc:3055] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:21.510838  7646 raft_consensus.cc:3060] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:21.511152  7572 raft_consensus.cc:740] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 70a4ee1dd69f498d8bfc06683ed1c476, State: Running, Role: LEADER
I20260709 13:12:21.511948  7572 consensus_queue.cc:260] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:21.513375  7572 raft_consensus.cc:3060] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:21.519345  7646 raft_consensus.cc:2468] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19f3607054c14bd88d20be41d80b9b3b in term 2.
I20260709 13:12:21.520500  7687 leader_election.cc:304] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19f3607054c14bd88d20be41d80b9b3b, 5e15b6cc1f864cf4ac53f1a8548cd841; no voters: 
I20260709 13:12:21.521214  7879 raft_consensus.cc:2804] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:21.521469  7572 raft_consensus.cc:2468] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19f3607054c14bd88d20be41d80b9b3b in term 2.
I20260709 13:12:21.521955  7879 raft_consensus.cc:697] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [term 2 LEADER]: Becoming Leader. State: Replica: 19f3607054c14bd88d20be41d80b9b3b, State: Running, Role: LEADER
I20260709 13:12:21.523000  7879 consensus_queue.cc:237] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } }
I20260709 13:12:21.530443  7468 catalog_manager.cc:5697] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b reported cstate change: term changed from 1 to 2, leader changed from 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65) to 19f3607054c14bd88d20be41d80b9b3b (127.6.198.67). New cstate: current_term: 2 leader_uuid: "19f3607054c14bd88d20be41d80b9b3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:21.861049  7572 raft_consensus.cc:1275] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 4 FOLLOWER]: Refusing update from remote peer 5e15b6cc1f864cf4ac53f1a8548cd841: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:21.862258  7906 consensus_queue.cc:1048] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:21.875054  7720 raft_consensus.cc:1275] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 4 FOLLOWER]: Refusing update from remote peer 5e15b6cc1f864cf4ac53f1a8548cd841: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:12:21.877189  7907 consensus_queue.cc:1048] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:21.900099  7646 raft_consensus.cc:1275] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Refusing update from remote peer 19f3607054c14bd88d20be41d80b9b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:21.901569  7879 consensus_queue.cc:1048] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:21.911521  7572 raft_consensus.cc:1275] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Refusing update from remote peer 19f3607054c14bd88d20be41d80b9b3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:21.915350  7879 consensus_queue.cc:1048] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:21.949098  7720 raft_consensus.cc:1275] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Refusing update from remote peer 5e15b6cc1f864cf4ac53f1a8548cd841: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:21.951804  7907 consensus_queue.cc:1048] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19f3607054c14bd88d20be41d80b9b3b" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:21.965113  7572 raft_consensus.cc:1275] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Refusing update from remote peer 5e15b6cc1f864cf4ac53f1a8548cd841: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:21.966840  7906 consensus_queue.cc:1048] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:22.009014  7646 raft_consensus.cc:1275] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Refusing update from remote peer 19f3607054c14bd88d20be41d80b9b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:22.011206  7879 consensus_queue.cc:1048] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e15b6cc1f864cf4ac53f1a8548cd841" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:22.022847  7571 raft_consensus.cc:1275] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Refusing update from remote peer 19f3607054c14bd88d20be41d80b9b3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:22.025728  7915 consensus_queue.cc:1048] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a4ee1dd69f498d8bfc06683ed1c476" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 36351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:22.241217  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:22.247181  6937 tablet_server.cc:179] TabletServer@127.6.198.65:0 shutting down...
I20260709 13:12:22.286331  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:22.287437  6937 tablet_replica.cc:333] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476: stopping tablet replica
I20260709 13:12:22.288488  6937 raft_consensus.cc:2243] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.288926  6937 raft_consensus.cc:2272] T 6d390c1367c84b9b9f46eabe0aed2db6 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.294581  6937 tablet_replica.cc:333] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476: stopping tablet replica
I20260709 13:12:22.295245  6937 raft_consensus.cc:2243] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:22.296768  6937 raft_consensus.cc:2272] T fc634f51e1be45a19b4733c75507025a P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.299067  6937 tablet_replica.cc:333] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476: stopping tablet replica
I20260709 13:12:22.299640  6937 raft_consensus.cc:2243] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.300071  6937 raft_consensus.cc:2272] T 214a25d9d6b14349b684d78df67114e3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.302882  6937 tablet_replica.cc:333] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476: stopping tablet replica
I20260709 13:12:22.303486  6937 raft_consensus.cc:2243] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.303882  6937 raft_consensus.cc:2272] T 5b4479fdd61f4dbda42f65f064a04e70 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.305609  6937 tablet_replica.cc:333] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476: stopping tablet replica
I20260709 13:12:22.306119  6937 raft_consensus.cc:2243] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:22.306869  6937 raft_consensus.cc:2272] T 21b3171627a4498aab897b3a72b3f0c3 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.308518  6937 tablet_replica.cc:333] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476: stopping tablet replica
I20260709 13:12:22.308975  6937 raft_consensus.cc:2243] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:22.309695  6937 raft_consensus.cc:2272] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.312372  6937 tablet_replica.cc:333] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476: stopping tablet replica
I20260709 13:12:22.312888  6937 raft_consensus.cc:2243] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.313292  6937 raft_consensus.cc:2272] T 62ed00cf429347c9b053918ef1f89678 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.315407  6937 tablet_replica.cc:333] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476: stopping tablet replica
I20260709 13:12:22.315882  6937 raft_consensus.cc:2243] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.316295  6937 raft_consensus.cc:2272] T a95e66c97a08430c9b40ea701e97dc17 P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.319350  6937 tablet_replica.cc:333] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476: stopping tablet replica
I20260709 13:12:22.320161  6937 raft_consensus.cc:2243] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.320988  6937 raft_consensus.cc:2272] T 116b0660221643538da695f3d8d4595c P 70a4ee1dd69f498d8bfc06683ed1c476 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.322935  6937 tablet_replica.cc:333] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476: stopping tablet replica
I20260709 13:12:22.323427  6937 raft_consensus.cc:2243] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:22.324160  6937 raft_consensus.cc:2272] T 47f8fad043dd499caf70f67c2707d4cb P 70a4ee1dd69f498d8bfc06683ed1c476 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:22.357283  7610 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.198.65:36351: connect: Connection refused (error 111) [suppressed 3 similar messages]
I20260709 13:12:22.360461  6937 tablet_server.cc:196] TabletServer@127.6.198.65:0 shutdown complete.
W20260709 13:12:22.362776  7610 consensus_peers.cc:597] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 -> Peer 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351): Couldn't send request to peer 70a4ee1dd69f498d8bfc06683ed1c476. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:36351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:22.382745  6937 tablet_server.cc:179] TabletServer@127.6.198.66:0 shutting down...
I20260709 13:12:22.416898  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:22.417950  6937 tablet_replica.cc:333] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841: stopping tablet replica
I20260709 13:12:22.418704  6937 raft_consensus.cc:2243] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.419178  6937 raft_consensus.cc:2272] T 214a25d9d6b14349b684d78df67114e3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.421432  6937 tablet_replica.cc:333] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841: stopping tablet replica
I20260709 13:12:22.421989  6937 raft_consensus.cc:2243] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:12:22.422802  6937 raft_consensus.cc:2272] T 6d390c1367c84b9b9f46eabe0aed2db6 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.425285  6937 tablet_replica.cc:333] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841: stopping tablet replica
I20260709 13:12:22.425819  6937 raft_consensus.cc:2243] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.426417  6937 raft_consensus.cc:2272] T 5b4479fdd61f4dbda42f65f064a04e70 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.433382  6937 tablet_replica.cc:333] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841: stopping tablet replica
I20260709 13:12:22.434103  6937 raft_consensus.cc:2243] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.435708  6937 raft_consensus.cc:2272] T fc634f51e1be45a19b4733c75507025a P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.439520  6937 tablet_replica.cc:333] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841: stopping tablet replica
I20260709 13:12:22.440387  6937 raft_consensus.cc:2243] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.441057  6937 raft_consensus.cc:2272] T 47f8fad043dd499caf70f67c2707d4cb P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.443739  6937 tablet_replica.cc:333] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841: stopping tablet replica
I20260709 13:12:22.444435  6937 raft_consensus.cc:2243] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:22.445235  6937 raft_consensus.cc:2272] T 116b0660221643538da695f3d8d4595c P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.449853  6937 tablet_replica.cc:333] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841: stopping tablet replica
I20260709 13:12:22.453164  6937 raft_consensus.cc:2243] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:22.456761  6937 raft_consensus.cc:2272] T a95e66c97a08430c9b40ea701e97dc17 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.459784  6937 tablet_replica.cc:333] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841: stopping tablet replica
I20260709 13:12:22.460398  6937 raft_consensus.cc:2243] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.460865  6937 raft_consensus.cc:2272] T 62ed00cf429347c9b053918ef1f89678 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:22.461961  7687 consensus_peers.cc:597] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b -> Peer 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653): Couldn't send request to peer 5e15b6cc1f864cf4ac53f1a8548cd841. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:22.464087  6937 tablet_replica.cc:333] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841: stopping tablet replica
I20260709 13:12:22.464793  6937 raft_consensus.cc:2243] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.465358  6937 raft_consensus.cc:2272] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.467095  6937 tablet_replica.cc:333] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841: stopping tablet replica
I20260709 13:12:22.469463  6937 raft_consensus.cc:2243] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.470225  6937 raft_consensus.cc:2272] T 21b3171627a4498aab897b3a72b3f0c3 P 5e15b6cc1f864cf4ac53f1a8548cd841 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.499913  6937 tablet_server.cc:196] TabletServer@127.6.198.66:0 shutdown complete.
W20260709 13:12:22.501675  7684 consensus_peers.cc:597] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b -> Peer 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351): Couldn't send request to peer 70a4ee1dd69f498d8bfc06683ed1c476. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:36351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:22.523931  6937 tablet_server.cc:179] TabletServer@127.6.198.67:0 shutting down...
W20260709 13:12:22.542176  7684 consensus_peers.cc:597] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b -> Peer 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351): Couldn't send request to peer 70a4ee1dd69f498d8bfc06683ed1c476. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:36351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:22.547598  7687 consensus_peers.cc:597] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b -> Peer 5e15b6cc1f864cf4ac53f1a8548cd841 (127.6.198.66:38653): Couldn't send request to peer 5e15b6cc1f864cf4ac53f1a8548cd841. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:38653: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:22.561702  7684 consensus_peers.cc:597] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b -> Peer 70a4ee1dd69f498d8bfc06683ed1c476 (127.6.198.65:36351): Couldn't send request to peer 70a4ee1dd69f498d8bfc06683ed1c476. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:36351: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:22.569387  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:22.570192  6937 tablet_replica.cc:333] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b: stopping tablet replica
I20260709 13:12:22.571062  6937 raft_consensus.cc:2243] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:22.571985  6937 raft_consensus.cc:2272] T 5b4479fdd61f4dbda42f65f064a04e70 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.574049  6937 tablet_replica.cc:333] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b: stopping tablet replica
I20260709 13:12:22.574678  6937 raft_consensus.cc:2243] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.575078  6937 raft_consensus.cc:2272] T 116b0660221643538da695f3d8d4595c P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.576853  6937 tablet_replica.cc:333] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b: stopping tablet replica
I20260709 13:12:22.577347  6937 raft_consensus.cc:2243] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.577795  6937 raft_consensus.cc:2272] T 47f8fad043dd499caf70f67c2707d4cb P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.579650  6937 tablet_replica.cc:333] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b: stopping tablet replica
I20260709 13:12:22.580474  6937 raft_consensus.cc:2243] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:22.581634  6937 raft_consensus.cc:2272] T 214a25d9d6b14349b684d78df67114e3 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.584393  6937 tablet_replica.cc:333] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b: stopping tablet replica
I20260709 13:12:22.585000  6937 raft_consensus.cc:2243] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.585457  6937 raft_consensus.cc:2272] T a95e66c97a08430c9b40ea701e97dc17 P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.588199  6937 tablet_replica.cc:333] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b: stopping tablet replica
I20260709 13:12:22.588745  6937 raft_consensus.cc:2243] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:22.590029  6937 raft_consensus.cc:2272] T 62ed00cf429347c9b053918ef1f89678 P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.596254  6937 tablet_replica.cc:333] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b: stopping tablet replica
I20260709 13:12:22.597296  6937 raft_consensus.cc:2243] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.597990  6937 raft_consensus.cc:2272] T fc634f51e1be45a19b4733c75507025a P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.599893  6937 tablet_replica.cc:333] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b: stopping tablet replica
I20260709 13:12:22.600405  6937 raft_consensus.cc:2243] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.600781  6937 raft_consensus.cc:2272] T 6d390c1367c84b9b9f46eabe0aed2db6 P 19f3607054c14bd88d20be41d80b9b3b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.602964  6937 tablet_replica.cc:333] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b: stopping tablet replica
I20260709 13:12:22.603482  6937 raft_consensus.cc:2243] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.603884  6937 raft_consensus.cc:2272] T 1ce93d1307cb43ffbf2b5110b26d3a34 P 19f3607054c14bd88d20be41d80b9b3b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.605489  6937 tablet_replica.cc:333] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b: stopping tablet replica
I20260709 13:12:22.605919  6937 raft_consensus.cc:2243] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:22.606312  6937 raft_consensus.cc:2272] T 21b3171627a4498aab897b3a72b3f0c3 P 19f3607054c14bd88d20be41d80b9b3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.657598  6937 tablet_server.cc:196] TabletServer@127.6.198.67:0 shutdown complete.
I20260709 13:12:22.678138  6937 master.cc:562] Master@127.6.198.126:45895 shutting down...
I20260709 13:12:22.718174  6937 raft_consensus.cc:2243] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:22.718827  6937 raft_consensus.cc:2272] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:22.719177  6937 tablet_replica.cc:333] T 00000000000000000000000000000000 P acf6aea4f76643a385cf939478cb7631: stopping tablet replica
I20260709 13:12:22.740286  6937 master.cc:584] Master@127.6.198.126:45895 shutdown complete.
19f3607054c14bd88d20be41d80b9b3b  1
5e15b6cc1f864cf4ac53f1a8548cd841  4
70a4ee1dd69f498d8bfc06683ed1c476  5
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10720 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:12:22.788827  6937 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.198.126:38543
I20260709 13:12:22.789934  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:22.796065  7936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:22.796021  7937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:22.796959  7939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:22.797638  6937 server_base.cc:1061] running on GCE node
I20260709 13:12:22.798532  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:22.798707  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:22.798880  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602742798866 us; error 0 us; skew 500 ppm
I20260709 13:12:22.799419  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:22.801803  6937 webserver.cc:533] Webserver started at http://127.6.198.126:43467/ using document root <none> and password file <none>
I20260709 13:12:22.802304  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:22.802529  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:22.802863  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:22.804024  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/master-0-root/instance:
uuid: "e3b771f7515a4bb7beb66bc429c57201"
format_stamp: "Formatted at 2026-07-09 13:12:22 on dist-test-slave-tqzw"
I20260709 13:12:22.808616  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.005s
I20260709 13:12:22.812453  7944 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:22.813328  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:12:22.813611  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/master-0-root
uuid: "e3b771f7515a4bb7beb66bc429c57201"
format_stamp: "Formatted at 2026-07-09 13:12:22 on dist-test-slave-tqzw"
I20260709 13:12:22.813895  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:22.843528  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:22.844696  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:22.905777  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.126:38543
I20260709 13:12:22.905890  7995 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.126:38543 every 8 connection(s)
I20260709 13:12:22.913697  7996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:22.925222  7996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Bootstrap starting.
I20260709 13:12:22.937440  7996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:22.941900  7996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: No bootstrap required, opened a new log
I20260709 13:12:22.944134  7996 raft_consensus.cc:359] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b771f7515a4bb7beb66bc429c57201" member_type: VOTER }
I20260709 13:12:22.944497  7996 raft_consensus.cc:385] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:22.944741  7996 raft_consensus.cc:740] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3b771f7515a4bb7beb66bc429c57201, State: Initialized, Role: FOLLOWER
I20260709 13:12:22.945343  7996 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b771f7515a4bb7beb66bc429c57201" member_type: VOTER }
I20260709 13:12:22.945778  7996 raft_consensus.cc:399] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:12:22.946002  7996 raft_consensus.cc:493] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:12:22.946269  7996 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:22.955021  7996 raft_consensus.cc:515] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b771f7515a4bb7beb66bc429c57201" member_type: VOTER }
I20260709 13:12:22.955852  7996 leader_election.cc:304] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3b771f7515a4bb7beb66bc429c57201; no voters: 
I20260709 13:12:22.957001  7996 leader_election.cc:290] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:12:22.957309  7999 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:22.958606  7999 raft_consensus.cc:697] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [term 1 LEADER]: Becoming Leader. State: Replica: e3b771f7515a4bb7beb66bc429c57201, State: Running, Role: LEADER
I20260709 13:12:22.959259  7999 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b771f7515a4bb7beb66bc429c57201" member_type: VOTER }
I20260709 13:12:22.959923  7996 sys_catalog.cc:565] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:12:22.963697  8000 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3b771f7515a4bb7beb66bc429c57201" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b771f7515a4bb7beb66bc429c57201" member_type: VOTER } }
I20260709 13:12:22.964013  8001 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3b771f7515a4bb7beb66bc429c57201. Latest consensus state: current_term: 1 leader_uuid: "e3b771f7515a4bb7beb66bc429c57201" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3b771f7515a4bb7beb66bc429c57201" member_type: VOTER } }
I20260709 13:12:22.964298  8000 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:22.964476  8001 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:22.967288  8006 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:12:22.975842  8006 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:12:22.976785  6937 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:12:22.984318  8006 catalog_manager.cc:1383] Generated new cluster ID: 249e716321a9434c9573ba70d1599a4f
I20260709 13:12:22.984587  8006 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:12:23.008637  8006 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:12:23.009958  8006 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:12:23.021538  8006 catalog_manager.cc:6070] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Generated new TSK 0
I20260709 13:12:23.022199  8006 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:12:23.043032  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:23.049678  8017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:23.050153  8018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:23.052016  8020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:23.053241  6937 server_base.cc:1061] running on GCE node
I20260709 13:12:23.054190  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:23.054466  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:23.054653  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602743054629 us; error 0 us; skew 500 ppm
I20260709 13:12:23.055282  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:23.057895  6937 webserver.cc:533] Webserver started at http://127.6.198.65:41327/ using document root <none> and password file <none>
I20260709 13:12:23.058635  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:23.058818  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:23.059166  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:23.060428  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/instance:
uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
format_stamp: "Formatted at 2026-07-09 13:12:23 on dist-test-slave-tqzw"
I20260709 13:12:23.065094  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:12:23.068799  8025 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:23.070039  6937 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:12:23.072019  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root
uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
format_stamp: "Formatted at 2026-07-09 13:12:23 on dist-test-slave-tqzw"
I20260709 13:12:23.073704  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:23.083482  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:23.084730  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:23.086241  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:23.091660  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:23.091941  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:23.092913  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:23.094105  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20260709 13:12:23.147075  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.65:45919
I20260709 13:12:23.147199  8087 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.65:45919 every 8 connection(s)
I20260709 13:12:23.152731  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:23.160169  8092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:23.161046  8093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:23.164379  8095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:23.164691  6937 server_base.cc:1061] running on GCE node
I20260709 13:12:23.165741  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:23.166000  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:23.166205  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602743166184 us; error 0 us; skew 500 ppm
I20260709 13:12:23.166568  8088 heartbeater.cc:344] Connected to a master server at 127.6.198.126:38543
I20260709 13:12:23.166857  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:23.166904  8088 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:23.167860  8088 heartbeater.cc:507] Master 127.6.198.126:38543 requested a full tablet report, sending...
I20260709 13:12:23.169622  6937 webserver.cc:533] Webserver started at http://127.6.198.66:36533/ using document root <none> and password file <none>
I20260709 13:12:23.170817  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:23.170779  7961 ts_manager.cc:194] Registered new tserver with Master: 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
I20260709 13:12:23.173024  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:23.175360  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:23.177170  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/instance:
uuid: "8a968cada1e4446ebb2a5e167176e649"
format_stamp: "Formatted at 2026-07-09 13:12:23 on dist-test-slave-tqzw"
I20260709 13:12:23.177352  7961 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34120
I20260709 13:12:23.182250  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.006s
I20260709 13:12:23.185892  8100 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:23.187757  6937 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:12:23.188037  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root
uuid: "8a968cada1e4446ebb2a5e167176e649"
format_stamp: "Formatted at 2026-07-09 13:12:23 on dist-test-slave-tqzw"
I20260709 13:12:23.188287  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:23.202541  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:23.203720  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:23.205137  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:23.207476  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:23.207662  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:23.207953  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:23.208120  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:23.281020  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.66:45807
I20260709 13:12:23.281118  8162 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.66:45807 every 8 connection(s)
I20260709 13:12:23.285789  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:23.298133  8166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:23.299830  8167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:23.300385  8163 heartbeater.cc:344] Connected to a master server at 127.6.198.126:38543
I20260709 13:12:23.300868  8163 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:23.301688  8163 heartbeater.cc:507] Master 127.6.198.126:38543 requested a full tablet report, sending...
I20260709 13:12:23.302865  6937 server_base.cc:1061] running on GCE node
W20260709 13:12:23.303587  8170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:23.304195  7961 ts_manager.cc:194] Registered new tserver with Master: 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:23.304601  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:23.304826  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:23.305008  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602743304975 us; error 0 us; skew 500 ppm
I20260709 13:12:23.305575  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:23.305639  7961 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34134
I20260709 13:12:23.308480  6937 webserver.cc:533] Webserver started at http://127.6.198.67:33493/ using document root <none> and password file <none>
I20260709 13:12:23.309582  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:23.310124  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:23.310447  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:23.312077  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/instance:
uuid: "d9cd2999aa314b39997b423ec1ae509e"
format_stamp: "Formatted at 2026-07-09 13:12:23 on dist-test-slave-tqzw"
I20260709 13:12:23.317996  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.005s
I20260709 13:12:23.321382  8174 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:23.322217  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:12:23.322827  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root
uuid: "d9cd2999aa314b39997b423ec1ae509e"
format_stamp: "Formatted at 2026-07-09 13:12:23 on dist-test-slave-tqzw"
I20260709 13:12:23.323256  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:23.340042  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:23.341413  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:23.342978  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:23.345520  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:23.345752  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:23.345988  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:23.346163  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:23.406247  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.67:35745
I20260709 13:12:23.406422  8236 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.67:35745 every 8 connection(s)
I20260709 13:12:23.422098  8237 heartbeater.cc:344] Connected to a master server at 127.6.198.126:38543
I20260709 13:12:23.422478  8237 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:23.423293  8237 heartbeater.cc:507] Master 127.6.198.126:38543 requested a full tablet report, sending...
I20260709 13:12:23.425997  7961 ts_manager.cc:194] Registered new tserver with Master: d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:23.426486  6937 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016749652s
I20260709 13:12:23.428921  7961 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34146
I20260709 13:12:23.466146  7961 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34154:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:12:23.475204  7961 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:12:23.714093  8202 tablet_service.cc:1511] Processing CreateTablet for tablet 5124ddc96f984ec2ae0ee97be6732a90 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:12:23.714802  8201 tablet_service.cc:1511] Processing CreateTablet for tablet 1f229a5eb4514abc9183538dd97073d2 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:12:23.715531  8202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5124ddc96f984ec2ae0ee97be6732a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.716181  8201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f229a5eb4514abc9183538dd97073d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.716313  8200 tablet_service.cc:1511] Processing CreateTablet for tablet cc8454910b36474c90b678c57cea9773 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:12:23.717633  8200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc8454910b36474c90b678c57cea9773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.719540  8199 tablet_service.cc:1511] Processing CreateTablet for tablet 9871516a41a2419d8af8e6fe9e0a5fa1 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:12:23.720861  8199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9871516a41a2419d8af8e6fe9e0a5fa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.723546  8198 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0e2cd74dac4358aba60edf5382befd (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:12:23.724809  8198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0e2cd74dac4358aba60edf5382befd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.727531  8197 tablet_service.cc:1511] Processing CreateTablet for tablet 97bd47fbf9aa4d1bb8ec05287f69e0b0 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:12:23.728834  8197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97bd47fbf9aa4d1bb8ec05287f69e0b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.731472  8196 tablet_service.cc:1511] Processing CreateTablet for tablet 481500a64cdb4a7fb94658807295dcbe (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:12:23.732945  8196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 481500a64cdb4a7fb94658807295dcbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.735545  8194 tablet_service.cc:1511] Processing CreateTablet for tablet 053e9e9e90fe4d92853de2c0599484aa (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:12:23.736833  8194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 053e9e9e90fe4d92853de2c0599484aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.739054  8202 tablet_service.cc:1511] Processing CreateTablet for tablet 3ca98b6f039a45fb80c6e78ff2a0ae32 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:12:23.739504  8193 tablet_service.cc:1511] Processing CreateTablet for tablet 9fbc44c6e33e41698020173171d4ff21 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:12:23.740281  8202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca98b6f039a45fb80c6e78ff2a0ae32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.734176  8195 tablet_service.cc:1511] Processing CreateTablet for tablet c97c3defa48b473aaf762b3a27c0cba0 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:12:23.742237  8195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97c3defa48b473aaf762b3a27c0cba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.745345  8193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fbc44c6e33e41698020173171d4ff21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.760026  8053 tablet_service.cc:1511] Processing CreateTablet for tablet 5124ddc96f984ec2ae0ee97be6732a90 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:12:23.761557  8053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5124ddc96f984ec2ae0ee97be6732a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.781816  8200 tablet_service.cc:1511] Processing CreateTablet for tablet ac625d2d87774bc7be4ff2189dee7eb0 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:12:23.781839  8052 tablet_service.cc:1511] Processing CreateTablet for tablet 1f229a5eb4514abc9183538dd97073d2 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:12:23.784261  8200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac625d2d87774bc7be4ff2189dee7eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.784261  8052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f229a5eb4514abc9183538dd97073d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.791819  8051 tablet_service.cc:1511] Processing CreateTablet for tablet cc8454910b36474c90b678c57cea9773 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:12:23.793273  8051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc8454910b36474c90b678c57cea9773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.800001  8196 tablet_service.cc:1511] Processing CreateTablet for tablet d4f33fbb7b794128a4668f6fda160937 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:12:23.800781  8050 tablet_service.cc:1511] Processing CreateTablet for tablet 9871516a41a2419d8af8e6fe9e0a5fa1 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:12:23.801367  8196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f33fbb7b794128a4668f6fda160937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.801977  8050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9871516a41a2419d8af8e6fe9e0a5fa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.809552  8199 tablet_service.cc:1511] Processing CreateTablet for tablet a4992d168cb240009f8bfc8df07668e0 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:12:23.818571  8193 tablet_service.cc:1511] Processing CreateTablet for tablet b8b8bb0a8e3545218708596da648262f (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:12:23.819906  8193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8b8bb0a8e3545218708596da648262f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.821676  8199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4992d168cb240009f8bfc8df07668e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.848278  8048 tablet_service.cc:1511] Processing CreateTablet for tablet 97bd47fbf9aa4d1bb8ec05287f69e0b0 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:12:23.848513  8202 tablet_service.cc:1511] Processing CreateTablet for tablet f6b3f42115d64c26b2b412d669ed6010 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:12:23.849644  8202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6b3f42115d64c26b2b412d669ed6010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.847973  8049 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0e2cd74dac4358aba60edf5382befd (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:12:23.850184  8048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97bd47fbf9aa4d1bb8ec05287f69e0b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.850890  8049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0e2cd74dac4358aba60edf5382befd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.852509  8047 tablet_service.cc:1511] Processing CreateTablet for tablet 481500a64cdb4a7fb94658807295dcbe (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:12:23.855106  8047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 481500a64cdb4a7fb94658807295dcbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.860417  8046 tablet_service.cc:1511] Processing CreateTablet for tablet c97c3defa48b473aaf762b3a27c0cba0 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:12:23.861891  8046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97c3defa48b473aaf762b3a27c0cba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.862253  8045 tablet_service.cc:1511] Processing CreateTablet for tablet 053e9e9e90fe4d92853de2c0599484aa (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:12:23.862694  8044 tablet_service.cc:1511] Processing CreateTablet for tablet 9fbc44c6e33e41698020173171d4ff21 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:12:23.863843  8045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 053e9e9e90fe4d92853de2c0599484aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.864487  8044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fbc44c6e33e41698020173171d4ff21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.870818  8198 tablet_service.cc:1511] Processing CreateTablet for tablet 7c4e1687a1db419c8863789bb37c260a (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:12:23.872134  8198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c4e1687a1db419c8863789bb37c260a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.886060  8200 tablet_service.cc:1511] Processing CreateTablet for tablet 5b4358ace86545339036f21610735b47 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:12:23.892192  8201 tablet_service.cc:1511] Processing CreateTablet for tablet dad90abaa8b04508a5b086ff94502e4e (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:12:23.901280  8201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dad90abaa8b04508a5b086ff94502e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.899012  8252 tablet_bootstrap.cc:492] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:23.906183  8200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b4358ace86545339036f21610735b47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.915907  8202 tablet_service.cc:1511] Processing CreateTablet for tablet f265c49b04e34e9fa83eed8ecdb90607 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:12:23.917843  8202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f265c49b04e34e9fa83eed8ecdb90607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.908955  8196 tablet_service.cc:1511] Processing CreateTablet for tablet 51726a5d5cad44fd8e9b6803da998cb2 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:12:23.920452  8196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51726a5d5cad44fd8e9b6803da998cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.965277  8197 tablet_service.cc:1511] Processing CreateTablet for tablet 7a9aaa6e2a1c42a0a0db69d6406a3762 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:12:23.968763  8252 tablet_bootstrap.cc:654] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:23.968626  8053 tablet_service.cc:1511] Processing CreateTablet for tablet ac625d2d87774bc7be4ff2189dee7eb0 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:12:23.969980  8053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac625d2d87774bc7be4ff2189dee7eb0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:12:23.971240  8009 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:23.961555  8051 tablet_service.cc:1511] Processing CreateTablet for tablet 3ca98b6f039a45fb80c6e78ff2a0ae32 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:12:23.973045  8051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca98b6f039a45fb80c6e78ff2a0ae32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.989223  8049 tablet_service.cc:1511] Processing CreateTablet for tablet 7a9aaa6e2a1c42a0a0db69d6406a3762 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:12:24.006543  8197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a9aaa6e2a1c42a0a0db69d6406a3762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.011456  8194 tablet_service.cc:1511] Processing CreateTablet for tablet 6cbbc2cb4def4ea09659a9cdc5087f65 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:12:24.012801  8194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cbbc2cb4def4ea09659a9cdc5087f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:23.950495  8045 tablet_service.cc:1511] Processing CreateTablet for tablet f6b3f42115d64c26b2b412d669ed6010 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:12:23.994263  8048 tablet_service.cc:1511] Processing CreateTablet for tablet a4992d168cb240009f8bfc8df07668e0 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:12:24.023833  8195 tablet_service.cc:1511] Processing CreateTablet for tablet 542d60a75b684950a46a64c1d64b80a5 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:12:24.028015  8253 tablet_bootstrap.cc:492] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:24.027989  8046 tablet_service.cc:1511] Processing CreateTablet for tablet b8b8bb0a8e3545218708596da648262f (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:12:24.028949  8195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 542d60a75b684950a46a64c1d64b80a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.041175  8049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a9aaa6e2a1c42a0a0db69d6406a3762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.044482  8048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4992d168cb240009f8bfc8df07668e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.045544  8045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6b3f42115d64c26b2b412d669ed6010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.029317  8044 tablet_service.cc:1511] Processing CreateTablet for tablet 7c4e1687a1db419c8863789bb37c260a (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:12:24.063715  8253 tablet_bootstrap.cc:654] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.062271  8050 tablet_service.cc:1511] Processing CreateTablet for tablet 5b4358ace86545339036f21610735b47 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:12:24.067155  8047 tablet_service.cc:1511] Processing CreateTablet for tablet d4f33fbb7b794128a4668f6fda160937 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:12:24.068328  8127 tablet_service.cc:1511] Processing CreateTablet for tablet 1f229a5eb4514abc9183538dd97073d2 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:12:24.048030  8046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8b8bb0a8e3545218708596da648262f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.071385  8126 tablet_service.cc:1511] Processing CreateTablet for tablet cc8454910b36474c90b678c57cea9773 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:12:24.071470  8050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b4358ace86545339036f21610735b47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.058808  8193 tablet_service.cc:1511] Processing CreateTablet for tablet 3d785324a51949a79a4de6f0d836be56 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:12:24.061777  8252 tablet_bootstrap.cc:492] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:24.073915  8125 tablet_service.cc:1511] Processing CreateTablet for tablet 9871516a41a2419d8af8e6fe9e0a5fa1 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:12:24.074745  8252 ts_tablet_manager.cc:1403] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.176s	user 0.008s	sys 0.015s
I20260709 13:12:24.075093  8193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d785324a51949a79a4de6f0d836be56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.076530  8124 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0e2cd74dac4358aba60edf5382befd (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:12:24.079838  8128 tablet_service.cc:1511] Processing CreateTablet for tablet 5124ddc96f984ec2ae0ee97be6732a90 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:12:24.069940  8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f229a5eb4514abc9183538dd97073d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.082626  8044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c4e1687a1db419c8863789bb37c260a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.083868  8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9871516a41a2419d8af8e6fe9e0a5fa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.083967  8047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f33fbb7b794128a4668f6fda160937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.079535  8252 raft_consensus.cc:359] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:24.086835  8252 raft_consensus.cc:385] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.087123  8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc8454910b36474c90b678c57cea9773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.087177  8252 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.088033  8198 tablet_service.cc:1511] Processing CreateTablet for tablet 8dd7949df539418cbc00001111885883 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:12:24.088475  8252 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:24.089365  8198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dd7949df539418cbc00001111885883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.088460  8052 tablet_service.cc:1511] Processing CreateTablet for tablet dad90abaa8b04508a5b086ff94502e4e (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:12:24.089916  8123 tablet_service.cc:1511] Processing CreateTablet for tablet 97bd47fbf9aa4d1bb8ec05287f69e0b0 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:12:24.091156  8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0e2cd74dac4358aba60edf5382befd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.096001  8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97bd47fbf9aa4d1bb8ec05287f69e0b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.097216  8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5124ddc96f984ec2ae0ee97be6732a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.092597  8199 tablet_service.cc:1511] Processing CreateTablet for tablet a3e2b86010604477955f19a56daf6d04 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:12:24.092576  8052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dad90abaa8b04508a5b086ff94502e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.112800  8237 heartbeater.cc:499] Master 127.6.198.126:38543 was elected leader, sending a full tablet report...
I20260709 13:12:24.115926  8195 tablet_service.cc:1511] Processing CreateTablet for tablet 62c9c8ea5bb04f99add9743c31017f37 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:12:24.121363  8122 tablet_service.cc:1511] Processing CreateTablet for tablet 481500a64cdb4a7fb94658807295dcbe (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:12:24.122752  8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 481500a64cdb4a7fb94658807295dcbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.117414  8202 tablet_service.cc:1511] Processing CreateTablet for tablet 17c830696ad4418b88d85b6a97e41851 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:12:24.131484  8200 tablet_service.cc:1511] Processing CreateTablet for tablet b578d88c1c1f4ca58b7291fed4909f09 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:12:24.132807  8252 ts_tablet_manager.cc:1434] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.058s	user 0.013s	sys 0.001s
I20260709 13:12:24.133826  8252 tablet_bootstrap.cc:492] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:24.136091  8121 tablet_service.cc:1511] Processing CreateTablet for tablet c97c3defa48b473aaf762b3a27c0cba0 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:12:24.137517  8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97c3defa48b473aaf762b3a27c0cba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.138895  8195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62c9c8ea5bb04f99add9743c31017f37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.142864  8202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17c830696ad4418b88d85b6a97e41851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.144412  8120 tablet_service.cc:1511] Processing CreateTablet for tablet 053e9e9e90fe4d92853de2c0599484aa (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:12:24.145813  8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 053e9e9e90fe4d92853de2c0599484aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.148850  8200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b578d88c1c1f4ca58b7291fed4909f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.053828  8196 tablet_service.cc:1511] Processing CreateTablet for tablet 1dfd4edf9eab4bcc8b8aad94587349c3 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:12:24.151943  8119 tablet_service.cc:1511] Processing CreateTablet for tablet 9fbc44c6e33e41698020173171d4ff21 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:12:24.156591  8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fbc44c6e33e41698020173171d4ff21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.177052  8199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3e2b86010604477955f19a56daf6d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.183154  8088 heartbeater.cc:499] Master 127.6.198.126:38543 was elected leader, sending a full tablet report...
I20260709 13:12:24.185676  8196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dfd4edf9eab4bcc8b8aad94587349c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.197028  8046 tablet_service.cc:1511] Processing CreateTablet for tablet 51726a5d5cad44fd8e9b6803da998cb2 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:12:24.198333  8046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51726a5d5cad44fd8e9b6803da998cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.200606  8252 tablet_bootstrap.cc:654] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.204661  8197 tablet_service.cc:1511] Processing CreateTablet for tablet a478a4313e6149378cae4b6f833e3c2a (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:12:24.247994  8197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a478a4313e6149378cae4b6f833e3c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.249761  8048 tablet_service.cc:1511] Processing CreateTablet for tablet f265c49b04e34e9fa83eed8ecdb90607 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:12:24.251073  8048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f265c49b04e34e9fa83eed8ecdb90607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.253571  8201 tablet_service.cc:1511] Processing CreateTablet for tablet 1c85566831fd4f4283bc20a68e5ff448 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:12:24.259114  8201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c85566831fd4f4283bc20a68e5ff448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.271500  8253 tablet_bootstrap.cc:492] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:24.276015  8253 ts_tablet_manager.cc:1403] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.279s	user 0.051s	sys 0.040s
I20260709 13:12:24.296178  8194 tablet_service.cc:1511] Processing CreateTablet for tablet bd0294d309cb4f8696e693285a7b5b04 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:12:24.297493  8194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd0294d309cb4f8696e693285a7b5b04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.299448  8045 tablet_service.cc:1511] Processing CreateTablet for tablet 542d60a75b684950a46a64c1d64b80a5 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:12:24.300742  8045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 542d60a75b684950a46a64c1d64b80a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.305276  8052 tablet_service.cc:1511] Processing CreateTablet for tablet 6cbbc2cb4def4ea09659a9cdc5087f65 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:12:24.312523  8163 heartbeater.cc:499] Master 127.6.198.126:38543 was elected leader, sending a full tablet report...
I20260709 13:12:24.313879  8252 tablet_bootstrap.cc:492] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:24.314433  8252 ts_tablet_manager.cc:1403] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.181s	user 0.012s	sys 0.001s
I20260709 13:12:24.318048  8253 raft_consensus.cc:359] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:24.331607  8253 raft_consensus.cc:385] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.321939  8256 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.334139  8253 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.334641  8256 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:24.320103  8193 tablet_service.cc:1511] Processing CreateTablet for tablet ed7f54a64a14435691eeef2ece07e5dc (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:12:24.337574  8193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed7f54a64a14435691eeef2ece07e5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.320003  8252 raft_consensus.cc:359] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } }
I20260709 13:12:24.333379  8052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cbbc2cb4def4ea09659a9cdc5087f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.345613  8119 tablet_service.cc:1511] Processing CreateTablet for tablet a4992d168cb240009f8bfc8df07668e0 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:12:24.350936  8252 raft_consensus.cc:385] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.355362  8044 tablet_service.cc:1511] Processing CreateTablet for tablet a3e2b86010604477955f19a56daf6d04 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:12:24.355688  8252 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.356514  8252 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } }
I20260709 13:12:24.362398  8196 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee839503bc34e5fb2b1f4e09dfe9784 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:12:24.364132  8196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee839503bc34e5fb2b1f4e09dfe9784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.367712  8053 tablet_service.cc:1511] Processing CreateTablet for tablet 1dfd4edf9eab4bcc8b8aad94587349c3 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:12:24.372939  8256 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:24.390440  8195 tablet_service.cc:1511] Processing CreateTablet for tablet e529be00baff4bfab3a3d0a45d84c238 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:12:24.391987  8195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e529be00baff4bfab3a3d0a45d84c238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.391752  8253 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:24.388283  8044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3e2b86010604477955f19a56daf6d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.396835  8127 tablet_service.cc:1511] Processing CreateTablet for tablet 17c830696ad4418b88d85b6a97e41851 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:12:24.359086  8051 tablet_service.cc:1511] Processing CreateTablet for tablet 3d785324a51949a79a4de6f0d836be56 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:12:24.397264  8200 tablet_service.cc:1511] Processing CreateTablet for tablet 710721029c8047ff8b9fc6d5920859f4 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:12:24.399312  8047 tablet_service.cc:1511] Processing CreateTablet for tablet 8dd7949df539418cbc00001111885883 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:12:24.399763  8126 tablet_service.cc:1511] Processing CreateTablet for tablet 3ca98b6f039a45fb80c6e78ff2a0ae32 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:12:24.401976  8200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 710721029c8047ff8b9fc6d5920859f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.402688  8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4992d168cb240009f8bfc8df07668e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.401634  8125 tablet_service.cc:1511] Processing CreateTablet for tablet ac625d2d87774bc7be4ff2189dee7eb0 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:12:24.411242  8252 ts_tablet_manager.cc:1434] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.097s	user 0.006s	sys 0.012s
I20260709 13:12:24.425148  8258 tablet_bootstrap.cc:492] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:24.426615  8201 tablet_service.cc:1511] Processing CreateTablet for tablet b186be823643475982e269b73d886a42 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:12:24.427876  8201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b186be823643475982e269b73d886a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.427018  8198 tablet_service.cc:1511] Processing CreateTablet for tablet 07fe988cafa34bcaa51af457b7248ccc (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:12:24.435570  8198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07fe988cafa34bcaa51af457b7248ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.443884  8201 tablet_service.cc:1511] Processing CreateTablet for tablet 4489591b74b1469d81269da167e98f71 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:12:24.445218  8201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4489591b74b1469d81269da167e98f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.447870  8122 tablet_service.cc:1511] Processing CreateTablet for tablet 51726a5d5cad44fd8e9b6803da998cb2 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:12:24.453210  8049 tablet_service.cc:1511] Processing CreateTablet for tablet 62c9c8ea5bb04f99add9743c31017f37 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:12:24.453959  8050 tablet_service.cc:1511] Processing CreateTablet for tablet 17c830696ad4418b88d85b6a97e41851 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:12:24.454979  8253 ts_tablet_manager.cc:1434] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.179s	user 0.012s	sys 0.006s
I20260709 13:12:24.463918  8128 tablet_service.cc:1511] Processing CreateTablet for tablet d4f33fbb7b794128a4668f6fda160937 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:12:24.467454  8197 tablet_service.cc:1511] Processing CreateTablet for tablet 0e4fa00ecc35497e823a1329fe771042 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:12:24.469209  8197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e4fa00ecc35497e823a1329fe771042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.492551  8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac625d2d87774bc7be4ff2189dee7eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.423740  8051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d785324a51949a79a4de6f0d836be56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.495105  8053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dfd4edf9eab4bcc8b8aad94587349c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.495949  8123 tablet_service.cc:1511] Processing CreateTablet for tablet b8b8bb0a8e3545218708596da648262f (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:12:24.422601  8252 tablet_bootstrap.cc:492] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:24.497231  8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8b8bb0a8e3545218708596da648262f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.497738  8253 tablet_bootstrap.cc:492] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:24.393435  8199 tablet_service.cc:1511] Processing CreateTablet for tablet bc4cbdf31b174089be94b5f87a22f629 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:12:24.500082  8120 tablet_service.cc:1511] Processing CreateTablet for tablet dad90abaa8b04508a5b086ff94502e4e (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:12:24.501936  8199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc4cbdf31b174089be94b5f87a22f629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.502063  8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dad90abaa8b04508a5b086ff94502e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.497241  8050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17c830696ad4418b88d85b6a97e41851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.520398  8121 tablet_service.cc:1511] Processing CreateTablet for tablet f6b3f42115d64c26b2b412d669ed6010 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:12:24.521335  8046 tablet_service.cc:1511] Processing CreateTablet for tablet b578d88c1c1f4ca58b7291fed4909f09 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:12:24.521986  8124 tablet_service.cc:1511] Processing CreateTablet for tablet 7c4e1687a1db419c8863789bb37c260a (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:12:24.523483  8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17c830696ad4418b88d85b6a97e41851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.517923  8262 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.524679  8262 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:24.522444  8047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dd7949df539418cbc00001111885883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.527137  8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f33fbb7b794128a4668f6fda160937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.528820  8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6b3f42115d64c26b2b412d669ed6010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.530987  8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c4e1687a1db419c8863789bb37c260a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.532804  8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca98b6f039a45fb80c6e78ff2a0ae32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.539108  8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51726a5d5cad44fd8e9b6803da998cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.555429  8049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62c9c8ea5bb04f99add9743c31017f37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.556322  8046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b578d88c1c1f4ca58b7291fed4909f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.602639  8202 tablet_service.cc:1511] Processing CreateTablet for tablet d80e5571c873417aa73c79508290e836 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:12:24.639417  8262 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:24.667376  8194 tablet_service.cc:1511] Processing CreateTablet for tablet f4539f806feb4e37900025e737458841 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:12:24.668813  8194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4539f806feb4e37900025e737458841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.670817  8202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d80e5571c873417aa73c79508290e836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.673099  8258 tablet_bootstrap.cc:654] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.701361  8201 tablet_service.cc:1511] Processing CreateTablet for tablet b8fabe9526434f818c106b17c57b3b85 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:12:24.684489  8253 tablet_bootstrap.cc:654] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:24.709805  7948 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 167963us
I20260709 13:12:24.716490  8194 tablet_service.cc:1511] Processing CreateTablet for tablet ec8da6c869684688957d5323fb023bc5 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:12:24.758927  8045 tablet_service.cc:1511] Processing CreateTablet for tablet 1c85566831fd4f4283bc20a68e5ff448 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:12:24.750365  8201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8fabe9526434f818c106b17c57b3b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.760232  8045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c85566831fd4f4283bc20a68e5ff448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.764194  8261 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:24.764925  8261 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } }
I20260709 13:12:24.771634  8258 tablet_bootstrap.cc:492] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:24.772117  8258 ts_tablet_manager.cc:1403] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.350s	user 0.015s	sys 0.028s
I20260709 13:12:24.774456  8258 raft_consensus.cc:359] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:24.775211  8258 raft_consensus.cc:385] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.775493  8258 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.775533  8048 tablet_service.cc:1511] Processing CreateTablet for tablet bd0294d309cb4f8696e693285a7b5b04 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
W20260709 13:12:24.774451  7945 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 261158us
I20260709 13:12:24.774185  8053 tablet_service.cc:1511] Processing CreateTablet for tablet a478a4313e6149378cae4b6f833e3c2a (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:12:24.776294  8258 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:24.799784  8252 tablet_bootstrap.cc:654] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:24.815769  8193 tablet_service.cc:1511] Processing CreateTablet for tablet f7fb8453eda243f9856f489af50e67a2 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:12:24.820847  8119 tablet_service.cc:1511] Processing CreateTablet for tablet 5b4358ace86545339036f21610735b47 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:12:24.822144  8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b4358ace86545339036f21610735b47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.823683  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:24.845294  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:24.850934  8175 leader_election.cc:343] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.851576  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:24.851794  8261 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:12:24.852929  8175 leader_election.cc:343] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.853349  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:12:24.854571  8175 leader_election.cc:343] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.855800  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:24.857115  8261 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:12:24.857983  8175 leader_election.cc:343] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:24.858435  8175 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:24.859520  8261 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:24.840860  8194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec8da6c869684688957d5323fb023bc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.865340  8200 tablet_service.cc:1511] Processing CreateTablet for tablet 1002e40536a34729929022702d03444c (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:12:24.868011  8200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1002e40536a34729929022702d03444c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.870497  8193 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7fb8453eda243f9856f489af50e67a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.846481  8048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd0294d309cb4f8696e693285a7b5b04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.888216  8128 tablet_service.cc:1511] Processing CreateTablet for tablet 7a9aaa6e2a1c42a0a0db69d6406a3762 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:12:24.895521  8258 ts_tablet_manager.cc:1434] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.123s	user 0.020s	sys 0.040s
I20260709 13:12:24.896485  8253 tablet_bootstrap.cc:492] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:24.897168  8253 ts_tablet_manager.cc:1403] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.403s	user 0.034s	sys 0.024s
I20260709 13:12:24.903460  8252 tablet_bootstrap.cc:492] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:24.903970  8252 ts_tablet_manager.cc:1403] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.492s	user 0.019s	sys 0.006s
I20260709 13:12:24.910843  8201 tablet_service.cc:1511] Processing CreateTablet for tablet 18de311592f1492b8b460305c56e6749 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:12:24.912271  8201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18de311592f1492b8b460305c56e6749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.912776  8252 raft_consensus.cc:359] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:24.913487  8252 raft_consensus.cc:385] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.913789  8252 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.916225  8252 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:24.918327  8252 ts_tablet_manager.cc:1434] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.004s
I20260709 13:12:24.919313  8252 tablet_bootstrap.cc:492] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:24.932925  8201 tablet_service.cc:1511] Processing CreateTablet for tablet d013f25ff51143beb65d31cc33e5a248 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:12:24.934428  8201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d013f25ff51143beb65d31cc33e5a248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.937264  8252 tablet_bootstrap.cc:654] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:24.938048  7945 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50829.4us
I20260709 13:12:24.939730  8253 raft_consensus.cc:359] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:24.940757  8253 raft_consensus.cc:385] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:24.941425  8253 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.941923  8194 tablet_service.cc:1511] Processing CreateTablet for tablet fe73c60f6d9f4d25a3dffad1e7198694 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:12:24.944468  8194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe73c60f6d9f4d25a3dffad1e7198694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.944276  8047 tablet_service.cc:1511] Processing CreateTablet for tablet ed7f54a64a14435691eeef2ece07e5dc (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:12:24.963354  8201 tablet_service.cc:1511] Processing CreateTablet for tablet bf628848d7464a21aa8f76f7eb56a408 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:12:24.964804  8201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf628848d7464a21aa8f76f7eb56a408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.961947  8049 tablet_service.cc:1511] Processing CreateTablet for tablet e529be00baff4bfab3a3d0a45d84c238 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:12:24.967497  8053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a478a4313e6149378cae4b6f833e3c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:24.987927  8196 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59846 (request call id 37) took 1012 ms. Trace:
I20260709 13:12:24.988456  8196 rpcz_store.cc:276] 0709 13:12:23.975775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.358780 (+383005us) service_pool.cc:224] Handling call
0709 13:12:24.363497 (+  4717us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.363613 (+   116us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:24.756561 (+392948us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:24.979134 (+222573us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:24.987861 (+  8727us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:24.990186  8045 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 31) took 1036 ms. Trace:
I20260709 13:12:24.991608  8196 tablet_service.cc:1511] Processing CreateTablet for tablet a003b4584d3747a381522d67299aaed2 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:12:24.991233  8045 rpcz_store.cc:276] 0709 13:12:23.953476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.733099 (+779623us) service_pool.cc:224] Handling call
0709 13:12:24.759786 (+ 26687us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.759881 (+    95us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:24.962680 (+202799us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:24.989043 (+ 26363us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:24.990050 (+  1007us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:24.957684  8195 tablet_service.cc:1511] Processing CreateTablet for tablet 1b3101b6cc874539a962e23536fdabc9 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:12:24.994648  8052 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 23) took 1066 ms. Trace:
I20260709 13:12:24.995031  8052 rpcz_store.cc:276] 0709 13:12:23.928505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.303229 (+374724us) service_pool.cc:224] Handling call
0709 13:12:24.332804 (+ 29575us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.332987 (+   183us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:24.713497 (+380510us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:24.994164 (+280667us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:24.994582 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:24.995492  8044 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 26) took 1043 ms. Trace:
I20260709 13:12:24.995883  8044 rpcz_store.cc:276] 0709 13:12:23.951677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.353645 (+401968us) service_pool.cc:224] Handling call
0709 13:12:24.356226 (+  2581us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.356317 (+    91us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:24.915829 (+559512us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:24.995002 (+ 79173us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:24.995428 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:24.997057  8052 tablet_service.cc:1511] Processing CreateTablet for tablet bc4cbdf31b174089be94b5f87a22f629 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:12:24.997725  8044 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee839503bc34e5fb2b1f4e09dfe9784 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
W20260709 13:12:24.999253  8009 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:24.992969  8196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a003b4584d3747a381522d67299aaed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.001052  8045 tablet_service.cc:1511] Processing CreateTablet for tablet 710721029c8047ff8b9fc6d5920859f4 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:12:25.020181  8125 tablet_service.cc:1511] Processing CreateTablet for tablet 6cbbc2cb4def4ea09659a9cdc5087f65 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:12:25.003551  8202 tablet_service.cc:1511] Processing CreateTablet for tablet 8e397a319b11487fbbacec77723b5739 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:12:25.027871  8202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e397a319b11487fbbacec77723b5739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.028326  8121 tablet_service.cc:1511] Processing CreateTablet for tablet 3d785324a51949a79a4de6f0d836be56 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:12:25.029466  8126 tablet_service.cc:1511] Processing CreateTablet for tablet 1dfd4edf9eab4bcc8b8aad94587349c3 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:12:25.007723  8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a9aaa6e2a1c42a0a0db69d6406a3762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.007916  8258 tablet_bootstrap.cc:492] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:25.009542  8050 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 30) took 1056 ms. Trace:
I20260709 13:12:25.032205  8050 rpcz_store.cc:276] 0709 13:12:23.953194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.452679 (+499485us) service_pool.cc:224] Handling call
0709 13:12:24.493476 (+ 40797us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.496760 (+  3284us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:24.798472 (+301712us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:24.966736 (+168264us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.009481 (+ 42745us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.009955  8124 tablet_service.cc:1511] Processing CreateTablet for tablet f265c49b04e34e9fa83eed8ecdb90607 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:12:25.034121  8050 tablet_service.cc:1511] Processing CreateTablet for tablet 0e4fa00ecc35497e823a1329fe771042 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:12:25.022635  8047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed7f54a64a14435691eeef2ece07e5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.034137  8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f265c49b04e34e9fa83eed8ecdb90607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.037961  8258 tablet_bootstrap.cc:654] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.024025  8198 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59846 (request call id 40) took 1019 ms. Trace:
I20260709 13:12:25.038638  8045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 710721029c8047ff8b9fc6d5920859f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.038772  8198 rpcz_store.cc:276] 0709 13:12:24.004935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.420976 (+416041us) service_pool.cc:224] Handling call
0709 13:12:24.435102 (+ 14126us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.435196 (+    94us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:24.827378 (+392182us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.023511 (+196133us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.023960 (+   449us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.043882  8050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e4fa00ecc35497e823a1329fe771042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.045650  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:25.011116  8273 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.011437  8199 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59846 (request call id 36) took 1049 ms. Trace:
I20260709 13:12:25.012291  8201 tablet_service.cc:1511] Processing CreateTablet for tablet b9bd4708595b40438af51aba853a0f94 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:12:25.046602  8199 rpcz_store.cc:276] 0709 13:12:23.961902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.332649 (+370747us) service_pool.cc:224] Handling call
0709 13:12:24.501452 (+168803us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.501548 (+    96us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:24.969986 (+468438us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:24.984925 (+ 14939us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.011370 (+ 26445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.047735  8201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9bd4708595b40438af51aba853a0f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.013624  8127 tablet_service.cc:1511] Processing CreateTablet for tablet 542d60a75b684950a46a64c1d64b80a5 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
W20260709 13:12:25.047591  8029 leader_election.cc:343] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.048561  8199 tablet_service.cc:1511] Processing CreateTablet for tablet 6771ba07746943f6a958ca7be1990296 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:12:25.050062  8199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6771ba07746943f6a958ca7be1990296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.006616  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:12:25.051630  8026 leader_election.cc:343] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.052094  8026 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:25.053553  8264 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:25.041836  8198 tablet_service.cc:1511] Processing CreateTablet for tablet ade1b48685164d64a62772d186ac959f (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:12:25.055775  8198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade1b48685164d64a62772d186ac959f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.055614  8273 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.024152  8046 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 28) took 1071 ms. Trace:
I20260709 13:12:25.065742  8197 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:59846 (request call id 39) took 1069 ms. Trace:
I20260709 13:12:25.066571  8044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee839503bc34e5fb2b1f4e09dfe9784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.057478  8051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 24) took 1081 ms. Trace:
I20260709 13:12:25.067368  8051 rpcz_store.cc:276] 0709 13:12:23.950972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.345414 (+394442us) service_pool.cc:224] Handling call
0709 13:12:24.398022 (+ 52608us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.398122 (+   100us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:24.817537 (+419415us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.009932 (+192395us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.010319 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.066428  8197 rpcz_store.cc:276] 0709 13:12:23.996291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.419957 (+423666us) service_pool.cc:224] Handling call
0709 13:12:24.468577 (+ 48620us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.468694 (+   117us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:24.913781 (+445087us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.065232 (+151451us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.065681 (+   449us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.047036  8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dfd4edf9eab4bcc8b8aad94587349c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.062665  8046 rpcz_store.cc:276] 0709 13:12:23.952600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.386715 (+434115us) service_pool.cc:224] Handling call
0709 13:12:24.522289 (+135574us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.522412 (+   123us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:24.802066 (+279654us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.023696 (+221630us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.024093 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.089990  8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cbbc2cb4def4ea09659a9cdc5087f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.091235  8119 tablet_service.cc:1511] Processing CreateTablet for tablet a3e2b86010604477955f19a56daf6d04 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:12:25.105548  8195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b3101b6cc874539a962e23536fdabc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.113966  8261 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.114535  8261 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.116711  8261 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:25.118026  8252 tablet_bootstrap.cc:492] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:25.118551  8252 ts_tablet_manager.cc:1403] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.199s	user 0.011s	sys 0.017s
I20260709 13:12:25.096910  8273 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:25.085465  8051 tablet_service.cc:1511] Processing CreateTablet for tablet 07fe988cafa34bcaa51af457b7248ccc (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:12:25.121483  8252 raft_consensus.cc:359] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.123353  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:25.124825  8175 leader_election.cc:343] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.107388  8052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc4cbdf31b174089be94b5f87a22f629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.128651  8123 tablet_service.cc:1511] Processing CreateTablet for tablet b578d88c1c1f4ca58b7291fed4909f09 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:12:25.122184  8252 raft_consensus.cc:385] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.130553  8252 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.133888  8252 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.139200  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:25.140023  8122 tablet_service.cc:1511] Processing CreateTablet for tablet 8dd7949df539418cbc00001111885883 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:12:25.140154  8252 ts_tablet_manager.cc:1434] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:12:25.141258  8051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07fe988cafa34bcaa51af457b7248ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.022528  8253 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.141669  8252 tablet_bootstrap.cc:492] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:25.143096  8264 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.143738  8264 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.145442  8253 ts_tablet_manager.cc:1434] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.248s	user 0.006s	sys 0.005s
I20260709 13:12:25.125836  8046 tablet_service.cc:1511] Processing CreateTablet for tablet b186be823643475982e269b73d886a42 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:12:25.148465  8046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b186be823643475982e269b73d886a42. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:12:25.150199  8175 leader_election.cc:343] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.150796  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:25.151695  8261 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:25.125814  8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3e2b86010604477955f19a56daf6d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.183303  8047 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 34) took 1229 ms. Trace:
I20260709 13:12:25.183712  8047 rpcz_store.cc:276] 0709 13:12:23.954319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.942778 (+988459us) service_pool.cc:224] Handling call
0709 13:12:24.945083 (+  2305us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.945173 (+    90us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.059706 (+114533us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.182797 (+123091us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.183231 (+   434us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.186453  8050 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 39) took 1190 ms. Trace:
I20260709 13:12:25.186813  8050 rpcz_store.cc:276] 0709 13:12:23.995792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.033237 (+1037445us) service_pool.cc:224] Handling call
0709 13:12:25.034899 (+  1662us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.034977 (+    78us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.065438 (+ 30461us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.153772 (+ 88334us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.186393 (+ 32621us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.188145  8253 tablet_bootstrap.cc:492] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:25.189265  8050 tablet_service.cc:1511] Processing CreateTablet for tablet d80e5571c873417aa73c79508290e836 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:12:25.186213  8047 tablet_service.cc:1511] Processing CreateTablet for tablet 4489591b74b1469d81269da167e98f71 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:12:25.179057  8049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e529be00baff4bfab3a3d0a45d84c238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.193199  8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 542d60a75b684950a46a64c1d64b80a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.193958  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:12:25.197605  8029 leader_election.cc:343] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.199010  8120 tablet_service.cc:1511] Processing CreateTablet for tablet 62c9c8ea5bb04f99add9743c31017f37 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:12:25.200258  8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62c9c8ea5bb04f99add9743c31017f37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.202943  8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b578d88c1c1f4ca58b7291fed4909f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.205828  8045 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 38) took 1210 ms. Trace:
I20260709 13:12:25.211810  8258 tablet_bootstrap.cc:492] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:25.212311  8258 ts_tablet_manager.cc:1403] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.205s	user 0.021s	sys 0.006s
I20260709 13:12:25.214196  8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d785324a51949a79a4de6f0d836be56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.218425  8264 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:25.206221  8045 rpcz_store.cc:276] 0709 13:12:23.995515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.995422 (+999907us) service_pool.cc:224] Handling call
0709 13:12:25.026636 (+ 31214us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.026733 (+    97us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.131996 (+105263us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.205338 (+ 73342us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.205770 (+   432us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.227427  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:25.227732  8050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d80e5571c873417aa73c79508290e836. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:12:25.229141  8026 leader_election.cc:343] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.229635  8026 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:25.231096  8264 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:25.231318  8045 tablet_service.cc:1511] Processing CreateTablet for tablet f4539f806feb4e37900025e737458841 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:12:25.233596  8045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4539f806feb4e37900025e737458841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.232504  8128 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 21) took 1068 ms. Trace:
I20260709 13:12:25.234143  8128 rpcz_store.cc:276] 0709 13:12:24.163847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.886912 (+723065us) service_pool.cc:224] Handling call
0709 13:12:24.957842 (+ 70930us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.006393 (+ 48551us) spinlock_profiling.cc:243] Waited 9.03 ms on lock 0x7b58001c3e98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef230e 00007f6dc0e33bf9 00007f6dbff64e51 00007f6dc0027cc9 00007f6dbffd6400 00007f6db945baf4 00007f6db945ba82 00007f6db945b9b1 00007f6db945b92c
0709 13:12:25.007057 (+   664us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.045826 (+ 38769us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.232053 (+186227us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.232449 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20776832}
I20260709 13:12:25.236575  8128 tablet_service.cc:1511] Processing CreateTablet for tablet 1c85566831fd4f4283bc20a68e5ff448 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:12:25.238685  8047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4489591b74b1469d81269da167e98f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.249135  8252 tablet_bootstrap.cc:654] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.253012  8046 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 41) took 1246 ms. Trace:
I20260709 13:12:25.253417  8046 rpcz_store.cc:276] 0709 13:12:24.006221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.095103 (+1088882us) service_pool.cc:224] Handling call
0709 13:12:25.147885 (+ 52782us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.148008 (+   123us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.208962 (+ 60954us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.252522 (+ 43560us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.252950 (+   428us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.257619  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:25.259124  8101 leader_election.cc:343] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.261520  8045 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 44) took 1094 ms. Trace:
I20260709 13:12:25.262655  8261 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.261582  8258 raft_consensus.cc:359] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.263373  8261 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.264420  8253 tablet_bootstrap.cc:654] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.265618  8261 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:25.266484  8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dd7949df539418cbc00001111885883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.267656  8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c85566831fd4f4283bc20a68e5ff448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.267854  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:25.269075  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:25.271130  8175 leader_election.cc:343] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.271989  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:12:25.273496  8175 leader_election.cc:343] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.273943  8175 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:12:25.275234  8104 leader_election.cc:343] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.278975  8261 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:25.279194  8104 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:25.283143  8273 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:25.286983  8051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 40) took 1287 ms. Trace:
I20260709 13:12:25.287472  8051 rpcz_store.cc:276] 0709 13:12:23.999648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.084378 (+1084730us) service_pool.cc:224] Handling call
0709 13:12:25.121489 (+ 37111us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.121588 (+    99us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.255884 (+134296us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.286238 (+ 30354us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.286921 (+   683us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.290506  8127 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 22) took 1126 ms. Trace:
I20260709 13:12:25.290269  8051 tablet_service.cc:1511] Processing CreateTablet for tablet ec8da6c869684688957d5323fb023bc5 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:12:25.290890  8127 rpcz_store.cc:276] 0709 13:12:24.164133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.012697 (+848564us) service_pool.cc:224] Handling call
0709 13:12:25.048287 (+ 35590us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.048377 (+    90us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.246577 (+198200us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.277479 (+ 30902us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.290434 (+ 12955us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.291810  8051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec8da6c869684688957d5323fb023bc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.292903  8127 tablet_service.cc:1511] Processing CreateTablet for tablet a478a4313e6149378cae4b6f833e3c2a (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:12:25.294132  8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a478a4313e6149378cae4b6f833e3c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.280769  8124 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 20) took 1117 ms. Trace:
I20260709 13:12:25.297423  8048 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 33) took 1343 ms. Trace:
I20260709 13:12:25.297883  8046 tablet_service.cc:1511] Processing CreateTablet for tablet b8fabe9526434f818c106b17c57b3b85 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:12:25.297031  8120 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 27) took 1131 ms. Trace:
I20260709 13:12:25.278816  8258 raft_consensus.cc:385] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.290089  8052 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 36) took 1295 ms. Trace:
I20260709 13:12:25.306710  8119 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 26) took 1141 ms. Trace:
I20260709 13:12:25.308705  8258 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.319464  8046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8fabe9526434f818c106b17c57b3b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.319880  8258 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.322059  8258 ts_tablet_manager.cc:1434] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.109s	user 0.006s	sys 0.002s
I20260709 13:12:25.307300  8048 rpcz_store.cc:276] 0709 13:12:23.954024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.774642 (+820618us) service_pool.cc:224] Handling call
0709 13:12:24.776336 (+  1694us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.776487 (+   151us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.136809 (+360322us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.296968 (+160159us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.297367 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.323764  8049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 35) took 1343 ms. Trace:
I20260709 13:12:25.324029  8048 tablet_service.cc:1511] Processing CreateTablet for tablet 1002e40536a34729929022702d03444c (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:12:25.325315  8048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1002e40536a34729929022702d03444c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.262458  8045 rpcz_store.cc:276] 0709 13:12:24.167433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.230439 (+1063006us) service_pool.cc:224] Handling call
0709 13:12:25.233152 (+  2713us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.233255 (+   103us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.245914 (+ 12659us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.260647 (+ 14733us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.261432 (+   785us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.327950  8045 tablet_service.cc:1511] Processing CreateTablet for tablet f7fb8453eda243f9856f489af50e67a2 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:12:25.324172  8049 rpcz_store.cc:276] 0709 13:12:23.982668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.960916 (+978248us) service_pool.cc:224] Handling call
0709 13:12:24.967164 (+  6248us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.968886 (+  1722us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.281782 (+312896us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.323264 (+ 41482us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.323709 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1792}
I20260709 13:12:25.329159  8045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7fb8453eda243f9856f489af50e67a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.332468  8051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 46) took 1136 ms. Trace:
I20260709 13:12:25.335233  8051 rpcz_store.cc:276] 0709 13:12:24.196384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.289273 (+1092889us) service_pool.cc:224] Handling call
0709 13:12:25.291229 (+  1956us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.291319 (+    90us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.300651 (+  9332us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.332006 (+ 31355us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.332411 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.336272  8125 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 23) took 1171 ms. Trace:
I20260709 13:12:25.332463  8049 tablet_service.cc:1511] Processing CreateTablet for tablet 1b3101b6cc874539a962e23536fdabc9 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:12:25.336648  8125 rpcz_store.cc:276] 0709 13:12:24.164466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.017857 (+853391us) service_pool.cc:224] Handling call
0709 13:12:25.027196 (+  9339us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.027283 (+    87us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.216778 (+189495us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.335779 (+119001us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.336219 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.337412  8051 tablet_service.cc:1511] Processing CreateTablet for tablet 18de311592f1492b8b460305c56e6749 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:12:25.334848  8121 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 24) took 1170 ms. Trace:
I20260709 13:12:25.338331  8121 rpcz_store.cc:276] 0709 13:12:24.164743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.027453 (+862710us) service_pool.cc:224] Handling call
0709 13:12:25.029103 (+  1650us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.029190 (+    87us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.264748 (+235558us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.334403 (+ 69655us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.334791 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.340557  8252 tablet_bootstrap.cc:492] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:25.341037  8049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b3101b6cc874539a962e23536fdabc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.341360  8252 ts_tablet_manager.cc:1403] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.200s	user 0.020s	sys 0.013s
I20260709 13:12:25.345171  8044 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 37) took 1349 ms. Trace:
I20260709 13:12:25.312304  8052 rpcz_store.cc:276] 0709 13:12:23.994938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.996070 (+1001132us) service_pool.cc:224] Handling call
0709 13:12:24.997873 (+  1803us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.997963 (+    90us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.202011 (+204048us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.289608 (+ 87597us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.290028 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.340355  8121 tablet_service.cc:1511] Processing CreateTablet for tablet bd0294d309cb4f8696e693285a7b5b04 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:12:25.347656  8125 tablet_service.cc:1511] Processing CreateTablet for tablet ed7f54a64a14435691eeef2ece07e5dc (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:12:25.346248  8252 raft_consensus.cc:359] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.384030  8252 raft_consensus.cc:385] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.384454  8252 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.384555  8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed7f54a64a14435691eeef2ece07e5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.383697  8044 rpcz_store.cc:276] 0709 13:12:23.995248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.996834 (+1001586us) service_pool.cc:224] Handling call
0709 13:12:25.024854 (+ 28020us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.024955 (+   101us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.292704 (+267749us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.344575 (+ 51871us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.345089 (+   514us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.384980  8052 tablet_service.cc:1511] Processing CreateTablet for tablet fe73c60f6d9f4d25a3dffad1e7198694 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:12:25.385300  8252 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.386135  8044 tablet_service.cc:1511] Processing CreateTablet for tablet d013f25ff51143beb65d31cc33e5a248 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:12:25.387665  8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd0294d309cb4f8696e693285a7b5b04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.387801  8252 ts_tablet_manager.cc:1434] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.046s	user 0.002s	sys 0.005s
I20260709 13:12:25.350785  8123 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 28) took 1184 ms. Trace:
I20260709 13:12:25.388644  8123 rpcz_store.cc:276] 0709 13:12:24.165921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.127626 (+961705us) service_pool.cc:224] Handling call
0709 13:12:25.129461 (+  1835us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.129562 (+   101us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.218274 (+ 88712us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.350236 (+131962us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.350726 (+   490us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.390790  8123 tablet_service.cc:1511] Processing CreateTablet for tablet 07fe988cafa34bcaa51af457b7248ccc (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:12:25.314945  8119 rpcz_store.cc:276] 0709 13:12:24.165355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.090238 (+924883us) service_pool.cc:224] Handling call
0709 13:12:25.125344 (+ 35106us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.125440 (+    96us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.206072 (+ 80632us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.301920 (+ 95848us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.302638 (+   718us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.306960  8124 rpcz_store.cc:276] 0709 13:12:24.163552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.008992 (+845440us) service_pool.cc:224] Handling call
0709 13:12:25.033640 (+ 24648us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.033780 (+   140us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.183814 (+150034us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.280223 (+ 96409us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.280706 (+   483us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.361191  8258 tablet_bootstrap.cc:492] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:25.307998  8120 rpcz_store.cc:276] 0709 13:12:24.165636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.097808 (+932172us) service_pool.cc:224] Handling call
0709 13:12:25.199818 (+102010us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.199915 (+    97us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.259623 (+ 59708us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.296545 (+ 36922us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.296974 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.398327  8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07fe988cafa34bcaa51af457b7248ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.364398  8051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18de311592f1492b8b460305c56e6749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.401492  8052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe73c60f6d9f4d25a3dffad1e7198694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.377076  8046 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 45) took 1188 ms. Trace:
I20260709 13:12:25.401523  8258 tablet_bootstrap.cc:654] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.410522  8046 rpcz_store.cc:276] 0709 13:12:24.188652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.275329 (+1086677us) service_pool.cc:224] Handling call
0709 13:12:25.312384 (+ 37055us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.314837 (+  2453us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.347404 (+ 32567us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.376516 (+ 29112us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.377011 (+   495us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":136}
I20260709 13:12:25.412832  8046 tablet_service.cc:1511] Processing CreateTablet for tablet bf628848d7464a21aa8f76f7eb56a408 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:12:25.415208  8047 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 42) took 1404 ms. Trace:
I20260709 13:12:25.415633  8047 rpcz_store.cc:276] 0709 13:12:24.010704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.184703 (+1173999us) service_pool.cc:224] Handling call
0709 13:12:25.191895 (+  7192us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.191995 (+   100us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.338770 (+146775us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.414519 (+ 75749us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.415142 (+   623us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.417011  8122 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 29) took 1250 ms. Trace:
I20260709 13:12:25.417408  8122 rpcz_store.cc:276] 0709 13:12:24.166208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.138831 (+972623us) service_pool.cc:224] Handling call
0709 13:12:25.140985 (+  2154us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.141180 (+   195us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.349222 (+208042us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.416261 (+ 67039us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.416953 (+   692us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.417543  8047 tablet_service.cc:1511] Processing CreateTablet for tablet a003b4584d3747a381522d67299aaed2 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:12:25.420522  8122 tablet_service.cc:1511] Processing CreateTablet for tablet 710721029c8047ff8b9fc6d5920859f4 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:12:25.377076  8053 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 32) took 1423 ms. Trace:
I20260709 13:12:25.394141  8119 tablet_service.cc:1511] Processing CreateTablet for tablet e529be00baff4bfab3a3d0a45d84c238 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:12:25.394673  8126 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 25) took 1229 ms. Trace:
I20260709 13:12:25.395524  8124 tablet_service.cc:1511] Processing CreateTablet for tablet bc4cbdf31b174089be94b5f87a22f629 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:12:25.363965  8253 tablet_bootstrap.cc:492] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:25.423285  8253 ts_tablet_manager.cc:1403] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.245s	user 0.014s	sys 0.003s
I20260709 13:12:25.422806  8126 rpcz_store.cc:276] 0709 13:12:24.165034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.028600 (+863566us) service_pool.cc:224] Handling call
0709 13:12:25.046570 (+ 17970us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.046662 (+    92us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.223511 (+176849us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.394132 (+170621us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.394618 (+   486us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.424990  8126 tablet_service.cc:1511] Processing CreateTablet for tablet 0e4fa00ecc35497e823a1329fe771042 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:12:25.399118  8252 tablet_bootstrap.cc:492] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:25.399935  8120 tablet_service.cc:1511] Processing CreateTablet for tablet 0ee839503bc34e5fb2b1f4e09dfe9784 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:12:25.433893  8253 raft_consensus.cc:359] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.438057  8253 raft_consensus.cc:385] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.438477  8253 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.422290  8053 rpcz_store.cc:276] 0709 13:12:23.953751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:24.773094 (+819343us) service_pool.cc:224] Handling call
0709 13:12:24.787351 (+ 14257us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:24.787451 (+   100us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.251818 (+464367us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.376516 (+124698us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.377012 (+   496us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.440480  8253 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.444068  8253 ts_tablet_manager.cc:1434] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:12:25.437548  8052 tablet_service.cc:1511] Processing CreateTablet for tablet 8e397a319b11487fbbacec77723b5739 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:12:25.445935  8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 710721029c8047ff8b9fc6d5920859f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.414217  8046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf628848d7464a21aa8f76f7eb56a408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.455246  8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e529be00baff4bfab3a3d0a45d84c238. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.455971  8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ee839503bc34e5fb2b1f4e09dfe9784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.458881  8253 tablet_bootstrap.cc:492] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:25.447196  8045 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 48) took 1130 ms. Trace:
I20260709 13:12:25.470638  8045 rpcz_store.cc:276] 0709 13:12:24.316632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.326796 (+1010164us) service_pool.cc:224] Handling call
0709 13:12:25.328726 (+  1930us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.328817 (+    91us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.422924 (+ 94107us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.442552 (+ 19628us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.447089 (+  4537us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14720}
I20260709 13:12:25.473378  8045 tablet_service.cc:1511] Processing CreateTablet for tablet ade1b48685164d64a62772d186ac959f (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:12:25.474431  8128 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 31) took 1275 ms. Trace:
I20260709 13:12:25.437601  8252 tablet_bootstrap.cc:654] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.459394  8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc4cbdf31b174089be94b5f87a22f629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.474973  8128 rpcz_store.cc:276] 0709 13:12:24.198830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.235668 (+1036838us) service_pool.cc:224] Handling call
0709 13:12:25.238481 (+  2813us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.238571 (+    90us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.285043 (+ 46472us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.473740 (+188697us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.474333 (+   593us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":773}
I20260709 13:12:25.459452  8047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a003b4584d3747a381522d67299aaed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.480765  8052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e397a319b11487fbbacec77723b5739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.481348  8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e4fa00ecc35497e823a1329fe771042. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.484246  8045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade1b48685164d64a62772d186ac959f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.485783  8044 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d013f25ff51143beb65d31cc33e5a248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.441815  8053 tablet_service.cc:1511] Processing CreateTablet for tablet b9bd4708595b40438af51aba853a0f94 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:12:25.469391  8125 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 34) took 1269 ms. Trace:
I20260709 13:12:25.469942  8258 tablet_bootstrap.cc:492] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:25.487983  8258 ts_tablet_manager.cc:1403] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.127s	user 0.011s	sys 0.011s
I20260709 13:12:25.488260  8053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9bd4708595b40438af51aba853a0f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.490300  8125 rpcz_store.cc:276] 0709 13:12:24.199940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.346568 (+1146628us) service_pool.cc:224] Handling call
0709 13:12:25.384105 (+ 37537us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.384195 (+    90us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.439385 (+ 55190us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.454650 (+ 15265us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.469328 (+ 14678us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.492468  8125 tablet_service.cc:1511] Processing CreateTablet for tablet b186be823643475982e269b73d886a42 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:12:25.493680  8127 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 32) took 1294 ms. Trace:
I20260709 13:12:25.494160  8127 rpcz_store.cc:276] 0709 13:12:24.199253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.292001 (+1092748us) service_pool.cc:224] Handling call
0709 13:12:25.293704 (+  1703us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.293794 (+    90us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.389671 (+ 95877us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.493014 (+103343us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.493609 (+   595us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.496556  8127 tablet_service.cc:1511] Processing CreateTablet for tablet f4539f806feb4e37900025e737458841 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:12:25.515214  8128 tablet_service.cc:1511] Processing CreateTablet for tablet d80e5571c873417aa73c79508290e836 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:12:25.527732  8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4539f806feb4e37900025e737458841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.530321  8051 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 50) took 1127 ms. Trace:
I20260709 13:12:25.530432  8258 raft_consensus.cc:359] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } }
I20260709 13:12:25.535501  8050 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 43) took 1517 ms. Trace:
I20260709 13:12:25.539153  8258 raft_consensus.cc:385] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.539559  8258 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.539260  8050 rpcz_store.cc:276] 0709 13:12:24.021045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.188280 (+1167235us) service_pool.cc:224] Handling call
0709 13:12:25.190298 (+  2018us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.191624 (+  1326us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.287657 (+ 96033us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.535002 (+247345us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.535441 (+   439us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.540556  8258 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } }
I20260709 13:12:25.542982  8258 ts_tablet_manager.cc:1434] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.011s
I20260709 13:12:25.437438  8049 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 49) took 1076 ms. Trace:
I20260709 13:12:25.547732  8049 rpcz_store.cc:276] 0709 13:12:24.361004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.330147 (+969143us) service_pool.cc:224] Handling call
0709 13:12:25.337308 (+  7161us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.337520 (+   212us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.380712 (+ 43192us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.436882 (+ 56170us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.437376 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.538576  8051 rpcz_store.cc:276] 0709 13:12:24.403442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.336254 (+932812us) service_pool.cc:224] Handling call
0709 13:12:25.338523 (+  2269us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.338621 (+    98us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.448816 (+110195us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.529827 (+ 81011us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.530261 (+   434us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.549293  8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b186be823643475982e269b73d886a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.551082  8050 tablet_service.cc:1511] Processing CreateTablet for tablet 6771ba07746943f6a958ca7be1990296 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:12:25.555204  8048 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:49202 (request call id 47) took 1324 ms. Trace:
I20260709 13:12:25.561563  8122 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 38) took 1360 ms. Trace:
I20260709 13:12:25.552129  8253 tablet_bootstrap.cc:654] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.575557  8128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d80e5571c873417aa73c79508290e836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.581439  8126 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 39) took 1379 ms. Trace:
I20260709 13:12:25.581799  8126 rpcz_store.cc:276] 0709 13:12:24.201733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.423875 (+1222142us) service_pool.cc:224] Handling call
0709 13:12:25.442402 (+ 18527us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.442492 (+    90us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.555254 (+112762us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.581000 (+ 25746us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.581388 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.575938  8127 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 44) took 1367 ms. Trace:
I20260709 13:12:25.584857  8127 rpcz_store.cc:276] 0709 13:12:24.208917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.495352 (+1286435us) service_pool.cc:224] Handling call
0709 13:12:25.527239 (+ 31887us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.527381 (+   142us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.559159 (+ 31778us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.575234 (+ 16075us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.575865 (+   631us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.576627  8258 tablet_bootstrap.cc:492] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:25.586998  8124 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 36) took 1386 ms. Trace:
I20260709 13:12:25.587365  8124 rpcz_store.cc:276] 0709 13:12:24.200681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.394899 (+1194218us) service_pool.cc:224] Handling call
0709 13:12:25.440095 (+ 45196us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.440196 (+   101us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.567973 (+127777us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.586557 (+ 18584us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.586944 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208000}
I20260709 13:12:25.578231  8119 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 35) took 1377 ms. Trace:
I20260709 13:12:25.578549  8050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6771ba07746943f6a958ca7be1990296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.577601  8122 rpcz_store.cc:276] 0709 13:12:24.201377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.419665 (+1218288us) service_pool.cc:224] Handling call
0709 13:12:25.421957 (+  2292us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.439144 (+ 17187us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.526847 (+ 87703us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.537324 (+ 10477us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.561485 (+ 24161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.588876  8119 rpcz_store.cc:276] 0709 13:12:24.200287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.393252 (+1192965us) service_pool.cc:224] Handling call
0709 13:12:25.440377 (+ 47125us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.440492 (+   115us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.464812 (+ 24320us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.557165 (+ 92353us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.577658 (+ 20493us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":459008}
I20260709 13:12:25.590152  8122 tablet_service.cc:1511] Processing CreateTablet for tablet ec8da6c869684688957d5323fb023bc5 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:12:25.590849  8119 tablet_service.cc:1511] Processing CreateTablet for tablet b8fabe9526434f818c106b17c57b3b85 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:12:25.591341  8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec8da6c869684688957d5323fb023bc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.591048  8126 tablet_service.cc:1511] Processing CreateTablet for tablet 4489591b74b1469d81269da167e98f71 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:12:25.591475  8127 tablet_service.cc:1511] Processing CreateTablet for tablet 1002e40536a34729929022702d03444c (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:12:25.592020  8119 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8fabe9526434f818c106b17c57b3b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.593093  8124 tablet_service.cc:1511] Processing CreateTablet for tablet f7fb8453eda243f9856f489af50e67a2 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:12:25.579309  8125 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 41) took 1376 ms. Trace:
I20260709 13:12:25.579872  8121 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 33) took 1380 ms. Trace:
I20260709 13:12:25.595594  8258 tablet_bootstrap.cc:654] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.597195  8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1002e40536a34729929022702d03444c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.602592  8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4489591b74b1469d81269da167e98f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.605110  8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7fb8453eda243f9856f489af50e67a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.609495  8252 tablet_bootstrap.cc:492] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:25.610035  8252 ts_tablet_manager.cc:1403] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.211s	user 0.007s	sys 0.025s
I20260709 13:12:25.594794  8125 rpcz_store.cc:276] 0709 13:12:24.207851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.491465 (+1283614us) service_pool.cc:224] Handling call
0709 13:12:25.548808 (+ 57343us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.548928 (+   120us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.557730 (+  8802us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.569207 (+ 11477us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.579251 (+ 10044us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7936}
I20260709 13:12:25.611258  8127 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 47) took 1356 ms. Trace:
I20260709 13:12:25.611613  8127 rpcz_store.cc:276] 0709 13:12:24.254864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.590557 (+1335693us) service_pool.cc:224] Handling call
0709 13:12:25.592255 (+  1698us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.592336 (+    81us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.604848 (+ 12512us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.610827 (+  5979us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.611202 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.612629  8125 tablet_service.cc:1511] Processing CreateTablet for tablet 1b3101b6cc874539a962e23536fdabc9 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:12:25.594825  8121 rpcz_store.cc:276] 0709 13:12:24.199594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.339460 (+1139866us) service_pool.cc:224] Handling call
0709 13:12:25.387167 (+ 47707us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b58001c3e98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef21a2 00007f6dc0e33bb9 00007f6dbff64d88 00007f6dc0027b80 00007f6dbffd6400 00007f6db945baf4 00007f6db945ba82 00007f6db945b9b1 00007f6db945b92c
0709 13:12:25.387241 (+    74us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.387339 (+    98us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.421274 (+ 33935us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.574814 (+153540us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.579819 (+  5005us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5472000}
I20260709 13:12:25.613137  8252 raft_consensus.cc:359] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.614197  8252 raft_consensus.cc:385] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.614053  8127 tablet_service.cc:1511] Processing CreateTablet for tablet 18de311592f1492b8b460305c56e6749 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:12:25.615149  8121 tablet_service.cc:1511] Processing CreateTablet for tablet fe73c60f6d9f4d25a3dffad1e7198694 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:12:25.616652  8123 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 40) took 1414 ms. Trace:
I20260709 13:12:25.618233  8123 rpcz_store.cc:276] 0709 13:12:24.202112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.389883 (+1187771us) service_pool.cc:224] Handling call
0709 13:12:25.397877 (+  7994us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.397970 (+    93us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.524430 (+126460us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.616197 (+ 91767us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.616599 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.619776  8120 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 37) took 1418 ms. Trace:
I20260709 13:12:25.620110  8120 rpcz_store.cc:276] 0709 13:12:24.201035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.398894 (+1197859us) service_pool.cc:224] Handling call
0709 13:12:25.439954 (+ 41060us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.440069 (+   115us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.465317 (+ 25248us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.619330 (+154013us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.619724 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28928}
I20260709 13:12:25.620615  8123 tablet_service.cc:1511] Processing CreateTablet for tablet d013f25ff51143beb65d31cc33e5a248 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:12:25.622926  8253 tablet_bootstrap.cc:492] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:25.623414  8253 ts_tablet_manager.cc:1403] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.165s	user 0.009s	sys 0.002s
I20260709 13:12:25.625536  8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d013f25ff51143beb65d31cc33e5a248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.626467  8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe73c60f6d9f4d25a3dffad1e7198694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.617291  8252 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.630714  8125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b3101b6cc874539a962e23536fdabc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.635859  8253 raft_consensus.cc:359] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } }
I20260709 13:12:25.636703  8253 raft_consensus.cc:385] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.577329  8048 rpcz_store.cc:276] 0709 13:12:24.230972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.323086 (+1092114us) service_pool.cc:224] Handling call
0709 13:12:25.324878 (+  1792us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.324967 (+    89us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.445400 (+120433us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.554733 (+109333us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.555149 (+   416us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.639073  8252 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.641690  8252 ts_tablet_manager.cc:1434] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.001s
I20260709 13:12:25.642839  8252 tablet_bootstrap.cc:492] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:25.646462  8120 tablet_service.cc:1511] Processing CreateTablet for tablet bf628848d7464a21aa8f76f7eb56a408 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:12:25.648104  8122 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 46) took 1438 ms. Trace:
I20260709 13:12:25.637059  8253 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.648499  8122 rpcz_store.cc:276] 0709 13:12:24.209588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.589261 (+1379673us) service_pool.cc:224] Handling call
0709 13:12:25.590916 (+  1655us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.591004 (+    88us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.601896 (+ 10892us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.623652 (+ 21756us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.639478 (+ 15826us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b58001c3e98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef21a2 00007f6dc0e33bb9 00007f6dbff64d88 00007f6dc002b973 00007f6dc002770f 00007f6dc0027ecd 00007f6dbffd6400 00007f6db945baf4 00007f6db945ba82
0709 13:12:25.648049 (+  8571us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30669696}
I20260709 13:12:25.650882  8122 tablet_service.cc:1511] Processing CreateTablet for tablet 8e397a319b11487fbbacec77723b5739 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:12:25.648187  8127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18de311592f1492b8b460305c56e6749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.655216  8253 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } }
I20260709 13:12:25.658367  8261 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.658622  8120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf628848d7464a21aa8f76f7eb56a408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.658903  8261 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.661729  8122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e397a319b11487fbbacec77723b5739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.661056  8261 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:25.662608  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:25.664124  8175 leader_election.cc:343] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.667142  8253 ts_tablet_manager.cc:1434] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.002s
I20260709 13:12:25.668182  8253 tablet_bootstrap.cc:492] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:25.671066  8258 tablet_bootstrap.cc:492] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:25.671521  8258 ts_tablet_manager.cc:1403] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.095s	user 0.012s	sys 0.012s
I20260709 13:12:25.680682  8126 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 42) took 1472 ms. Trace:
I20260709 13:12:25.674291  8258 raft_consensus.cc:359] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.674554  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:25.688247  8126 rpcz_store.cc:276] 0709 13:12:24.208305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.588955 (+1380650us) service_pool.cc:224] Handling call
0709 13:12:25.591866 (+  2911us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.591962 (+    96us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.646211 (+ 54249us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.655934 (+  9723us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.680613 (+ 24679us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:12:25.689929  8175 leader_election.cc:343] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.678107  8252 tablet_bootstrap.cc:654] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.690424  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:25.680087  8128 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 43) took 1471 ms. Trace:
I20260709 13:12:25.690654  8126 tablet_service.cc:1511] Processing CreateTablet for tablet ade1b48685164d64a62772d186ac959f (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:12:25.692602  8126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade1b48685164d64a62772d186ac959f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.692756  8261 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:25.680043  8124 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 48) took 1324 ms. Trace:
I20260709 13:12:25.694078  8124 rpcz_store.cc:276] 0709 13:12:24.355430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.592263 (+1236833us) service_pool.cc:224] Handling call
0709 13:12:25.593889 (+  1626us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.593978 (+    89us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.612698 (+ 18720us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.657784 (+ 45086us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.679983 (+ 22199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.695386  8125 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 49) took 1304 ms. Trace:
I20260709 13:12:25.695760  8125 rpcz_store.cc:276] 0709 13:12:24.391418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.611729 (+1220311us) service_pool.cc:224] Handling call
0709 13:12:25.625432 (+ 13703us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.625608 (+   176us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.669517 (+ 43909us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.694924 (+ 25407us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.695318 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.697731  8124 tablet_service.cc:1511] Processing CreateTablet for tablet 6771ba07746943f6a958ca7be1990296 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:12:25.684983  8121 tablet_service.cc:1511] Processing CreateTablet for tablet a003b4584d3747a381522d67299aaed2 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:12:25.688258  8123 tablet_service.cc:1511] Processing CreateTablet for tablet b9bd4708595b40438af51aba853a0f94 (DEFAULT_TABLE table=test-workload [id=0c145eb23c244f65946ed4d70486b91d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:12:25.698438  8128 rpcz_store.cc:276] 0709 13:12:24.208619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.513896 (+1305277us) service_pool.cc:224] Handling call
0709 13:12:25.575102 (+ 61206us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.575217 (+   115us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.606038 (+ 30821us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.649799 (+ 43761us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.680038 (+ 30239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1792}
I20260709 13:12:25.699890  8124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6771ba07746943f6a958ca7be1990296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.687579  8119 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 45) took 1478 ms. Trace:
I20260709 13:12:25.703028  8123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9bd4708595b40438af51aba853a0f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.702759  8119 rpcz_store.cc:276] 0709 13:12:24.209193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.589987 (+1380794us) service_pool.cc:224] Handling call
0709 13:12:25.591609 (+  1622us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.591695 (+    86us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.644386 (+ 52691us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.687119 (+ 42733us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.687524 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.704607  8258 raft_consensus.cc:385] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.704963  8258 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.705734  8258 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.707795  8252 tablet_bootstrap.cc:492] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:25.708143  8258 ts_tablet_manager.cc:1434] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.002s
I20260709 13:12:25.708236  8252 ts_tablet_manager.cc:1403] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.066s	user 0.009s	sys 0.001s
I20260709 13:12:25.710588  8253 tablet_bootstrap.cc:654] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.711249  8252 raft_consensus.cc:359] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.718545  8252 raft_consensus.cc:385] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.718883  8252 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.719640  8252 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.721606  8127 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36038 (request call id 50) took 1312 ms. Trace:
I20260709 13:12:25.712083  8121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a003b4584d3747a381522d67299aaed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:25.712882  8258 tablet_bootstrap.cc:492] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:25.722075  8252 ts_tablet_manager.cc:1434] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:12:25.727936  8252 tablet_bootstrap.cc:492] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:25.728739  8258 tablet_bootstrap.cc:654] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.722025  8127 rpcz_store.cc:276] 0709 13:12:24.408768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:25.613156 (+1204388us) service_pool.cc:224] Handling call
0709 13:12:25.638795 (+ 25639us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:12:25.647771 (+  8976us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:12:25.690977 (+ 43206us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:25.721139 (+ 30162us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:25.721549 (+   410us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:12:25.734862  8252 tablet_bootstrap.cc:654] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.741663  8258 tablet_bootstrap.cc:492] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:25.743238  8258 ts_tablet_manager.cc:1403] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.002s
I20260709 13:12:25.746264  8258 raft_consensus.cc:359] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.747025  8258 raft_consensus.cc:385] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.747321  8258 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.749912  8253 tablet_bootstrap.cc:492] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:25.750442  8253 ts_tablet_manager.cc:1403] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.082s	user 0.006s	sys 0.006s
I20260709 13:12:25.753326  8253 raft_consensus.cc:359] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.754067  8253 raft_consensus.cc:385] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.754374  8253 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.755067  8253 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.757473  8253 ts_tablet_manager.cc:1434] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:12:25.758024  8258 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.760933  8252 tablet_bootstrap.cc:492] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:25.761360  8252 ts_tablet_manager.cc:1403] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.003s
I20260709 13:12:25.761816  8258 ts_tablet_manager.cc:1434] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:12:25.762750  8258 tablet_bootstrap.cc:492] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:25.763729  8252 raft_consensus.cc:359] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.764360  8252 raft_consensus.cc:385] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.764622  8252 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.765450  8252 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.767390  8252 ts_tablet_manager.cc:1434] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:12:25.769284  8258 tablet_bootstrap.cc:654] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.774824  8252 tablet_bootstrap.cc:492] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:25.776789  8253 tablet_bootstrap.cc:492] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:25.779385  8273 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.779879  8273 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.781070  8252 tablet_bootstrap.cc:654] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.782097  8273 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:25.783907  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:25.786046  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:25.786715  8212 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:25.789956  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:25.790635  8258 tablet_bootstrap.cc:492] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:25.791214  8258 ts_tablet_manager.cc:1403] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.000s
W20260709 13:12:25.791780  8101 leader_election.cc:343] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.794143  8258 raft_consensus.cc:359] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.794883  8258 raft_consensus.cc:385] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.795163  8258 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.795888  8258 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.798110  8273 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:25.798770  8273 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.799337  8273 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.820813  8273 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.826172  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:12:25.828217  8101 leader_election.cc:343] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.822738  8253 tablet_bootstrap.cc:654] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.834532  8258 ts_tablet_manager.cc:1434] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.043s	user 0.029s	sys 0.015s
I20260709 13:12:25.834949  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:25.835515  8212 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.835980  8258 tablet_bootstrap.cc:492] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:25.842170  8212 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:25.843462  8258 tablet_bootstrap.cc:654] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.843369  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:25.844388  8290 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:25.850589  8273 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:25.851198  8252 tablet_bootstrap.cc:492] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:25.851872  8252 ts_tablet_manager.cc:1403] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.077s	user 0.022s	sys 0.032s
I20260709 13:12:25.853317  8273 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.853905  8273 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.855350  8252 raft_consensus.cc:359] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.856215  8252 raft_consensus.cc:385] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.856572  8252 raft_consensus.cc:740] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.857164  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:25.857770  8062 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:25.857467  8252 consensus_queue.cc:260] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.858928  8253 tablet_bootstrap.cc:492] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:25.859377  8253 ts_tablet_manager.cc:1403] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.083s	user 0.010s	sys 0.004s
I20260709 13:12:25.860234  8252 ts_tablet_manager.cc:1434] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:12:25.861879  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:25.864248  8252 tablet_bootstrap.cc:492] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:25.864392  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:25.864962  8212 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:25.870059  8252 tablet_bootstrap.cc:654] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.873623  8273 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:25.874629  8273 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:25.875042  8273 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.875449  8273 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.862191  8253 raft_consensus.cc:359] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.878737  8253 raft_consensus.cc:385] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.879061  8253 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.879837  8253 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.882059  8253 ts_tablet_manager.cc:1434] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.000s
I20260709 13:12:25.884332  8253 tablet_bootstrap.cc:492] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:25.885483  8258 tablet_bootstrap.cc:492] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:25.886005  8258 ts_tablet_manager.cc:1403] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.050s	user 0.021s	sys 0.023s
I20260709 13:12:25.886595  8292 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.887018  8290 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:25.887051  8292 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.888067  8290 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.889285  8258 raft_consensus.cc:359] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.889999  8258 raft_consensus.cc:385] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.890363  8258 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.891129  8258 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.889999  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:25.891937  8252 tablet_bootstrap.cc:492] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:25.892630  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:25.896514  8212 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:25.896521  8252 ts_tablet_manager.cc:1403] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.002s
W20260709 13:12:25.893235  8101 leader_election.cc:343] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.898095  8104 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:25.895262  8273 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.899523  8252 raft_consensus.cc:359] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.900203  8252 raft_consensus.cc:385] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.900452  8252 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.900607  8273 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:25.901417  8273 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:25.901820  8273 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.895887  8292 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:25.896032  8253 tablet_bootstrap.cc:654] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.902652  8292 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.903064  8292 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.905037  8292 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:25.901983  8252 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.907405  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:25.902134  8273 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.907934  8252 ts_tablet_manager.cc:1434] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:12:25.908239  8063 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:25.908856  8252 tablet_bootstrap.cc:492] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:25.912381  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:25.913132  8062 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.916752  8101 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:25.917797  8258 ts_tablet_manager.cc:1434] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.031s	user 0.014s	sys 0.015s
I20260709 13:12:25.918422  8261 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:25.917689  8295 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:25.918963  8258 tablet_bootstrap.cc:492] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:25.918949  8295 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:25.918846  8261 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.921239  8261 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:25.923815  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:25.925561  8258 tablet_bootstrap.cc:654] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:25.925319  8175 leader_election.cc:343] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.925007  8295 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.930751  8252 tablet_bootstrap.cc:654] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.932933  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:25.933523  8212 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.941922  8212 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:25.943133  8252 tablet_bootstrap.cc:492] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:25.943598  8252 ts_tablet_manager.cc:1403] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.000s
I20260709 13:12:25.943696  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:25.943970  8104 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:12:25.945642  8104 leader_election.cc:343] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.945688  8273 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.949169  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:25.949928  8062 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
W20260709 13:12:25.950863  8101 leader_election.cc:343] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:25.952301  8297 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:25.953614  8252 raft_consensus.cc:359] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.954330  8297 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:25.955214  8252 raft_consensus.cc:385] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.955842  8252 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.955499  8297 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.961313  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:25.961644  8258 tablet_bootstrap.cc:492] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:25.961897  8211 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.962255  8258 ts_tablet_manager.cc:1403] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.027s
I20260709 13:12:25.964627  8295 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.965917  8258 raft_consensus.cc:359] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.966786  8258 raft_consensus.cc:385] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.967032  8258 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.959306  8273 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:25.967382  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:25.967916  8062 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:25.967641  8258 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.969269  8253 tablet_bootstrap.cc:492] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:25.969571  8258 ts_tablet_manager.cc:1434] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:12:25.969784  8253 ts_tablet_manager.cc:1403] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.086s	user 0.016s	sys 0.009s
I20260709 13:12:25.969379  7960 catalog_manager.cc:5697] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:25.970494  8258 tablet_bootstrap.cc:492] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:25.975364  8062 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:25.975456  8253 raft_consensus.cc:359] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.976213  8253 raft_consensus.cc:385] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:25.976511  8253 raft_consensus.cc:740] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:25.976514  8101 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:25.977519  8258 tablet_bootstrap.cc:654] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:25.977357  8253 consensus_queue.cc:260] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.978492  8298 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:25.987080  8253 ts_tablet_manager.cc:1434] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.004s
I20260709 13:12:25.978761  8211 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:25.996897  8298 raft_consensus.cc:697] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:25.997891  8104 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:25.998135  8298 consensus_queue.cc:237] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.980093  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:25.989003  8252 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:25.993445  8290 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.007153  8290 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } }
W20260709 13:12:26.009055  8104 leader_election.cc:343] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.009426  8290 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
I20260709 13:12:25.979192  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:25.995894  8273 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.010218  8138 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:26.010514  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:26.010632  8273 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.011221  8264 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.011664  8264 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.012180  8261 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.012602  8261 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.013235  8273 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:25.999639  8292 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.013687  8264 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.014118  8292 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:26.014513  8261 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:26.015192  8292 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.016115  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:12:26.017436  8029 leader_election.cc:343] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.018181  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.019331  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.011109  8212 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:25.978854  8295 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:12:26.019464  8175 leader_election.cc:343] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.019963  8063 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:26.020582  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.021971  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:26.021911  8175 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:26.021205  8062 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:26.023625  8101 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:26.024382  8273 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.024736  8273 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.001226  8253 tablet_bootstrap.cc:492] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:26.024881  8101 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:26.025154  8273 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.025599  8308 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.025985  8261 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.026618  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:26.028144  8261 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.028565  8261 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.029397  8252 ts_tablet_manager.cc:1434] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.085s	user 0.008s	sys 0.000s
W20260709 13:12:26.024802  8175 leader_election.cc:343] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.030736  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:26.031037  8252 tablet_bootstrap.cc:492] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:26.028156  8292 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.032760  8292 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.032752  8273 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } }
I20260709 13:12:26.033263  8292 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.035094  8273 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
I20260709 13:12:26.036548  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:26.036661  8261 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.037376  8062 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.042014  8261 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:26.042111  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:26.042634  8261 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.044560  8261 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.044592  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:26.045182  8137 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.045064  8261 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.047148  8261 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:12:26.050843  8175 leader_election.cc:343] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.047756  8062 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:26.039212  8308 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.052227  8101 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:26.053053  8273 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.053745  8273 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:26.054184  8308 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:26.053988  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:26.058982  8273 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } }
I20260709 13:12:26.065485  8137 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:26.070863  8292 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:26.076229  8175 leader_election.cc:343] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.077175  8175 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:26.072618  8252 tablet_bootstrap.cc:654] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.073323  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:26.075248  8253 tablet_bootstrap.cc:654] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:26.075479  8009 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:26.076421  8138 raft_consensus.cc:2393] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 1: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:12:26.072192  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:26.076658  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.078759  8211 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.079393  8307 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.079264  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.080287  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:26.083984  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:26.084962  8061 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:26.085136  8175 leader_election.cc:343] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:26.085958  8175 leader_election.cc:343] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:26.086483  8104 leader_election.cc:343] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.086333  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:26.090406  8026 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:26.091745  8258 tablet_bootstrap.cc:492] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:26.092180  8258 ts_tablet_manager.cc:1403] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.122s	user 0.056s	sys 0.028s
I20260709 13:12:26.093058  8292 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.094735  8258 raft_consensus.cc:359] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.094976  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:26.095392  8258 raft_consensus.cc:385] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.095649  8258 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
W20260709 13:12:26.096374  8104 leader_election.cc:343] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.096261  8258 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.098214  8258 ts_tablet_manager.cc:1434] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:12:26.099361  8258 tablet_bootstrap.cc:492] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:26.100528  8307 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:26.101706  8307 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.104501  8252 tablet_bootstrap.cc:492] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:26.104935  8252 ts_tablet_manager.cc:1403] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.074s	user 0.004s	sys 0.011s
I20260709 13:12:26.103003  8308 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:12:26.106297  8175 leader_election.cc:343] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.107345  8311 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.107813  8311 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.108783  8211 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:26.110155  8311 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:26.112386  8258 tablet_bootstrap.cc:654] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.113495  8292 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.114985  8292 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.116046  8061 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:26.106751  8175 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:26.116508  8264 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.116842  8252 raft_consensus.cc:359] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.116499  8273 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.117687  8252 raft_consensus.cc:385] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.117805  8292 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.118110  8252 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.118925  8252 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.117764  8273 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.119776  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.120347  8063 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:26.121469  8273 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.121622  8252 ts_tablet_manager.cc:1434] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:12:26.122923  8252 tablet_bootstrap.cc:492] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:26.122725  8101 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:26.123102  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.123608  8273 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.123714  8063 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:26.124163  8273 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.124665  8273 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.125191  8101 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:26.126092  8292 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.126534  8292 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.126897  8292 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.127202  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:26.127321  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:26.127902  8211 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:26.127967  8212 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:26.131484  8311 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.132602  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.133185  8063 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:26.133561  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:26.134658  8311 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.134949  8101 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:26.135274  8311 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.135811  8298 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.136305  8298 raft_consensus.cc:697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:26.136504  8175 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:26.137343  8308 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.137126  8298 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.137702  8308 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.138176  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:26.139568  8175 leader_election.cc:343] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.138041  8308 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.141947  8311 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:26.145911  8292 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.147189  8138 raft_consensus.cc:2393] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 1: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:12:26.149116  8253 tablet_bootstrap.cc:492] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:26.149768  8253 ts_tablet_manager.cc:1403] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.149s	user 0.020s	sys 0.011s
I20260709 13:12:26.157240  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:26.157531  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:26.157823  8063 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.158984  8308 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.159224  8273 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.160107  8211 raft_consensus.cc:2393] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 1: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:12:26.166460  8063 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:26.162173  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:26.167670  8101 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:26.168478  8295 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.168512  8253 raft_consensus.cc:359] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.169062  8295 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:12:26.169207  8175 leader_election.cc:343] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.165032  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:26.169342  8253 raft_consensus.cc:385] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.160912  8308 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:26.162531  8292 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.175895  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:26.175741  8295 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.170439  8253 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.169988  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:26.180331  8061 raft_consensus.cc:2393] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 1: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:12:26.189368  8252 tablet_bootstrap.cc:654] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.190745  8253 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.192898  8175 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:26.195137  8253 ts_tablet_manager.cc:1434] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.003s
I20260709 13:12:26.193645  8308 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.198812  8253 tablet_bootstrap.cc:492] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:26.200120  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:26.201911  8307 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.201953  8061 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.206723  7960 catalog_manager.cc:5697] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 0 to 1, leader changed from <none> to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.211912  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:26.212456  8211 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.213342  8061 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:26.214509  8101 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:26.209781  8273 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.220266  8258 tablet_bootstrap.cc:492] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:26.220749  8258 ts_tablet_manager.cc:1403] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.122s	user 0.015s	sys 0.018s
I20260709 13:12:26.220952  8298 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.221515  8253 tablet_bootstrap.cc:654] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.221483  8298 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:26.222489  8298 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.224934  8211 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:26.227360  7958 catalog_manager.cc:5697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.228762  8258 raft_consensus.cc:359] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.229488  8258 raft_consensus.cc:385] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.229780  8258 raft_consensus.cc:740] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.229175  7958 catalog_manager.cc:5697] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.230381  8253 tablet_bootstrap.cc:492] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:26.230772  8253 ts_tablet_manager.cc:1403] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.000s
I20260709 13:12:26.230669  8258 consensus_queue.cc:260] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.230942  7958 catalog_manager.cc:5697] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.232751  8258 ts_tablet_manager.cc:1434] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.006s
I20260709 13:12:26.232533  7958 catalog_manager.cc:5697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.233096  8253 raft_consensus.cc:359] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.233629  8253 raft_consensus.cc:385] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.233781  8258 tablet_bootstrap.cc:492] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:26.234530  8264 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.233860  8253 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.236097  8264 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.236438  8253 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.238129  8264 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.239902  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:26.240161  8252 tablet_bootstrap.cc:492] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:26.240669  8137 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 1: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:12:26.240666  8252 ts_tablet_manager.cc:1403] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.118s	user 0.013s	sys 0.000s
I20260709 13:12:26.241827  8253 ts_tablet_manager.cc:1434] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:12:26.241940  8258 tablet_bootstrap.cc:654] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.234170  7958 catalog_manager.cc:5697] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.243585  8252 raft_consensus.cc:359] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.244783  8252 raft_consensus.cc:385] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.245044  8252 raft_consensus.cc:740] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.245476  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:26.245659  8252 consensus_queue.cc:260] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.247262  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:26.247661  8252 ts_tablet_manager.cc:1434] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:12:26.244459  8253 tablet_bootstrap.cc:492] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:26.248309  8264 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.248865  8252 tablet_bootstrap.cc:492] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:26.256145  8252 tablet_bootstrap.cc:654] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.257912  8258 tablet_bootstrap.cc:492] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:26.258317  8258 ts_tablet_manager.cc:1403] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.025s	user 0.005s	sys 0.010s
I20260709 13:12:26.260536  8264 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.261020  8258 raft_consensus.cc:359] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.266878  8258 raft_consensus.cc:385] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.267208  8258 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.267861  8258 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.269318  8258 ts_tablet_manager.cc:1434] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.003s
I20260709 13:12:26.263967  8253 tablet_bootstrap.cc:654] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.264220  8252 tablet_bootstrap.cc:492] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:26.269909  8252 ts_tablet_manager.cc:1403] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.000s
I20260709 13:12:26.272060  8258 tablet_bootstrap.cc:492] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:26.273324  8252 raft_consensus.cc:359] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.274266  8252 raft_consensus.cc:385] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.274672  8252 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.275710  8252 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.275671  7958 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.278105  8258 tablet_bootstrap.cc:654] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.277642  7958 catalog_manager.cc:5697] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.292833  8307 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.293416  8307 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.295989  8307 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:26.292186  8252 ts_tablet_manager.cc:1434] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:12:26.298228  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.298602  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:26.299288  8061 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 1: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:12:26.302114  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:26.303257  8307 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.306900  8252 tablet_bootstrap.cc:492] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:26.311473  8307 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.314546  8253 tablet_bootstrap.cc:492] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:26.315006  8253 ts_tablet_manager.cc:1403] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.071s	user 0.012s	sys 0.024s
I20260709 13:12:26.317759  8253 raft_consensus.cc:359] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.318393  8253 raft_consensus.cc:385] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.318636  8253 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.320875  8252 tablet_bootstrap.cc:654] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.321110  8258 tablet_bootstrap.cc:492] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:26.321517  8258 ts_tablet_manager.cc:1403] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.033s
I20260709 13:12:26.323051  8253 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.323778  8258 raft_consensus.cc:359] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.324447  8258 raft_consensus.cc:385] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.324704  8258 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.325412  8253 ts_tablet_manager.cc:1434] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:12:26.325397  8258 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.326795  8253 tablet_bootstrap.cc:492] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:26.327383  8258 ts_tablet_manager.cc:1434] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:12:26.328544  8258 tablet_bootstrap.cc:492] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:26.330498  8264 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.331223  8264 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.333194  8264 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.333385  8253 tablet_bootstrap.cc:654] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.333926  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:26.337289  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:26.338133  8211 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 1: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:12:26.339202  8252 tablet_bootstrap.cc:492] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:26.339658  8252 ts_tablet_manager.cc:1403] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.000s
I20260709 13:12:26.339735  8029 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:26.341045  8258 tablet_bootstrap.cc:654] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.342612  8252 raft_consensus.cc:359] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.343107  8252 raft_consensus.cc:385] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.343351  8252 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.343935  8252 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.345415  8252 ts_tablet_manager.cc:1434] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:12:26.346644  8252 tablet_bootstrap.cc:492] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:26.346493  8264 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.349915  8258 tablet_bootstrap.cc:492] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:26.350386  8258 ts_tablet_manager.cc:1403] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 13:12:26.352216  8252 tablet_bootstrap.cc:654] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.352857  8258 raft_consensus.cc:359] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.352983  8298 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:26.353551  8258 raft_consensus.cc:385] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.353844  8258 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.354676  8258 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.355105  8307 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.355746  8307 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.357304  8258 ts_tablet_manager.cc:1434] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 13:12:26.358304  8307 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:26.361169  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:12:26.364841  8175 leader_election.cc:343] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.362185  8258 tablet_bootstrap.cc:492] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:26.375283  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:26.377468  8175 leader_election.cc:343] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.380143  8258 tablet_bootstrap.cc:654] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.380182  8252 tablet_bootstrap.cc:492] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:26.381381  8252 ts_tablet_manager.cc:1403] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.016s
I20260709 13:12:26.377911  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:26.382774  8307 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.384008  8252 raft_consensus.cc:359] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.384694  8252 raft_consensus.cc:385] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.384982  8252 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.385637  8252 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.387542  8252 ts_tablet_manager.cc:1434] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:12:26.387444  8307 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.388471  8252 tablet_bootstrap.cc:492] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:26.388114  8307 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.391743  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.392731  8061 raft_consensus.cc:2393] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 1: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:12:26.394011  8307 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:26.396792  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:26.400332  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:26.400403  8295 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:26.401000  8252 tablet_bootstrap.cc:654] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.401497  8307 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.402328  8258 tablet_bootstrap.cc:492] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:26.402812  8258 ts_tablet_manager.cc:1403] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.002s
I20260709 13:12:26.403776  8253 tablet_bootstrap.cc:492] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:26.404197  8253 ts_tablet_manager.cc:1403] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.078s	user 0.022s	sys 0.026s
I20260709 13:12:26.405535  8258 raft_consensus.cc:359] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.406410  8258 raft_consensus.cc:385] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.406764  8258 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.407107  8253 raft_consensus.cc:359] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.407766  8253 raft_consensus.cc:385] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.407724  8258 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.408530  8253 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.409585  8253 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.419059  8253 ts_tablet_manager.cc:1434] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:12:26.419665  8308 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.420058  8253 tablet_bootstrap.cc:492] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:26.420123  8308 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.422152  8308 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:26.423911  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:26.424553  8137 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:26.426303  8258 ts_tablet_manager.cc:1434] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:12:26.426222  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.426692  8062 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.426952  8063 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:26.428395  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:26.429136  8308 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.429473  8308 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.429803  8308 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.437969  8258 tablet_bootstrap.cc:492] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:26.437747  8308 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.438687  8295 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:26.440385  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:26.440943  8063 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.446133  8258 tablet_bootstrap.cc:654] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.448160  8063 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:26.449664  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:26.450572  8311 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.451079  8311 raft_consensus.cc:697] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:26.451922  8311 consensus_queue.cc:237] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.457665  8258 tablet_bootstrap.cc:492] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:26.458132  8258 ts_tablet_manager.cc:1403] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.006s
I20260709 13:12:26.461347  8258 raft_consensus.cc:359] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.462419  8258 raft_consensus.cc:385] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.462764  8258 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.463757  8258 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.464722  8253 tablet_bootstrap.cc:654] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.476698  8258 ts_tablet_manager.cc:1434] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:12:26.487977  8258 tablet_bootstrap.cc:492] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
W20260709 13:12:26.504351  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 54776.4us
I20260709 13:12:26.504546  8292 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:26.505268  8258 tablet_bootstrap.cc:654] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.512702  8305 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:26.518689  8305 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.519436  8305 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.523259  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:26.525094  8101 leader_election.cc:343] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.525580  8308 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:26.526465  8305 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.530381  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:26.531281  8209 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:26.536192  8104 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:26.537933  8292 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.538470  8292 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.538825  8292 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.538975  8298 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:26.549371  8305 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:26.545921  8292 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.554080  8252 tablet_bootstrap.cc:492] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:26.554821  8252 ts_tablet_manager.cc:1403] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.167s	user 0.038s	sys 0.061s
I20260709 13:12:26.563083  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:26.567988  7960 catalog_manager.cc:5697] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 0 to 1, leader changed from <none> to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:26.558133  8273 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
W20260709 13:12:26.570269  8101 leader_election.cc:343] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.571640  8292 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.572861  8252 raft_consensus.cc:359] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.573666  8252 raft_consensus.cc:385] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.573961  8252 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.575098  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:26.575855  8137 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.578176  8252 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.584062  8264 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.597962  8292 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.598513  8292 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.583302  8307 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.611866  8252 ts_tablet_manager.cc:1434] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 13:12:26.613118  8252 tablet_bootstrap.cc:492] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:26.600507  8308 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.663691  8273 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:26.671715  8307 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:12:26.673455  8258 tablet_bootstrap.cc:492] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:26.673966  8258 ts_tablet_manager.cc:1403] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.186s	user 0.031s	sys 0.014s
I20260709 13:12:26.673729  8311 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:12:26.675302  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 128575us
I20260709 13:12:26.678786  8258 raft_consensus.cc:359] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.679683  8258 raft_consensus.cc:385] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.680012  8258 raft_consensus.cc:740] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.680852  8258 consensus_queue.cc:260] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.683060  8352 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.683795  8352 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.656415  8308 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.690428  8258 ts_tablet_manager.cc:1434] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:12:26.691797  8258 tablet_bootstrap.cc:492] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:26.693790  8252 tablet_bootstrap.cc:654] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.698910  8308 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:26.700357  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.701813  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:12:26.703028  8310 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.088s	user 0.019s	sys 0.026s
I20260709 13:12:26.705288  8292 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:12:26.706073  8175 leader_election.cc:343] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:26.706729  8175 leader_election.cc:343] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.707134  8175 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:26.707810  8311 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.714236  8352 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.714743  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:26.721163  8210 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.724166  8253 tablet_bootstrap.cc:492] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:26.725206  8253 ts_tablet_manager.cc:1403] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.305s	user 0.077s	sys 0.089s
I20260709 13:12:26.724903  8352 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.728729  8137 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:26.731441  8352 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.726547  8305 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:12:26.733491  8352 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.733662  8292 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:26.750665  8253 raft_consensus.cc:359] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.751564  8253 raft_consensus.cc:385] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.751886  8253 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.753882  8253 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.756012  8253 ts_tablet_manager.cc:1434] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.001s
I20260709 13:12:26.760507  8258 tablet_bootstrap.cc:654] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.762195  8253 tablet_bootstrap.cc:492] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:26.766957  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.776986  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:26.777863  8208 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:26.779562  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:26.780054  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:26.781276  8252 tablet_bootstrap.cc:492] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:26.782299  8252 ts_tablet_manager.cc:1403] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.169s	user 0.019s	sys 0.010s
I20260709 13:12:26.780175  8207 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:26.789233  8273 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.789697  8273 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.791695  8273 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.797421  8351 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.797581  8104 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:26.787897  8292 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.798200  8292 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.797902  8351 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.798612  8292 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.800045  8351 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.800580  8351 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.800798  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.800957  8351 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.804857  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.805410  8253 tablet_bootstrap.cc:654] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:26.805462  8062 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
W20260709 13:12:26.797912  8101 leader_election.cc:343] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.807126  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:26.807703  8209 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:26.808425  8305 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:26.809907  8252 raft_consensus.cc:359] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.812229  8252 raft_consensus.cc:385] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.812594  8252 raft_consensus.cc:740] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.809350  8351 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.808830  8295 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:12:26.822445  8350 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
W20260709 13:12:26.810122  8101 leader_election.cc:343] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.810657  8273 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:12:26.811650  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.820370  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:26.824198  8104 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:26.841328  8350 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.841110  8292 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.841843  8350 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.842299  8350 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.865340  8210 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
W20260709 13:12:26.867619  8104 leader_election.cc:343] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.867787  8292 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.873487  8351 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.870723  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:26.879667  8258 tablet_bootstrap.cc:492] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:26.879925  8351 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.880116  8258 ts_tablet_manager.cc:1403] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.189s	user 0.018s	sys 0.024s
I20260709 13:12:26.871783  8252 consensus_queue.cc:260] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.883757  8258 raft_consensus.cc:359] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.884450  8258 raft_consensus.cc:385] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.884688  8264 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.884802  8258 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.885174  8264 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.885630  8258 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.886087  8350 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.884469  8252 ts_tablet_manager.cc:1434] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.102s	user 0.003s	sys 0.005s
I20260709 13:12:26.888512  8252 tablet_bootstrap.cc:492] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:26.888609  8350 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.889853  8258 ts_tablet_manager.cc:1434] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:12:26.890731  8258 tablet_bootstrap.cc:492] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:26.896214  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:26.896780  8208 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.897333  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:26.898643  8212 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.906816  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:26.908030  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:26.908152  8210 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.908708  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:26.917023  8104 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:26.918038  8307 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:12:26.922465  8350 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.922935  8350 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:12:26.923879  8026 leader_election.cc:343] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.923756  8350 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.926554  8264 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.928036  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:26.928610  8211 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:26.929829  8029 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:26.930895  8264 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:26.931243  8264 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:26.931569  8264 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.932433  8253 tablet_bootstrap.cc:492] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
W20260709 13:12:26.935683  8104 leader_election.cc:343] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.936106  8104 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:26.937307  8208 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:26.937876  8253 ts_tablet_manager.cc:1403] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.177s	user 0.010s	sys 0.007s
I20260709 13:12:26.939024  8351 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.943408  8351 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:26.944049  8253 raft_consensus.cc:359] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.947939  8264 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.948323  8210 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:26.948490  8104 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:26.957986  8253 raft_consensus.cc:385] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:26.958259  8351 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:26.958751  8253 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:26.951704  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:26.952452  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:26.959872  8057 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.952625  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:26.949162  8258 tablet_bootstrap.cc:654] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:26.955621  8101 leader_election.cc:343] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.958858  8351 raft_consensus.cc:697] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:26.952960  8212 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:26.960985  8253 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:26.962045  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:12:26.961712  8101 leader_election.cc:343] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.964790  8058 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.965953  8101 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:12:26.968037  8101 leader_election.cc:343] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:26.969832  8253 ts_tablet_manager.cc:1434] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:12:26.970371  8292 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:26.965489  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:26.971918  8210 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:26.960444  8264 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:26.977524  8351 consensus_queue.cc:237] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:26.983878  8026 leader_election.cc:343] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.009694  8307 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:26.986202  8104 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:27.025967  8307 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.027559  8253 tablet_bootstrap.cc:492] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:27.031678  8353 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.032198  8353 raft_consensus.cc:697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:12:27.029901  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 47140.6us
I20260709 13:12:27.033028  8353 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.033242  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:27.034363  8252 tablet_bootstrap.cc:654] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.035811  8350 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.036306  8350 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:27.037117  8350 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.030200  8308 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:12:27.028342  8057 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:27.044639  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:27.050208  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:27.051190  8258 tablet_bootstrap.cc:492] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:27.051494  8258 ts_tablet_manager.cc:1403] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.161s	user 0.013s	sys 0.017s
W20260709 13:12:27.050771  8175 leader_election.cc:343] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:27.055270  8175 leader_election.cc:343] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.055684  8175 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:27.056466  8308 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:27.057618  8058 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:27.059072  8258 raft_consensus.cc:359] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.059719  8258 raft_consensus.cc:385] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.059994  8258 raft_consensus.cc:740] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.060837  8210 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:27.060634  8258 consensus_queue.cc:260] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.045701  8307 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:27.065397  8258 ts_tablet_manager.cc:1434] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:12:27.066535  8258 tablet_bootstrap.cc:492] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:27.074820  8253 tablet_bootstrap.cc:654] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.080061  8350 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.080507  8350 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.082423  8350 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.084728  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:27.085561  8208 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:27.087105  8258 tablet_bootstrap.cc:654] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.087136  8104 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:27.088279  8352 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.088627  8352 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.088977  8352 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.067553  8029 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:27.090520  8264 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.091115  8264 raft_consensus.cc:697] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:27.092054  8264 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.095824  8252 tablet_bootstrap.cc:492] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:27.096271  8252 ts_tablet_manager.cc:1403] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.208s	user 0.009s	sys 0.005s
I20260709 13:12:27.098774  8252 raft_consensus.cc:359] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.099406  8252 raft_consensus.cc:385] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.099659  8252 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.100292  8252 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.102238  8252 ts_tablet_manager.cc:1434] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:12:27.102424  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:27.103086  8252 tablet_bootstrap.cc:492] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
W20260709 13:12:27.103768  8101 leader_election.cc:343] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.102715  8352 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.111300  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:27.111858  8208 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.112259  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:12:27.113389  8101 leader_election.cc:343] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.120183  8208 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:27.121413  8104 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:27.122200  8352 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.122963  8352 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.125434  8352 raft_consensus.cc:697] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:27.128409  8352 consensus_queue.cc:237] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.131289  8252 tablet_bootstrap.cc:654] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.122998  7958 catalog_manager.cc:5697] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:27.135358  7958 catalog_manager.cc:5697] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:27.140166  8009 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:27.148797  8252 tablet_bootstrap.cc:492] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:27.149259  8252 ts_tablet_manager.cc:1403] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.005s
I20260709 13:12:27.152601  8252 raft_consensus.cc:359] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.153228  8252 raft_consensus.cc:385] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.153492  8252 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.154268  8253 tablet_bootstrap.cc:492] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:27.154143  8252 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.154697  8253 ts_tablet_manager.cc:1403] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.133s	user 0.022s	sys 0.001s
I20260709 13:12:27.154862  8258 tablet_bootstrap.cc:492] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:27.155526  8258 ts_tablet_manager.cc:1403] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.089s	user 0.020s	sys 0.008s
I20260709 13:12:27.156124  8252 ts_tablet_manager.cc:1434] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:12:27.156950  8252 tablet_bootstrap.cc:492] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:27.157315  8253 raft_consensus.cc:359] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.157932  8253 raft_consensus.cc:385] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.158315  8253 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.157927  8258 raft_consensus.cc:359] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.158886  8258 raft_consensus.cc:385] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.159178  8258 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.158965  8253 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.161242  8253 ts_tablet_manager.cc:1434] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.004s
I20260709 13:12:27.162585  8252 tablet_bootstrap.cc:654] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.163130  8253 tablet_bootstrap.cc:492] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:27.159893  8258 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.168282  8258 ts_tablet_manager.cc:1434] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:12:27.169411  8258 tablet_bootstrap.cc:492] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:27.181823  8253 tablet_bootstrap.cc:654] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.182583  8258 tablet_bootstrap.cc:654] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.182953  7959 catalog_manager.cc:5697] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:27.186352  8252 tablet_bootstrap.cc:492] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:27.186939  8252 ts_tablet_manager.cc:1403] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.011s
I20260709 13:12:27.190862  8252 raft_consensus.cc:359] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.191849  8252 raft_consensus.cc:385] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.192193  8252 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.193136  8252 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.195923  8252 ts_tablet_manager.cc:1434] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:12:27.197225  8252 tablet_bootstrap.cc:492] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:27.202744  8252 tablet_bootstrap.cc:654] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.214597  8252 tablet_bootstrap.cc:492] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:27.215130  8252 ts_tablet_manager.cc:1403] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.007s
I20260709 13:12:27.218281  8252 raft_consensus.cc:359] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.219295  8252 raft_consensus.cc:385] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.219647  8252 raft_consensus.cc:740] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.220609  8252 consensus_queue.cc:260] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.223325  8252 ts_tablet_manager.cc:1434] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:12:27.224443  8252 tablet_bootstrap.cc:492] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:27.230906  8264 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.231407  8264 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.232086  8252 tablet_bootstrap.cc:654] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.233440  8264 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.235087  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:27.235221  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:27.235913  8208 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 1: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:12:27.238726  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:27.239691  8264 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.246016  8264 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:27.245349  7958 catalog_manager.cc:5697] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:27.248827  8253 tablet_bootstrap.cc:492] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:27.249109  8252 tablet_bootstrap.cc:492] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:27.249248  8253 ts_tablet_manager.cc:1403] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.086s	user 0.023s	sys 0.039s
I20260709 13:12:27.249523  8252 ts_tablet_manager.cc:1403] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.025s	user 0.006s	sys 0.011s
I20260709 13:12:27.248657  7958 catalog_manager.cc:5697] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:27.250097  8353 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.251148  8353 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.250707  7958 catalog_manager.cc:5697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:27.251828  8253 raft_consensus.cc:359] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.252467  8253 raft_consensus.cc:385] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.252708  8253 raft_consensus.cc:740] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.253506  8353 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.253317  8253 consensus_queue.cc:260] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.254648  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:27.255249  8208 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:27.255370  8253 ts_tablet_manager.cc:1434] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:12:27.255524  8252 raft_consensus.cc:359] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.256268  8253 tablet_bootstrap.cc:492] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:27.256157  8252 raft_consensus.cc:385] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.256143  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:27.256403  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:27.256767  8252 raft_consensus.cc:740] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.256958  8057 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:27.257709  8353 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.257498  8252 consensus_queue.cc:260] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.258081  8353 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.258453  8353 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.259472  8252 ts_tablet_manager.cc:1434] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:12:27.260368  8252 tablet_bootstrap.cc:492] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:27.262571  8253 tablet_bootstrap.cc:654] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.262871  8258 tablet_bootstrap.cc:492] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:27.263298  8258 ts_tablet_manager.cc:1403] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.094s	user 0.020s	sys 0.032s
I20260709 13:12:27.265698  8252 tablet_bootstrap.cc:654] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.265775  8353 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.265918  8258 raft_consensus.cc:359] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.266873  8258 raft_consensus.cc:385] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.267235  8258 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.268915  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:27.269613  8057 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.280277  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:27.281023  8208 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.291939  8208 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:27.292580  8353 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.292970  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:27.293462  8057 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:27.294242  8353 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.295775  8308 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.296207  8308 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.297957  8308 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:27.298610  8258 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.300037  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:27.300930  8258 ts_tablet_manager.cc:1434] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
W20260709 13:12:27.301570  8175 leader_election.cc:343] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.302317  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:27.303526  8175 leader_election.cc:343] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.303939  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:27.301843  8258 tablet_bootstrap.cc:492] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:27.310073  8253 tablet_bootstrap.cc:492] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:27.314908  8253 ts_tablet_manager.cc:1403] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.059s	user 0.016s	sys 0.024s
I20260709 13:12:27.310660  8352 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.315618  8352 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.317682  8352 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.317517  8253 raft_consensus.cc:359] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.318310  8253 raft_consensus.cc:385] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.319334  8261 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:27.318667  8253 raft_consensus.cc:740] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.319154  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:27.319058  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:27.321600  8208 raft_consensus.cc:2393] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 1: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:12:27.295440  8353 raft_consensus.cc:697] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:27.324311  8253 consensus_queue.cc:260] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.324096  8353 consensus_queue.cc:237] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.326051  8104 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:27.326472  8253 ts_tablet_manager.cc:1434] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.004s
I20260709 13:12:27.327299  8292 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.327505  8253 tablet_bootstrap.cc:492] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:27.333499  8253 tablet_bootstrap.cc:654] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.341513  8292 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:27.314491  8258 tablet_bootstrap.cc:654] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.344519  8253 tablet_bootstrap.cc:492] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:27.345006  8253 ts_tablet_manager.cc:1403] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.005s
I20260709 13:12:27.347465  8253 raft_consensus.cc:359] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.348073  8253 raft_consensus.cc:385] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.348356  8253 raft_consensus.cc:740] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.349406  8253 consensus_queue.cc:260] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.351476  8253 ts_tablet_manager.cc:1434] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:12:27.357523  8253 tablet_bootstrap.cc:492] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:27.360975  8252 tablet_bootstrap.cc:492] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:27.361438  8252 ts_tablet_manager.cc:1403] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.101s	user 0.029s	sys 0.056s
I20260709 13:12:27.362210  8258 tablet_bootstrap.cc:492] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:27.362767  8258 ts_tablet_manager.cc:1403] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.061s	user 0.009s	sys 0.003s
I20260709 13:12:27.362262  7958 catalog_manager.cc:5697] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:27.363847  8253 tablet_bootstrap.cc:654] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.364001  8252 raft_consensus.cc:359] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.364722  8252 raft_consensus.cc:385] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.365000  8252 raft_consensus.cc:740] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.365188  8258 raft_consensus.cc:359] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.365856  8258 raft_consensus.cc:385] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.365664  8252 consensus_queue.cc:260] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.366139  8258 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.367982  8252 ts_tablet_manager.cc:1434] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:12:27.369102  8252 tablet_bootstrap.cc:492] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:27.375721  8258 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.377986  8258 ts_tablet_manager.cc:1434] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:12:27.378489  8252 tablet_bootstrap.cc:654] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.378816  8258 tablet_bootstrap.cc:492] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:27.389746  8057 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.402225  8353 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:27.410576  8258 tablet_bootstrap.cc:654] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.411391  8252 tablet_bootstrap.cc:492] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:27.411413  8253 tablet_bootstrap.cc:492] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:27.411934  8253 ts_tablet_manager.cc:1403] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.055s	user 0.021s	sys 0.008s
I20260709 13:12:27.411934  8252 ts_tablet_manager.cc:1403] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.043s	user 0.017s	sys 0.004s
I20260709 13:12:27.414552  8253 raft_consensus.cc:359] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.415177  8253 raft_consensus.cc:385] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.415454  8253 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.416150  8252 raft_consensus.cc:359] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.417075  8252 raft_consensus.cc:385] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.416211  8253 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.417384  8252 raft_consensus.cc:740] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.418104  8252 consensus_queue.cc:260] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.420162  8252 ts_tablet_manager.cc:1434] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
W20260709 13:12:27.422333  8101 consensus_peers.cc:597] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:27.423238  8252 tablet_bootstrap.cc:492] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:27.423547  8350 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:27.424518  8253 ts_tablet_manager.cc:1434] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:12:27.425624  8253 tablet_bootstrap.cc:492] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:27.429239  8252 tablet_bootstrap.cc:654] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.431463  8253 tablet_bootstrap.cc:654] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.438581  8258 tablet_bootstrap.cc:492] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:27.439090  8258 ts_tablet_manager.cc:1403] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.005s
I20260709 13:12:27.441573  8253 tablet_bootstrap.cc:492] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:27.441998  8253 ts_tablet_manager.cc:1403] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.004s
I20260709 13:12:27.441704  8258 raft_consensus.cc:359] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.442330  8258 raft_consensus.cc:385] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.442652  8258 raft_consensus.cc:740] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.444862  8253 raft_consensus.cc:359] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.445580  8253 raft_consensus.cc:385] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.445387  8258 consensus_queue.cc:260] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.445868  8253 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.446480  8253 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.447633  8308 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.448050  8308 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.448626  8253 ts_tablet_manager.cc:1434] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:12:27.449494  8253 tablet_bootstrap.cc:492] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:27.449914  8308 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:27.450104  8252 tablet_bootstrap.cc:492] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:27.450636  8252 ts_tablet_manager.cc:1403] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.000s
I20260709 13:12:27.450816  8258 ts_tablet_manager.cc:1434] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:12:27.451630  8258 tablet_bootstrap.cc:492] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:27.452064  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:27.453156  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:27.453469  8252 raft_consensus.cc:359] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.454280  8252 raft_consensus.cc:385] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.454612  8252 raft_consensus.cc:740] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.454907  8253 tablet_bootstrap.cc:654] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:27.455089  8175 leader_election.cc:343] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:27.455765  8175 leader_election.cc:343] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.456175  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:27.455969  8252 consensus_queue.cc:260] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.457273  8261 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:27.458458  8252 ts_tablet_manager.cc:1434] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:27.459380  8252 tablet_bootstrap.cc:492] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:27.459268  8350 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:27.465375  8252 tablet_bootstrap.cc:654] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.465946  8261 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.466457  8261 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.468529  8261 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:27.470525  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:27.471315  8057 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:27.471604  8258 tablet_bootstrap.cc:654] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.474748  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:27.476289  8352 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:27.476619  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:27.477989  8138 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:27.479720  8308 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.480211  8308 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.480634  8308 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.499154  8352 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:27.507081  8258 tablet_bootstrap.cc:492] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:27.507680  8258 ts_tablet_manager.cc:1403] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.001s
I20260709 13:12:27.514686  8258 raft_consensus.cc:359] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.515307  8258 raft_consensus.cc:385] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.515381  8253 tablet_bootstrap.cc:492] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:27.515590  8258 raft_consensus.cc:740] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.515794  8253 ts_tablet_manager.cc:1403] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.066s	user 0.022s	sys 0.025s
I20260709 13:12:27.516360  8258 consensus_queue.cc:260] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.518170  8253 raft_consensus.cc:359] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.520881  8252 tablet_bootstrap.cc:492] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:27.521319  8252 ts_tablet_manager.cc:1403] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.062s	user 0.019s	sys 0.030s
I20260709 13:12:27.518570  8258 ts_tablet_manager.cc:1434] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:12:27.522512  8258 tablet_bootstrap.cc:492] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:27.523947  8252 raft_consensus.cc:359] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.524585  8252 raft_consensus.cc:385] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.518837  8253 raft_consensus.cc:385] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.524844  8252 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.524935  8253 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.525548  8252 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.525743  8253 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.527911  8253 ts_tablet_manager.cc:1434] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:12:27.527508  8308 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.530983  8308 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:27.528092  8252 ts_tablet_manager.cc:1434] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:12:27.531697  8258 tablet_bootstrap.cc:654] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.532337  8253 tablet_bootstrap.cc:492] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:27.532362  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:27.533063  8138 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.539187  8252 tablet_bootstrap.cc:492] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:27.547458  8252 tablet_bootstrap.cc:654] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.549805  8138 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:27.552218  8253 tablet_bootstrap.cc:654] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.553212  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:27.556263  8057 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.559278  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:27.560956  8308 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.561635  8308 raft_consensus.cc:697] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:27.562665  8308 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.561039  8264 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.565330  8264 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.569301  8252 tablet_bootstrap.cc:492] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:27.569072  8352 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:27.570098  8252 ts_tablet_manager.cc:1403] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.002s
I20260709 13:12:27.570366  8264 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.569929  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:27.571312  8208 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:27.572453  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:27.577374  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:27.578286  8138 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:27.580124  8352 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:27.576332  8264 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.582194  8264 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.582650  8264 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.586212  8352 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.586702  8352 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.589078  8252 raft_consensus.cc:359] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.589740  8252 raft_consensus.cc:385] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.589622  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:27.590044  8252 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.591022  8352 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.591486  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:12:27.591698  8101 leader_election.cc:343] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.592082  8210 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:27.593521  8104 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:27.593564  8252 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.595295  8351 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.595647  8351 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.595992  8351 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.598187  8253 tablet_bootstrap.cc:492] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:27.595821  8252 ts_tablet_manager.cc:1434] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:12:27.598630  8253 ts_tablet_manager.cc:1403] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.066s	user 0.014s	sys 0.012s
I20260709 13:12:27.599304  8252 tablet_bootstrap.cc:492] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:27.600885  8253 raft_consensus.cc:359] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.601508  8253 raft_consensus.cc:385] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.601774  8253 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.602584  8253 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.604683  8253 ts_tablet_manager.cc:1434] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 13:12:27.605521  8253 tablet_bootstrap.cc:492] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:27.607581  8057 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:27.607704  8252 tablet_bootstrap.cc:654] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.610862  8253 tablet_bootstrap.cc:654] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.611827  8351 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.615414  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:27.615976  8367 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:27.615981  8061 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.622542  8308 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.623016  8308 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.624928  8308 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:27.626430  8061 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:27.627540  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:27.628814  8210 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.629312  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:27.627849  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:27.633074  8101 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:12:27.633046  8175 leader_election.cc:343] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.634853  8252 tablet_bootstrap.cc:492] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
W20260709 13:12:27.635140  8175 leader_election.cc:343] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.635386  8252 ts_tablet_manager.cc:1403] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.004s
I20260709 13:12:27.635667  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:27.637056  8261 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:27.637208  8210 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:27.637951  8258 tablet_bootstrap.cc:492] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:27.638068  8353 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.638463  8258 ts_tablet_manager.cc:1403] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.116s	user 0.027s	sys 0.005s
I20260709 13:12:27.638558  8252 raft_consensus.cc:359] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.638767  8353 raft_consensus.cc:697] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:27.640739  8252 raft_consensus.cc:385] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.641166  8252 raft_consensus.cc:740] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.641052  8258 raft_consensus.cc:359] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.641808  8258 raft_consensus.cc:385] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.641390  8353 consensus_queue.cc:237] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.641976  8252 consensus_queue.cc:260] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.643112  8253 tablet_bootstrap.cc:492] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:27.643577  8253 ts_tablet_manager.cc:1403] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.004s
I20260709 13:12:27.642086  8258 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.643114  8273 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.644747  8273 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.646063  8292 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:27.647056  8252 ts_tablet_manager.cc:1434] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 13:12:27.648293  8252 tablet_bootstrap.cc:492] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:27.613941  8351 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.648418  8258 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.654592  8253 raft_consensus.cc:359] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.659749  8258 ts_tablet_manager.cc:1434] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:12:27.648895  8264 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.655426  8253 raft_consensus.cc:385] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.666689  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:27.673437  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:27.666024  8273 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.670109  8264 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.668339  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:27.663892  8253 raft_consensus.cc:740] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.676155  8210 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.669629  8367 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.678550  8367 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.679896  8058 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:27.681331  8308 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.670516  8292 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:12:27.674993  8258 tablet_bootstrap.cc:492] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:27.689395  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:27.695552  8212 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:27.696244  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:27.696820  8207 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:27.697072  8252 tablet_bootstrap.cc:654] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.698043  8029 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:27.698693  8367 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.698824  8253 consensus_queue.cc:260] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.681777  8308 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.683856  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:27.679477  8138 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.693544  8210 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:27.703292  8418 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:27.667922  7959 catalog_manager.cc:5697] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 0 to 1, leader changed from <none> to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:27.707602  8253 ts_tablet_manager.cc:1434] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.064s	user 0.009s	sys 0.000s
I20260709 13:12:27.709311  8308 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:27.712016  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:27.713055  8351 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.713577  8351 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.713814  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:27.705049  8367 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.715157  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:27.713022  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:27.715358  8367 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.716985  8367 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.713938  8351 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.716616  8418 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.718394  8418 raft_consensus.cc:697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:27.723492  8418 consensus_queue.cc:237] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.724304  8138 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:27.720603  8261 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.725550  8261 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.727474  8261 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:27.737915  8253 tablet_bootstrap.cc:492] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
W20260709 13:12:27.736090  8175 leader_election.cc:343] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:27.742252  8175 leader_election.cc:343] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.747061  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:27.754714  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:27.768473  8308 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.770203  8308 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.841930  7958 catalog_manager.cc:5697] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:27.753950  8261 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:27.845835  8252 tablet_bootstrap.cc:492] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
W20260709 13:12:27.791421  8175 leader_election.cc:343] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.847755  8252 ts_tablet_manager.cc:1403] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.200s	user 0.031s	sys 0.033s
I20260709 13:12:27.784549  8417 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.784809  8367 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:27.847488  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 56130.9us
I20260709 13:12:27.851116  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:27.857389  8353 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.857879  8353 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.859926  8353 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.854847  8308 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:27.845862  8424 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.867719  8252 raft_consensus.cc:359] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.869226  8252 raft_consensus.cc:385] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:27.869580  8252 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:27.865973  8424 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.883503  8424 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:27.921913  8253 tablet_bootstrap.cc:654] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.851707  8417 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.923601  8252 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.927834  8252 ts_tablet_manager.cc:1434] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.080s	user 0.028s	sys 0.019s
I20260709 13:12:27.927676  8351 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.921716  8367 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.928794  8137 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 1: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:12:27.929507  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:27.934883  8175 leader_election.cc:343] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.935266  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:27.935345  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:27.935662  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:27.936233  8424 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:27.935443  8061 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:27.944166  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:27.944506  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:27.944911  8136 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 1: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:12:27.944840  8417 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.946671  8061 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.951282  8252 tablet_bootstrap.cc:492] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:27.948551  8424 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:27.951473  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:27.952035  8212 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:27.951857  8424 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.951427  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:27.954157  8210 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.955809  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:12:27.963099  8175 leader_election.cc:343] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:27.964148  8175 leader_election.cc:343] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.965095  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:12:27.965435  8175 leader_election.cc:343] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.965698  8207 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:27.965847  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:27.966609  8308 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:27.966776  8029 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:27.967819  8367 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:27.968156  8367 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:27.968458  8367 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:27.968947  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:27.972946  8210 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:27.974229  8029 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:27.975178  8418 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:27.975785  8418 raft_consensus.cc:697] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:27.975884  7959 catalog_manager.cc:5697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:27.978410  8367 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:27.978052  8175 leader_election.cc:343] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:27.976845  8418 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:27.979207  8351 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:27.992923  8424 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:27.979784  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:28.003705  8258 tablet_bootstrap.cc:654] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:27.981546  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:28.004614  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:28.006637  8353 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.007452  8210 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 1: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:12:28.007264  8353 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.008227  8292 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:28.008987  8353 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.005404  8351 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.009516  8351 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.009812  8351 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.005578  8424 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.012152  8367 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.013326  8273 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:28.015398  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:28.015477  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:28.016157  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:28.016782  8207 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
W20260709 13:12:28.017079  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 49422us
I20260709 13:12:28.016710  8253 tablet_bootstrap.cc:492] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:28.017844  8253 ts_tablet_manager.cc:1403] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.280s	user 0.026s	sys 0.013s
I20260709 13:12:28.021651  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:28.022861  8104 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:28.023753  8417 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:28.023931  8273 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.024479  8273 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.024274  8417 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.024974  8253 raft_consensus.cc:359] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.026400  8253 raft_consensus.cc:385] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.026643  8253 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.027153  8253 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.024956  8273 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.032184  8351 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.033835  8136 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 1: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:12:28.035285  8273 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.038843  7958 catalog_manager.cc:5697] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:28.043256  8351 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:12:28.026088  8175 leader_election.cc:343] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.045611  8253 ts_tablet_manager.cc:1434] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.000s
I20260709 13:12:28.046298  8253 tablet_bootstrap.cc:492] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:28.047451  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:28.038564  8273 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.040575  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:28.051748  8417 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:12:28.052758  8175 leader_election.cc:343] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.052664  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:28.052968  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:28.054909  8208 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.059202  8253 tablet_bootstrap.cc:654] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.053288  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:28.060875  8424 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.062075  8208 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:28.052714  8210 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.064930  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:28.066232  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:28.066795  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:28.071292  8061 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:28.072175  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:28.072988  8353 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.068727  8252 tablet_bootstrap.cc:654] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.069380  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:28.069753  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:28.074425  8058 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 1: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:12:28.070387  8104 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:28.076891  8273 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:28.077349  8273 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:28.078186  8273 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.079990  8026 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:28.081029  8417 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:28.085641  8258 tablet_bootstrap.cc:492] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:28.086067  8258 ts_tablet_manager.cc:1403] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.420s	user 0.011s	sys 0.001s
I20260709 13:12:28.090030  8210 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:28.091019  8029 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:28.091636  8417 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:28.092094  8417 raft_consensus.cc:697] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:28.092697  8417 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.095412  8104 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:28.102496  8353 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.099001  8258 raft_consensus.cc:359] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.103662  8258 raft_consensus.cc:385] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.103946  8258 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.104573  8258 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.105486  8350 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:28.107601  8252 tablet_bootstrap.cc:492] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:28.108108  8253 tablet_bootstrap.cc:492] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:28.108289  8252 ts_tablet_manager.cc:1403] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.157s	user 0.014s	sys 0.005s
I20260709 13:12:28.108579  8253 ts_tablet_manager.cc:1403] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.062s	user 0.009s	sys 0.000s
I20260709 13:12:28.111557  8253 raft_consensus.cc:359] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.112273  8253 raft_consensus.cc:385] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.112538  8253 raft_consensus.cc:740] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.113204  8253 consensus_queue.cc:260] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.111457  8252 raft_consensus.cc:359] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.115316  8252 raft_consensus.cc:385] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.115590  8252 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.115720  8253 ts_tablet_manager.cc:1434] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:12:28.116814  8253 tablet_bootstrap.cc:492] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:28.117297  7958 catalog_manager.cc:5697] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:28.116336  8252 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.119140  8258 ts_tablet_manager.cc:1434] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:12:28.119948  8258 tablet_bootstrap.cc:492] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:28.123333  8253 tablet_bootstrap.cc:654] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.129405  8252 ts_tablet_manager.cc:1434] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.002s
I20260709 13:12:28.130385  8252 tablet_bootstrap.cc:492] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:28.139254  8258 tablet_bootstrap.cc:654] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.142789  8424 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.143253  8424 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.145198  8424 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:28.146591  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:28.146997  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:12:28.148175  8175 leader_election.cc:343] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.151091  8353 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:12:28.161659  8175 leader_election.cc:343] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.166935  8175 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:28.167685  8424 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.170105  8424 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:28.172709  8253 tablet_bootstrap.cc:492] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:28.172861  7961 catalog_manager.cc:5697] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:28.173367  8253 ts_tablet_manager.cc:1403] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.019s
I20260709 13:12:28.182969  8253 raft_consensus.cc:359] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.183635  8253 raft_consensus.cc:385] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.183897  8253 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.184482  8253 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.192004  8253 ts_tablet_manager.cc:1434] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:12:28.192847  8253 tablet_bootstrap.cc:492] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:28.213860  8424 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:28.217033  8252 tablet_bootstrap.cc:654] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.221378  8367 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:28.228780  8258 tablet_bootstrap.cc:492] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:28.229367  8258 ts_tablet_manager.cc:1403] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.110s	user 0.016s	sys 0.024s
I20260709 13:12:28.266796  8253 tablet_bootstrap.cc:654] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.270032  8258 raft_consensus.cc:359] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.270731  8258 raft_consensus.cc:385] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.270996  8258 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.271636  8258 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.275156  8258 ts_tablet_manager.cc:1434] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.045s	user 0.005s	sys 0.013s
W20260709 13:12:28.275996  8009 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:28.277541  8252 tablet_bootstrap.cc:492] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:28.277961  8252 ts_tablet_manager.cc:1403] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.148s	user 0.008s	sys 0.009s
I20260709 13:12:28.276006  8258 tablet_bootstrap.cc:492] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:28.289736  8252 raft_consensus.cc:359] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.290448  8252 raft_consensus.cc:385] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.290761  8252 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.291479  8252 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.294538  8252 ts_tablet_manager.cc:1434] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.003s
I20260709 13:12:28.295416  8252 tablet_bootstrap.cc:492] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:28.298707  8264 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:28.303128  8258 tablet_bootstrap.cc:654] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.309631  8252 tablet_bootstrap.cc:654] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.310245  8264 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.310721  8264 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.313421  8264 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.314067  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:28.314637  8136 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:28.315639  8026 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:28.316411  8264 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.317137  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:28.317700  8210 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:28.316769  8264 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.318739  8264 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.326320  8264 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.328907  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:28.330710  8136 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.331847  8253 tablet_bootstrap.cc:492] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:28.332307  8253 ts_tablet_manager.cc:1403] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.140s	user 0.009s	sys 0.000s
I20260709 13:12:28.335238  8136 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:28.335341  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:28.335891  8210 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.335542  8253 raft_consensus.cc:359] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.336433  8253 raft_consensus.cc:385] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.336784  8253 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.337561  8253 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.339958  8253 ts_tablet_manager.cc:1434] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:12:28.340811  8253 tablet_bootstrap.cc:492] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:28.341161  8210 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:28.342192  8029 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:28.343776  8258 tablet_bootstrap.cc:492] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:28.344210  8258 ts_tablet_manager.cc:1403] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.068s	user 0.018s	sys 0.011s
I20260709 13:12:28.346534  8367 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:28.347095  8367 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:28.348173  8367 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.350104  8253 tablet_bootstrap.cc:654] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.352566  8258 raft_consensus.cc:359] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.353231  8258 raft_consensus.cc:385] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.353484  8258 raft_consensus.cc:740] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.353799  8252 tablet_bootstrap.cc:492] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:28.354261  8252 ts_tablet_manager.cc:1403] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.059s	user 0.014s	sys 0.009s
I20260709 13:12:28.354190  8258 consensus_queue.cc:260] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.355285  8264 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.356487  8258 ts_tablet_manager.cc:1434] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:12:28.357358  8258 tablet_bootstrap.cc:492] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:28.357445  8252 raft_consensus.cc:359] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.358791  8252 raft_consensus.cc:385] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.359131  8252 raft_consensus.cc:740] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.359977  8252 consensus_queue.cc:260] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.362092  8252 ts_tablet_manager.cc:1434] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:12:28.364308  8252 tablet_bootstrap.cc:492] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:28.367383  8258 tablet_bootstrap.cc:654] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.368492  8367 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.368883  8367 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.371207  8367 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.372632  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:28.373188  8210 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:28.374518  8252 tablet_bootstrap.cc:654] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.374393  8029 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:28.374653  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:28.375515  8367 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.375936  8367 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:28.376159  8026 leader_election.cc:343] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.376366  8367 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.382941  8253 tablet_bootstrap.cc:492] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:28.383392  8253 ts_tablet_manager.cc:1403] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.000s
I20260709 13:12:28.385113  8253 raft_consensus.cc:359] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.385552  8253 raft_consensus.cc:385] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.386153  8253 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.386863  8253 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.387329  8252 tablet_bootstrap.cc:492] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:28.388033  8252 ts_tablet_manager.cc:1403] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.006s
I20260709 13:12:28.389168  8424 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.389650  8424 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.390990  8252 raft_consensus.cc:359] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.391597  8252 raft_consensus.cc:385] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.391912  8252 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.392508  8252 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.385542  8367 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.393327  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:28.396071  8252 ts_tablet_manager.cc:1434] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.002s
I20260709 13:12:28.396938  8252 tablet_bootstrap.cc:492] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
W20260709 13:12:28.396972  8175 leader_election.cc:343] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.397324  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:28.397837  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:12:28.399199  8175 leader_election.cc:343] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:28.399608  8026 leader_election.cc:343] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.399783  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:28.391638  8424 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:28.400744  8261 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.401340  8253 ts_tablet_manager.cc:1434] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:12:28.402168  8253 tablet_bootstrap.cc:492] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:28.402822  8367 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.404070  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:28.404623  8210 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.404770  7960 catalog_manager.cc:5697] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:28.406673  8353 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.407107  8353 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.409001  8353 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.410645  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:28.410679  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:28.411307  8210 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:28.411412  8207 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
W20260709 13:12:28.412027  8101 leader_election.cc:343] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.412878  8252 tablet_bootstrap.cc:654] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.412796  8104 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:28.413861  8353 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.414220  8353 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.414577  8353 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.414870  8029 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:28.415982  8367 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:28.416412  8367 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:28.417128  8367 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.421101  8264 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.421542  8264 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.423941  8264 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.426003  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:28.427460  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:28.428267  8207 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:28.429925  8258 tablet_bootstrap.cc:492] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:28.429829  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:28.432751  8253 tablet_bootstrap.cc:654] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:28.437582  8026 leader_election.cc:343] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.437795  8367 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.438181  8367 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.438565  8367 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.424182  8353 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.440711  8258 ts_tablet_manager.cc:1403] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.084s	user 0.018s	sys 0.023s
I20260709 13:12:28.441154  8353 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.443044  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:12:28.444445  8101 leader_election.cc:343] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.444964  8367 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.445317  8264 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:28.446812  8252 tablet_bootstrap.cc:492] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:28.447260  8252 ts_tablet_manager.cc:1403] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.050s	user 0.021s	sys 0.008s
I20260709 13:12:28.442041  7961 catalog_manager.cc:5697] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:28.447507  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:12:28.448779  8026 leader_election.cc:343] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.446836  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:28.449611  8207 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.449729  8367 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.451012  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:28.451719  8211 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.453173  8252 raft_consensus.cc:359] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.453799  8252 raft_consensus.cc:385] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.454041  8252 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.454701  8252 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.456609  8252 ts_tablet_manager.cc:1434] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:12:28.461006  8258 raft_consensus.cc:359] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.461668  8258 raft_consensus.cc:385] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.461941  8258 raft_consensus.cc:740] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.461875  8207 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:28.462594  8252 tablet_bootstrap.cc:492] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:28.463306  8104 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:28.464534  8353 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:28.465020  8353 raft_consensus.cc:697] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:28.465873  8353 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.467334  8258 consensus_queue.cc:260] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.468782  8258 ts_tablet_manager.cc:1434] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.000s
I20260709 13:12:28.469609  8258 tablet_bootstrap.cc:492] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:28.471253  8252 tablet_bootstrap.cc:654] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.477954  8367 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:28.487392  8258 tablet_bootstrap.cc:654] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.488127  8211 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:28.493083  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:28.494234  8367 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:28.494773  8367 raft_consensus.cc:697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:28.488510  7958 catalog_manager.cc:5697] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:28.499229  8367 consensus_queue.cc:237] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.506193  8253 tablet_bootstrap.cc:492] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:28.505949  8261 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.508152  8253 ts_tablet_manager.cc:1403] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.106s	user 0.015s	sys 0.020s
I20260709 13:12:28.508131  8261 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.515854  8261 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:28.515738  8253 raft_consensus.cc:359] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.517577  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:28.518175  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:28.520639  8418 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:12:28.522015  8175 leader_election.cc:343] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.518193  8058 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:28.525104  8252 tablet_bootstrap.cc:492] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:28.531402  8252 ts_tablet_manager.cc:1403] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.069s	user 0.022s	sys 0.008s
I20260709 13:12:28.516677  8253 raft_consensus.cc:385] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.532205  8253 raft_consensus.cc:740] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.535110  8253 consensus_queue.cc:260] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.537312  8253 ts_tablet_manager.cc:1434] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.001s
I20260709 13:12:28.539275  8175 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:28.540649  8261 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.540776  8252 raft_consensus.cc:359] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.541751  8252 raft_consensus.cc:385] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.542613  8252 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.544839  8350 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:28.547683  8253 tablet_bootstrap.cc:492] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:28.549302  8252 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.541013  8261 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.553107  8261 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.556564  8252 ts_tablet_manager.cc:1434] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:12:28.558283  8252 tablet_bootstrap.cc:492] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:28.563830  8258 tablet_bootstrap.cc:492] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:28.564247  8258 ts_tablet_manager.cc:1403] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.095s	user 0.006s	sys 0.025s
I20260709 13:12:28.564129  7958 catalog_manager.cc:5697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:28.567374  8261 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.570631  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:28.571031  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:28.571230  8058 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:28.572319  8175 leader_election.cc:343] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.572805  8261 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:28.572872  8258 raft_consensus.cc:359] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.573541  8258 raft_consensus.cc:385] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.573742  8258 raft_consensus.cc:740] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.574271  8258 consensus_queue.cc:260] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.576238  8258 ts_tablet_manager.cc:1434] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.000s
I20260709 13:12:28.577239  8258 tablet_bootstrap.cc:492] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:28.577102  8417 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:28.577966  8252 tablet_bootstrap.cc:654] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.581632  8058 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:28.583458  8258 tablet_bootstrap.cc:654] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.589103  8175 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:28.590126  8261 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:28.590756  8261 raft_consensus.cc:697] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:28.594466  8424 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.594916  8424 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.596676  8058 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.598448  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:12:28.599714  8175 leader_election.cc:343] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.600940  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:28.602054  8175 leader_election.cc:343] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.602494  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:28.603169  8308 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.596781  8424 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:28.605665  8261 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.608278  8253 tablet_bootstrap.cc:654] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.616154  8351 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:28.616675  8258 tablet_bootstrap.cc:492] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:28.617081  8258 ts_tablet_manager.cc:1403] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.040s	user 0.013s	sys 0.018s
I20260709 13:12:28.617580  8253 tablet_bootstrap.cc:492] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:28.617997  8253 ts_tablet_manager.cc:1403] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.071s	user 0.007s	sys 0.003s
I20260709 13:12:28.620014  8258 raft_consensus.cc:359] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.620347  8253 raft_consensus.cc:359] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.620836  8258 raft_consensus.cc:385] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.621006  8253 raft_consensus.cc:385] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.621197  8258 raft_consensus.cc:740] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.621382  8253 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.622054  8258 consensus_queue.cc:260] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.622460  8253 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.627365  8258 ts_tablet_manager.cc:1434] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:12:28.628295  8258 tablet_bootstrap.cc:492] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:28.629383  8253 ts_tablet_manager.cc:1434] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:12:28.630237  8253 tablet_bootstrap.cc:492] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:28.631961  7958 catalog_manager.cc:5697] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 0 to 1, leader changed from <none> to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:28.634829  8258 tablet_bootstrap.cc:654] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.635597  8253 tablet_bootstrap.cc:654] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.641407  8252 tablet_bootstrap.cc:492] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:28.641948  8252 ts_tablet_manager.cc:1403] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.084s	user 0.009s	sys 0.004s
I20260709 13:12:28.644438  8252 raft_consensus.cc:359] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.645123  8252 raft_consensus.cc:385] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.645430  8252 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.646095  8252 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.648156  8252 ts_tablet_manager.cc:1434] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:12:28.648996  8252 tablet_bootstrap.cc:492] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:28.654273  8258 tablet_bootstrap.cc:492] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:28.654702  8258 ts_tablet_manager.cc:1403] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.007s
I20260709 13:12:28.657033  8258 raft_consensus.cc:359] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.657634  8258 raft_consensus.cc:385] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.657888  8258 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.658495  8258 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.660467  8258 ts_tablet_manager.cc:1434] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:12:28.661298  8258 tablet_bootstrap.cc:492] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:28.667496  8258 tablet_bootstrap.cc:654] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.668882  8252 tablet_bootstrap.cc:654] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.686560  8258 tablet_bootstrap.cc:492] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:28.687088  8258 ts_tablet_manager.cc:1403] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.009s
I20260709 13:12:28.689620  8258 raft_consensus.cc:359] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.690230  8258 raft_consensus.cc:385] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.690510  8258 raft_consensus.cc:740] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.691177  8350 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.691087  8258 consensus_queue.cc:260] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.691618  8350 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.693482  8350 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.694396  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:28.693056  8258 ts_tablet_manager.cc:1434] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:12:28.696323  8252 tablet_bootstrap.cc:492] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:28.696798  8252 ts_tablet_manager.cc:1403] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.048s	user 0.020s	sys 0.001s
I20260709 13:12:28.697589  8258 tablet_bootstrap.cc:492] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:28.699745  8252 raft_consensus.cc:359] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.700423  8252 raft_consensus.cc:385] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.700763  8252 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.700817  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:28.701884  8210 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 1: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:12:28.701691  8252 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.703997  8258 tablet_bootstrap.cc:654] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.704053  8252 ts_tablet_manager.cc:1434] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:12:28.705102  8252 tablet_bootstrap.cc:492] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:28.711690  8104 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:28.713097  8350 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.713680  8252 tablet_bootstrap.cc:654] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.714812  8258 tablet_bootstrap.cc:492] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:28.715242  8258 ts_tablet_manager.cc:1403] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.004s
I20260709 13:12:28.718145  8258 raft_consensus.cc:359] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.718570  8292 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.718804  8258 raft_consensus.cc:385] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.719075  8258 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.718991  8292 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.719825  8258 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.720975  8292 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.723807  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:28.724587  8210 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:28.724836  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:28.726644  8101 leader_election.cc:343] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.727394  8104 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:28.728327  8292 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.728782  8292 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.729212  8292 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.729453  8252 tablet_bootstrap.cc:492] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:28.729965  8252 ts_tablet_manager.cc:1403] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.007s
I20260709 13:12:28.732414  8252 raft_consensus.cc:359] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.733144  8252 raft_consensus.cc:385] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.733465  8252 raft_consensus.cc:740] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.734092  8252 consensus_queue.cc:260] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.735365  8258 ts_tablet_manager.cc:1434] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:12:28.736224  8258 tablet_bootstrap.cc:492] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:28.736336  8252 ts_tablet_manager.cc:1434] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:12:28.737224  8252 tablet_bootstrap.cc:492] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:28.739177  8367 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:28.742106  8258 tablet_bootstrap.cc:654] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.743119  8252 tablet_bootstrap.cc:654] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.749001  8350 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.755430  8252 tablet_bootstrap.cc:492] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:28.756098  8252 ts_tablet_manager.cc:1403] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.008s
I20260709 13:12:28.759552  8252 raft_consensus.cc:359] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.760718  8252 raft_consensus.cc:385] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.761138  8252 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.762032  8252 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.764916  8252 ts_tablet_manager.cc:1434] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:12:28.766038  8252 tablet_bootstrap.cc:492] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:28.770251  8273 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.770882  8273 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.773757  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:28.774678  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:28.775486  8210 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:28.777230  8104 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:28.778172  8479 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.778298  8273 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.778591  8479 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.778965  8479 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:28.780478  8101 leader_election.cc:343] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.785580  8479 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.787097  8252 tablet_bootstrap.cc:654] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.788074  8258 tablet_bootstrap.cc:492] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:28.788499  8258 ts_tablet_manager.cc:1403] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.052s	user 0.013s	sys 0.017s
I20260709 13:12:28.791715  8292 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.792309  8253 tablet_bootstrap.cc:492] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:28.792704  8253 ts_tablet_manager.cc:1403] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.163s	user 0.051s	sys 0.096s
I20260709 13:12:28.795019  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:28.795636  8252 tablet_bootstrap.cc:492] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:28.795519  8253 raft_consensus.cc:359] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.796125  8252 ts_tablet_manager.cc:1403] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.002s
W20260709 13:12:28.796327  8101 leader_election.cc:343] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.811874  8252 raft_consensus.cc:359] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.812510  8252 raft_consensus.cc:385] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.812757  8252 raft_consensus.cc:740] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.813407  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:28.813575  8252 consensus_queue.cc:260] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.799796  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:28.814522  8210 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.815404  8253 raft_consensus.cc:385] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.802264  8261 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.815766  8253 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.816112  8261 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.816421  8253 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.817237  8057 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.817561  8308 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.817996  8308 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.803820  8424 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.818869  8424 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.819876  8308 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:28.820763  8424 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:28.822239  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:28.823396  8175 leader_election.cc:343] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.824195  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:28.824590  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:28.825230  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:28.818104  8261 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:28.797580  8258 raft_consensus.cc:359] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.828049  8258 raft_consensus.cc:385] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.828341  8258 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.825210  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:28.829018  8258 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.815202  8367 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:28.830458  8417 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:28.826550  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:28.818503  8253 ts_tablet_manager.cc:1434] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.001s
I20260709 13:12:28.832361  8138 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:28.832847  8258 ts_tablet_manager.cc:1434] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.044s	user 0.005s	sys 0.002s
I20260709 13:12:28.833740  8258 tablet_bootstrap.cc:492] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:28.837066  8252 ts_tablet_manager.cc:1434] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.027s	user 0.020s	sys 0.002s
I20260709 13:12:28.837930  8252 tablet_bootstrap.cc:492] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:28.833748  8253 tablet_bootstrap.cc:492] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
W20260709 13:12:28.840581  8175 leader_election.cc:343] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:28.841317  8175 leader_election.cc:343] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.841161  8367 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:12:28.841965  8175 leader_election.cc:343] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.828332  8350 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.842378  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:28.846942  8252 tablet_bootstrap.cc:654] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.800391  8479 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.851163  8253 tablet_bootstrap.cc:654] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:28.868783  8258 tablet_bootstrap.cc:654] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:28.849982  8175 leader_election.cc:343] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.851548  8308 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.872728  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:28.846000  8350 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.878872  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:28.881152  8424 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.881496  8424 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.881810  8424 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.878126  8308 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.898092  8210 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:28.898481  8424 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.899398  8104 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:28.900076  8057 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:28.905568  8351 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:28.906077  8351 raft_consensus.cc:697] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:28.907099  8351 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.910295  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:28.910873  8210 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.911154  8424 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:28.919082  8292 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.928834  8210 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:28.930300  8417 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.934496  8417 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.935084  8104 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:28.936008  8353 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:28.935949  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:28.936403  8417 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.936540  8353 raft_consensus.cc:697] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:28.937347  8353 consensus_queue.cc:237] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.939713  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:28.940385  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:28.940419  8135 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:28.941725  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:28.936625  8137 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.942375  8207 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:28.942461  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:28.943557  8417 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.943903  8417 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.944212  8417 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:28.947618  8175 leader_election.cc:343] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.946666  8479 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:28.950507  8253 tablet_bootstrap.cc:492] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:28.950939  8253 ts_tablet_manager.cc:1403] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.117s	user 0.019s	sys 0.009s
I20260709 13:12:28.956983  8252 tablet_bootstrap.cc:492] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:28.957049  8487 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.957502  8252 ts_tablet_manager.cc:1403] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.120s	user 0.016s	sys 0.001s
I20260709 13:12:28.957755  8487 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.958906  8479 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.958969  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:28.959443  8479 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.961915  8479 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.961966  8252 raft_consensus.cc:359] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.962471  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:28.963001  8350 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:28.963881  8252 raft_consensus.cc:385] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.964172  8252 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.963384  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:28.964885  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:28.964813  8253 raft_consensus.cc:359] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.965480  8253 raft_consensus.cc:385] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:28.965502  8208 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:28.965891  8253 raft_consensus.cc:740] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:28.965150  8210 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 1: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:12:28.970089  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:28.971845  8252 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:28.972357  8175 leader_election.cc:343] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.966395  8417 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.967178  8104 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:28.967512  8487 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:28.967303  8253 consensus_queue.cc:260] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.968788  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:28.974609  8175 leader_election.cc:343] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:28.974798  8137 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:28.975044  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:28.974167  8252 ts_tablet_manager.cc:1434] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:12:28.976128  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:28.976424  8252 tablet_bootstrap.cc:492] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:28.978041  8253 ts_tablet_manager.cc:1434] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.001s
I20260709 13:12:28.978421  8264 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:28.980111  8057 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 1: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:12:28.980597  8424 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:28.980319  8264 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.981195  8424 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:28.981662  8424 raft_consensus.cc:697] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:28.981115  7958 catalog_manager.cc:5697] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:28.982431  8424 consensus_queue.cc:237] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:28.990512  8350 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:28.990945  8350 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:28.991343  8350 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:28.999672  8351 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.000308  8351 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.003430  8104 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:29.006783  8351 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.008644  8367 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:29.009994  8487 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:29.010774  8253 tablet_bootstrap.cc:492] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:29.015632  8258 tablet_bootstrap.cc:492] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:29.016193  8258 ts_tablet_manager.cc:1403] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.183s	user 0.011s	sys 0.030s
I20260709 13:12:29.029306  8350 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.031186  8264 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.023227  8479 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.033255  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:29.033851  8211 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:29.037118  8104 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:29.037845  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:29.038640  8057 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.039535  8273 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:29.039880  8273 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:29.039831  8258 raft_consensus.cc:359] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.040385  8273 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.040654  8258 raft_consensus.cc:385] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:29.040948  8258 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.043934  8258 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.051015  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:29.051167  8261 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.051657  8261 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.057673  8211 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 1: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:12:29.060600  8479 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:29.061110  8417 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.078871  8026 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:29.079450  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:29.067292  8292 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.100626  8258 ts_tablet_manager.cc:1434] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.084s	user 0.007s	sys 0.001s
I20260709 13:12:29.089080  8487 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.090389  8211 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.092149  8292 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.106611  8252 tablet_bootstrap.cc:654] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:29.100469  8494 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.112707  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:29.102695  8261 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:29.107980  8264 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.107306  8350 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.130896  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:29.134279  8253 tablet_bootstrap.cc:654] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:29.121260  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:29.113276  8273 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.107928  8487 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.151527  8292 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.153666  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:29.154168  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:29.154289  8210 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:29.154922  8137 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 1: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:12:29.155736  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:29.156466  8058 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 1: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:12:29.158764  8258 tablet_bootstrap.cc:492] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
W20260709 13:12:29.137391  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 58670.4us
W20260709 13:12:29.143410  8175 leader_election.cc:343] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:29.168367  8101 leader_election.cc:343] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.125576  8494 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.176750  8417 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:29.177937  8211 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
W20260709 13:12:29.218416  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 47955.7us
I20260709 13:12:29.191720  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:29.220531  8175 leader_election.cc:343] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.220978  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:29.190941  8261 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.178237  8264 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:29.188030  8273 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.182866  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:29.179672  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:29.189196  7960 catalog_manager.cc:5697] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:29.224160  8481 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:29.170377  8292 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.059s
I20260709 13:12:29.227095  8479 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.250764  8479 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.252167  8252 tablet_bootstrap.cc:492] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:29.252890  8252 ts_tablet_manager.cc:1403] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.277s	user 0.013s	sys 0.006s
I20260709 13:12:29.231806  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:12:29.230890  8175 leader_election.cc:343] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.184109  8487 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:12:29.259467  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 93318.9us
I20260709 13:12:29.256177  8208 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 1: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:12:29.262120  8104 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:29.232158  8261 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.251890  8029 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:29.273473  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:29.277230  8495 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:29.272583  8487 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.287248  8258 tablet_bootstrap.cc:654] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:29.290114  8253 tablet_bootstrap.cc:492] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:29.290791  8253 ts_tablet_manager.cc:1403] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.287s	user 0.035s	sys 0.031s
W20260709 13:12:29.292410  8101 leader_election.cc:343] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.293067  8502 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:12:29.293315  8101 leader_election.cc:343] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.293571  8502 raft_consensus.cc:697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:29.282146  8495 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:29.285302  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 53334.7us
I20260709 13:12:29.294453  8502 consensus_queue.cc:237] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.307647  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:29.308223  8210 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.310586  8252 raft_consensus.cc:359] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.311079  8252 raft_consensus.cc:385] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:29.311306  8252 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.285840  8487 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.311877  8252 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.299552  8495 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.319119  8210 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:29.310282  8479 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:12:29.316669  8175 leader_election.cc:343] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.329164  8252 ts_tablet_manager.cc:1434] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.076s	user 0.005s	sys 0.001s
I20260709 13:12:29.331092  8350 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.333281  8350 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.335271  8350 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.337980  8252 tablet_bootstrap.cc:492] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:29.369827  8104 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:29.335745  8175 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:29.389797  8253 raft_consensus.cc:359] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.392894  8253 raft_consensus.cc:385] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:29.393268  8253 raft_consensus.cc:740] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.376816  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:29.402485  8253 consensus_queue.cc:260] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.389007  8261 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:29.395813  8308 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:12:29.407127  8496 log.cc:927] Time spent T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.247s	user 0.004s	sys 0.001s
I20260709 13:12:29.408905  8253 ts_tablet_manager.cc:1434] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.118s	user 0.007s	sys 0.000s
I20260709 13:12:29.410182  8253 tablet_bootstrap.cc:492] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:29.411845  8273 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:12:29.413604  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 43859.6us
I20260709 13:12:29.414413  8104 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:29.415047  8273 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:29.415706  8273 raft_consensus.cc:697] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:29.416878  8494 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.420687  8273 consensus_queue.cc:237] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:29.398619  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 83671.2us
I20260709 13:12:29.432384  8252 tablet_bootstrap.cc:654] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:29.441326  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:29.443153  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:29.433090  8487 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:29.443763  8208 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:29.448714  8104 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:29.432412  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:29.451164  8104 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:29.453567  8506 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:29.439869  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:29.457613  8505 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:29.451901  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:29.458010  8505 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:29.459789  8101 leader_election.cc:343] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.481196  8258 tablet_bootstrap.cc:492] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:29.481603  8258 ts_tablet_manager.cc:1403] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.323s	user 0.029s	sys 0.016s
I20260709 13:12:29.465274  8505 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.476377  8495 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.459200  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:29.508150  8258 raft_consensus.cc:359] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.508970  8253 tablet_bootstrap.cc:654] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:29.509162  8258 raft_consensus.cc:385] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:29.509577  8258 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.510601  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:12:29.510742  8175 leader_election.cc:343] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.511551  8258 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.514462  8350 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:29.515195  8258 ts_tablet_manager.cc:1434] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:12:29.516144  8258 tablet_bootstrap.cc:492] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:29.517006  8133 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:29.519057  8499 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.262s	user 0.004s	sys 0.002s
I20260709 13:12:29.522925  8258 tablet_bootstrap.cc:654] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:29.516610  8505 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.523964  8132 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 1: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:12:29.524523  8495 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.527998  8252 tablet_bootstrap.cc:492] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:29.528441  8252 ts_tablet_manager.cc:1403] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.191s	user 0.014s	sys 0.010s
I20260709 13:12:29.530126  8258 tablet_bootstrap.cc:492] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:29.530568  8258 ts_tablet_manager.cc:1403] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.002s
I20260709 13:12:29.530978  8252 raft_consensus.cc:359] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.530634  8505 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.531665  8252 raft_consensus.cc:385] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:29.532030  8252 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.532775  8252 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.534003  8258 raft_consensus.cc:359] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.534730  8258 raft_consensus.cc:385] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:29.534996  8258 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.535053  8252 ts_tablet_manager.cc:1434] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:12:29.535840  8253 tablet_bootstrap.cc:492] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:29.535640  8258 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.535864  8252 tablet_bootstrap.cc:492] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:29.536402  8253 ts_tablet_manager.cc:1403] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.126s	user 0.011s	sys 0.001s
I20260709 13:12:29.537624  8258 ts_tablet_manager.cc:1434] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:12:29.538527  8258 tablet_bootstrap.cc:492] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:29.538779  8253 raft_consensus.cc:359] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.539414  8253 raft_consensus.cc:385] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:29.539665  8253 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.540625  8252 tablet_bootstrap.cc:654] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:29.540238  8253 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.542889  8253 ts_tablet_manager.cc:1434] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:12:29.543889  8253 tablet_bootstrap.cc:492] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:29.504676  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:29.585939  8029 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:12:29.591778  8175 leader_election.cc:343] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.592453  8258 tablet_bootstrap.cc:654] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:29.595954  8308 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.612182  8261 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:29.618332  8253 tablet_bootstrap.cc:654] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:29.620890  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:29.621837  8207 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.623639  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:29.624488  8210 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.624902  7958 catalog_manager.cc:5697] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 0 to 1, leader changed from <none> to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:29.598097  8308 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.636955  8308 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:29.596119  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:29.639148  8264 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:29.646734  8507 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:29.656958  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:29.674150  8481 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.674903  8481 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.677196  8481 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:29.678917  8481 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:29.680182  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:29.680213  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:12:29.680982  8175 leader_election.cc:343] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.681453  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:29.682332  8424 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:29.683166  8502 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.683697  8502 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.685918  8502 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.693459  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:12:29.698985  8175 leader_election.cc:343] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:29.699687  8175 leader_election.cc:343] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.704502  8210 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:29.705930  8505 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:29.710065  8207 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:29.710700  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:29.715826  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:12:29.717847  8026 leader_election.cc:343] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.719246  8487 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:29.720373  8252 tablet_bootstrap.cc:492] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:29.722077  8252 ts_tablet_manager.cc:1403] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.186s	user 0.009s	sys 0.004s
I20260709 13:12:29.728317  8367 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.728559  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:29.729130  8367 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:29.731609  8175 consensus_peers.cc:597] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:29.732223  8029 leader_election.cc:343] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.732829  8029 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:29.733709  8264 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:29.733784  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:29.735172  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:29.736299  8367 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.736394  8505 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.737057  8505 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.739446  8505 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.751917  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:29.752677  8253 tablet_bootstrap.cc:492] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
W20260709 13:12:29.754184  8175 leader_election.cc:343] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.754922  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:29.767992  8253 ts_tablet_manager.cc:1403] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.224s	user 0.020s	sys 0.016s
W20260709 13:12:29.768322  8175 leader_election.cc:343] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.768869  8175 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:29.771381  8424 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:29.792475  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:29.793354  8208 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:29.778916  8308 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:29.796504  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:29.797459  8208 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 1: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:12:29.802888  8104 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:12:29.803706  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 56769.4us
I20260709 13:12:29.815550  8505 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:29.820000  8505 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:29.821249  8505 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.823033  7958 catalog_manager.cc:5697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:29.832309  8350 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:29.832780  8350 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.834873  8350 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.840102  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:29.841403  8029 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:29.842289  8367 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:29.849080  8101 leader_election.cc:343] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:29.849645  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 92082.8us
I20260709 13:12:29.819135  8252 raft_consensus.cc:359] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.862916  8252 raft_consensus.cc:385] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:29.863293  8252 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.864058  8252 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.866899  8252 ts_tablet_manager.cc:1434] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.144s	user 0.008s	sys 0.000s
I20260709 13:12:29.868316  7960 catalog_manager.cc:5697] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:29.873617  8252 tablet_bootstrap.cc:492] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:29.825201  8253 raft_consensus.cc:359] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.883365  8253 raft_consensus.cc:385] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:29.883744  8253 raft_consensus.cc:740] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.861179  8104 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:29.894044  8253 consensus_queue.cc:260] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.896409  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:29.897128  8212 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:29.900329  8252 tablet_bootstrap.cc:654] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:29.899623  8009 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:29.903507  8350 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:29.904075  8350 raft_consensus.cc:697] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:29.904198  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:29.905339  8292 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:29.905761  8292 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:29.906122  8292 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.908118  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:29.908741  8505 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:29.910164  8367 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:12:29.911358  8101 leader_election.cc:343] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.917109  8104 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:29.892131  8502 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:12:29.919162  8505 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:29.919526  8505 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:29.919909  8505 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.923385  8350 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:29.923882  8101 leader_election.cc:343] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:29.924481  8253 ts_tablet_manager.cc:1434] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.156s	user 0.008s	sys 0.000s
I20260709 13:12:29.965044  8505 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.967142  8505 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:29.968487  8258 tablet_bootstrap.cc:492] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:29.969014  8258 ts_tablet_manager.cc:1403] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.431s	user 0.034s	sys 0.027s
W20260709 13:12:29.971127  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44059us
I20260709 13:12:29.978093  8253 tablet_bootstrap.cc:492] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:29.978729  8258 raft_consensus.cc:359] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.995535  8258 raft_consensus.cc:385] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:29.995836  8258 raft_consensus.cc:740] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.003041  8258 consensus_queue.cc:260] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:29.992898  8292 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.011950  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:30.012602  8207 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.997699  8264 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:12:30.016827  8258 ts_tablet_manager.cc:1434] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.048s	user 0.001s	sys 0.007s
I20260709 13:12:30.021610  8252 tablet_bootstrap.cc:492] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:30.022049  8252 ts_tablet_manager.cc:1403] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.149s	user 0.009s	sys 0.018s
I20260709 13:12:30.025696  8253 tablet_bootstrap.cc:654] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.028978  8258 tablet_bootstrap.cc:492] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:30.029036  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:12:30.033602  8101 leader_election.cc:343] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.048923  8508 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:12:30.049114  8101 leader_election.cc:343] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.055684  8252 raft_consensus.cc:359] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.056393  8252 raft_consensus.cc:385] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.056672  8252 raft_consensus.cc:740] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.057363  8252 consensus_queue.cc:260] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.059298  8252 ts_tablet_manager.cc:1434] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:12:30.058524  7958 catalog_manager.cc:5697] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.060278  8207 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:30.061827  8252 tablet_bootstrap.cc:492] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:30.062391  7958 catalog_manager.cc:5697] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.071200  8253 tablet_bootstrap.cc:492] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:30.071628  8253 ts_tablet_manager.cc:1403] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.100s	user 0.020s	sys 0.008s
I20260709 13:12:30.072572  8292 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.075707  8252 tablet_bootstrap.cc:654] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.078689  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:30.081004  8104 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:30.081707  8505 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:12:30.083381  8101 leader_election.cc:343] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.086587  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:30.087155  8211 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.080884  8253 raft_consensus.cc:359] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.088649  8253 raft_consensus.cc:385] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.089573  8292 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.092728  8258 tablet_bootstrap.cc:654] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.091390  8292 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:30.097820  8292 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.088936  8253 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.118865  8253 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.120708  8253 ts_tablet_manager.cc:1434] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.000s
I20260709 13:12:30.121203  8500 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.121739  8253 tablet_bootstrap.cc:492] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:30.121673  8500 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.102386  8211 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:30.126605  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:30.127259  8350 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.127678  8350 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:30.128520  8350 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.132907  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:30.131753  8500 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.135372  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.137392  8026 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:30.138371  8417 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:30.146029  7958 catalog_manager.cc:5697] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.147095  8253 tablet_bootstrap.cc:654] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.148569  8252 tablet_bootstrap.cc:492] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:30.148962  8252 ts_tablet_manager.cc:1403] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.087s	user 0.012s	sys 0.010s
I20260709 13:12:30.151146  8258 tablet_bootstrap.cc:492] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:30.151535  8258 ts_tablet_manager.cc:1403] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.123s	user 0.014s	sys 0.004s
I20260709 13:12:30.153257  8252 raft_consensus.cc:359] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.153760  8252 raft_consensus.cc:385] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.154174  8252 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.153864  8258 raft_consensus.cc:359] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.154623  8258 raft_consensus.cc:385] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.154907  8258 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.155041  8252 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.155563  8258 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.157811  8258 ts_tablet_manager.cc:1434] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:12:30.158751  8258 tablet_bootstrap.cc:492] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:30.158576  8252 ts_tablet_manager.cc:1434] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:12:30.160817  8252 tablet_bootstrap.cc:492] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:30.166205  8258 tablet_bootstrap.cc:654] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.168330  8252 tablet_bootstrap.cc:654] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.178064  8062 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.182550  8258 tablet_bootstrap.cc:492] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:30.183096  8258 ts_tablet_manager.cc:1403] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.006s
I20260709 13:12:30.185822  7958 catalog_manager.cc:5697] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.199724  8253 tablet_bootstrap.cc:492] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:30.200207  8253 ts_tablet_manager.cc:1403] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.079s	user 0.017s	sys 0.033s
I20260709 13:12:30.202702  8253 raft_consensus.cc:359] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.203351  8253 raft_consensus.cc:385] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.203601  8253 raft_consensus.cc:740] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.204211  8253 consensus_queue.cc:260] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.210140  8253 ts_tablet_manager.cc:1434] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:12:30.211099  8258 raft_consensus.cc:359] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.211818  8258 raft_consensus.cc:385] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.212230  8253 tablet_bootstrap.cc:492] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:30.212193  8258 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.213351  8258 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.216291  8258 ts_tablet_manager.cc:1434] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 13:12:30.217468  8258 tablet_bootstrap.cc:492] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:30.220666  8253 tablet_bootstrap.cc:654] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.232357  8252 tablet_bootstrap.cc:492] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:30.233513  8258 tablet_bootstrap.cc:654] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.236357  8252 ts_tablet_manager.cc:1403] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.072s	user 0.010s	sys 0.005s
I20260709 13:12:30.239596  8252 raft_consensus.cc:359] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.240211  8258 tablet_bootstrap.cc:492] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:30.240206  8252 raft_consensus.cc:385] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.240705  8252 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.240869  8258 ts_tablet_manager.cc:1403] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.005s
I20260709 13:12:30.241472  8252 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.243961  8258 raft_consensus.cc:359] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.244725  8258 raft_consensus.cc:385] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.245045  8258 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.245764  8258 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.247723  8252 ts_tablet_manager.cc:1434] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:12:30.248728  8252 tablet_bootstrap.cc:492] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:30.251156  8258 ts_tablet_manager.cc:1434] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:12:30.252702  8253 tablet_bootstrap.cc:492] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:30.253166  8253 ts_tablet_manager.cc:1403] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.042s	user 0.016s	sys 0.000s
I20260709 13:12:30.255455  8252 tablet_bootstrap.cc:654] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.255589  8258 tablet_bootstrap.cc:492] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:30.255638  8253 raft_consensus.cc:359] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.256516  8253 raft_consensus.cc:385] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.256810  8253 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.257436  8253 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.261762  8258 tablet_bootstrap.cc:654] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.265131  8253 ts_tablet_manager.cc:1434] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:12:30.266687  8253 tablet_bootstrap.cc:492] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:30.283908  8253 tablet_bootstrap.cc:654] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.285300  8258 tablet_bootstrap.cc:492] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:30.285701  8258 ts_tablet_manager.cc:1403] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.017s
I20260709 13:12:30.288053  8258 raft_consensus.cc:359] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.288594  8252 tablet_bootstrap.cc:492] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:30.288652  8258 raft_consensus.cc:385] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.288892  8253 tablet_bootstrap.cc:492] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:30.289088  8252 ts_tablet_manager.cc:1403] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.016s
I20260709 13:12:30.289050  8258 raft_consensus.cc:740] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.289273  8253 ts_tablet_manager.cc:1403] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.002s
I20260709 13:12:30.289930  8258 consensus_queue.cc:260] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.292193  8252 raft_consensus.cc:359] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.292860  8258 ts_tablet_manager.cc:1434] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:30.292838  8252 raft_consensus.cc:385] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.293310  8252 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.294626  8062 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.293985  8252 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.294785  8253 raft_consensus.cc:359] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.296149  8253 raft_consensus.cc:385] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.296397  8253 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.297050  8253 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.299664  8253 ts_tablet_manager.cc:1434] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:12:30.302843  8253 tablet_bootstrap.cc:492] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:30.308555  8253 tablet_bootstrap.cc:654] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.317022  8252 ts_tablet_manager.cc:1434] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:12:30.320914  8253 tablet_bootstrap.cc:492] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:30.320885  8292 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:30.321465  8253 ts_tablet_manager.cc:1403] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.007s
I20260709 13:12:30.323659  8253 raft_consensus.cc:359] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.324126  8253 raft_consensus.cc:385] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.324339  8253 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.324986  8253 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.326848  8258 tablet_bootstrap.cc:492] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:30.327169  8252 tablet_bootstrap.cc:492] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:30.331265  8253 ts_tablet_manager.cc:1434] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:12:30.332237  8253 tablet_bootstrap.cc:492] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:30.333087  8258 tablet_bootstrap.cc:654] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.337520  8252 tablet_bootstrap.cc:654] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.340431  8481 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.340884  8481 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.341490  8253 tablet_bootstrap.cc:654] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.342911  8481 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:30.345073  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.345558  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:30.347162  8175 leader_election.cc:343] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:30.347921  8175 leader_election.cc:343] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.348388  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:30.349754  8258 tablet_bootstrap.cc:492] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:30.350147  8252 tablet_bootstrap.cc:492] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:30.350173  8258 ts_tablet_manager.cc:1403] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.004s
I20260709 13:12:30.350693  8481 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:30.350704  8252 ts_tablet_manager.cc:1403] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.006s
I20260709 13:12:30.352602  8253 tablet_bootstrap.cc:492] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:30.353015  8253 ts_tablet_manager.cc:1403] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.005s
I20260709 13:12:30.353098  8258 raft_consensus.cc:359] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.353900  8258 raft_consensus.cc:385] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.354190  8258 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.354754  8252 raft_consensus.cc:359] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.354796  8258 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.355461  8252 raft_consensus.cc:385] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.355742  8252 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.355878  8253 raft_consensus.cc:359] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.356551  8253 raft_consensus.cc:385] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.356390  8252 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.356834  8253 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.357553  8253 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.358397  8481 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.359004  8252 ts_tablet_manager.cc:1434] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:30.358817  8481 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.359082  8292 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:30.359846  8253 ts_tablet_manager.cc:1434] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:12:30.360009  8252 tablet_bootstrap.cc:492] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:30.360756  8253 tablet_bootstrap.cc:492] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:30.360816  8481 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:30.362502  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.364602  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:30.366114  8175 leader_election.cc:343] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:30.366777  8175 leader_election.cc:343] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.367285  8258 ts_tablet_manager.cc:1434] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:12:30.367178  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:30.367575  8253 tablet_bootstrap.cc:654] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.368007  8481 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:30.370951  8258 tablet_bootstrap.cc:492] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:30.373039  8252 tablet_bootstrap.cc:654] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.380003  8253 tablet_bootstrap.cc:492] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:30.380426  8253 ts_tablet_manager.cc:1403] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.001s
I20260709 13:12:30.383085  8504 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.383409  8253 raft_consensus.cc:359] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.384217  8253 raft_consensus.cc:385] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.384471  8253 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.383563  8504 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.385409  8253 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.387082  8504 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.387390  8253 ts_tablet_manager.cc:1434] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:12:30.388302  8253 tablet_bootstrap.cc:492] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:30.389220  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:30.389801  8062 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:30.390810  8101 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:30.391767  8495 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.392110  8495 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.392437  8495 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.393148  8481 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.393949  8253 tablet_bootstrap.cc:654] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.393729  8481 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.396332  8481 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:30.398923  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:30.399722  8057 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:30.400959  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.401721  8130 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:30.403543  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:30.405040  8253 tablet_bootstrap.cc:492] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:30.405447  8253 ts_tablet_manager.cc:1403] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:12:30.405218  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:30.406035  8211 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:30.407929  8253 raft_consensus.cc:359] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.408641  8253 raft_consensus.cc:385] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.408910  8253 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.409687  8253 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.410274  8481 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.411172  8481 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.410928  8495 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.411624  8481 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.414112  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:30.414660  8057 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.418241  8504 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.418300  8481 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.418996  8504 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.420440  8481 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:30.411176  8253 ts_tablet_manager.cc:1434] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:12:30.421823  8253 tablet_bootstrap.cc:492] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:30.422044  8057 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:30.422221  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:30.423012  8130 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.423205  8101 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:30.424106  8505 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.424607  8505 raft_consensus.cc:697] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:30.426077  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:30.425961  8505 consensus_queue.cc:237] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.426857  8062 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.435501  8504 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.436353  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:30.436950  8061 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:30.437410  8495 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.439363  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:30.439772  8062 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:30.439949  8212 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.440970  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:30.441588  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:30.444176  8481 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.450803  8481 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:30.451826  8130 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:30.451576  8481 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.444295  8495 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.454973  8495 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.455292  8495 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.458410  8258 tablet_bootstrap.cc:654] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:30.460542  8101 consensus_peers.cc:597] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:30.461299  8252 tablet_bootstrap.cc:492] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:30.461807  8252 ts_tablet_manager.cc:1403] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.102s	user 0.024s	sys 0.050s
I20260709 13:12:30.463002  8350 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.463672  8500 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.463475  8350 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.464085  8500 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.464417  8252 raft_consensus.cc:359] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.465058  8252 raft_consensus.cc:385] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.465317  8252 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.465881  8350 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.461284  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:30.468364  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.469080  8212 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:30.466148  8500 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.469400  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:30.466027  8252 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.481317  8252 ts_tablet_manager.cc:1434] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.019s	user 0.001s	sys 0.005s
I20260709 13:12:30.480336  8211 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:30.484225  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:30.486680  8252 tablet_bootstrap.cc:492] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:30.480707  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:30.492460  8208 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:30.496198  8495 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:30.496913  8104 leader_election.cc:343] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.500221  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:30.501978  8207 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.504340  8253 tablet_bootstrap.cc:654] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:30.501130  8101 leader_election.cc:343] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.506848  8101 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:30.512328  7959 catalog_manager.cc:5697] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.513545  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:30.521154  8505 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:30.521232  8264 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.521881  8264 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.522296  8264 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:30.524637  8026 leader_election.cc:343] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.526476  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:30.527021  8062 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.514004  7961 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 0 to 1, leader changed from <none> to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:30.505667  8495 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.518756  8532 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.532797  8532 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.534775  8532 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:30.534175  8252 tablet_bootstrap.cc:654] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.537897  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:30.539431  8175 leader_election.cc:343] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.539860  8258 tablet_bootstrap.cc:492] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:30.540174  8258 ts_tablet_manager.cc:1403] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.169s	user 0.045s	sys 0.027s
I20260709 13:12:30.542610  8207 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:30.544451  8062 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:30.544858  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.545688  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:30.546432  8292 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.547079  8292 raft_consensus.cc:697] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:12:30.548334  8175 leader_election.cc:343] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.548477  8509 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.548748  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:30.548429  8264 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.549000  8509 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.549428  8258 raft_consensus.cc:359] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.550060  8258 raft_consensus.cc:385] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.551141  8509 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.551734  8532 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:30.547875  8292 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.552347  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:30.552845  8264 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.552990  8211 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.550323  8258 raft_consensus.cc:740] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.555545  8258 consensus_queue.cc:260] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.557483  8258 ts_tablet_manager.cc:1434] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.017s	user 0.001s	sys 0.003s
I20260709 13:12:30.558439  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:12:30.559655  8026 leader_election.cc:343] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.559247  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:30.561301  8207 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:30.560122  8211 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:30.563308  8253 tablet_bootstrap.cc:492] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:30.563616  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:30.564848  8253 ts_tablet_manager.cc:1403] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.143s	user 0.012s	sys 0.002s
I20260709 13:12:30.564047  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:30.566390  8505 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.566745  8505 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.567059  8505 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.570429  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:30.571238  8062 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:30.571534  8264 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.574051  8264 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:30.574875  8264 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.558457  8258 tablet_bootstrap.cc:492] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:30.580775  8253 raft_consensus.cc:359] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.581714  8253 raft_consensus.cc:385] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.582026  8253 raft_consensus.cc:740] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.582581  8252 tablet_bootstrap.cc:492] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:30.582995  8252 ts_tablet_manager.cc:1403] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.097s	user 0.009s	sys 0.019s
I20260709 13:12:30.583863  8253 consensus_queue.cc:260] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.584946  7958 catalog_manager.cc:5697] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.585738  8252 raft_consensus.cc:359] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.586390  8258 tablet_bootstrap.cc:654] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.586422  8252 raft_consensus.cc:385] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.587096  8252 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.587782  8252 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.586522  8253 ts_tablet_manager.cc:1434] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:12:30.591782  8253 tablet_bootstrap.cc:492] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:30.593694  8481 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.594416  8481 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.596277  8252 ts_tablet_manager.cc:1434] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:12:30.596562  8532 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.597002  8532 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.597224  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:30.598146  8481 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:12:30.598701  8175 leader_election.cc:343] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.598881  8532 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:30.600822  8481 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.602560  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:30.604218  8062 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:30.604127  8481 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.602433  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.605441  8132 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:30.597226  8252 tablet_bootstrap.cc:492] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:30.605975  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:30.606364  8481 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:30.608069  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:30.602066  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.617954  8252 tablet_bootstrap.cc:654] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.618150  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.620167  8253 tablet_bootstrap.cc:654] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:30.620223  8175 leader_election.cc:343] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.621253  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:30.622020  8261 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:30.623019  8532 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.623595  8532 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.624015  8532 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.626017  8505 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.637967  8495 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:30.653692  8258 tablet_bootstrap.cc:492] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:30.653460  8350 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:30.654196  8258 ts_tablet_manager.cc:1403] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.096s	user 0.023s	sys 0.036s
I20260709 13:12:30.657565  8258 raft_consensus.cc:359] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.658233  8258 raft_consensus.cc:385] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.658504  8258 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.659085  8258 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.660485  8258 ts_tablet_manager.cc:1434] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:12:30.661119  8252 tablet_bootstrap.cc:492] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:30.661643  8252 ts_tablet_manager.cc:1403] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.065s	user 0.010s	sys 0.001s
I20260709 13:12:30.664527  8532 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.664721  8252 raft_consensus.cc:359] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.665586  8252 raft_consensus.cc:385] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.665901  8252 raft_consensus.cc:740] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.666077  8532 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:30.669853  8500 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.671635  8253 tablet_bootstrap.cc:492] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:30.672061  8253 ts_tablet_manager.cc:1403] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.080s	user 0.009s	sys 0.005s
I20260709 13:12:30.670467  8500 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.675192  8253 raft_consensus.cc:359] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.675843  8253 raft_consensus.cc:385] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.676108  8253 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.676887  8253 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.677834  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.679126  8253 ts_tablet_manager.cc:1434] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:12:30.679997  8253 tablet_bootstrap.cc:492] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:30.680925  8258 tablet_bootstrap.cc:492] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:30.683359  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:30.679960  8500 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.682310  7959 catalog_manager.cc:5697] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.687938  8258 tablet_bootstrap.cc:654] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:30.697804  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 77623.8us
W20260709 13:12:30.698433  8175 leader_election.cc:343] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:30.698968  8175 leader_election.cc:343] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.699304  8175 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:30.694093  8029 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:30.701066  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:30.701634  8057 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.701099  8500 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:30.701272  8505 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.708858  8062 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.709520  8057 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:30.710875  8101 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:30.711617  8509 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.712044  8509 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:30.712823  8509 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.720948  8253 tablet_bootstrap.cc:654] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.724797  8500 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.725237  8500 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.727089  8500 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.727536  8495 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.727751  8505 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:30.728003  8495 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.733286  8495 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.729492  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.731477  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:30.734889  8211 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.738710  8253 tablet_bootstrap.cc:492] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:30.739288  8253 ts_tablet_manager.cc:1403] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.059s	user 0.007s	sys 0.007s
I20260709 13:12:30.739208  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:30.739902  8059 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:30.742137  8532 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:30.743033  8062 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.745790  8253 raft_consensus.cc:359] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.746677  8253 raft_consensus.cc:385] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.746897  8101 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:30.747095  8253 raft_consensus.cc:740] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.748121  8350 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.748131  8532 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.748597  8350 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.748052  8253 consensus_queue.cc:260] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.748929  8350 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.749346  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:30.750320  8253 ts_tablet_manager.cc:1434] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:12:30.748930  8532 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.751786  8253 tablet_bootstrap.cc:492] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:30.752578  8532 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:30.755256  8029 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:30.755980  8500 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:30.761755  8253 tablet_bootstrap.cc:654] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.766264  8252 consensus_queue.cc:260] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.769619  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:30.770325  8208 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:30.783329  8252 ts_tablet_manager.cc:1434] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.121s	user 0.008s	sys 0.000s
I20260709 13:12:30.785792  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:30.787688  8253 tablet_bootstrap.cc:492] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:30.790428  8253 ts_tablet_manager.cc:1403] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.009s
I20260709 13:12:30.790585  8130 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.793147  8273 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:30.798856  8130 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:30.800670  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:30.801270  8059 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.804950  8264 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.805379  8264 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.806965  8258 tablet_bootstrap.cc:492] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:30.807165  8264 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.807511  8258 ts_tablet_manager.cc:1403] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.127s	user 0.039s	sys 0.051s
I20260709 13:12:30.808178  8059 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:30.809082  8252 tablet_bootstrap.cc:492] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:30.809129  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:30.809824  8212 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:30.810824  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:30.811054  8350 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.811825  8264 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.812224  8264 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.812580  8264 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.812919  8350 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.804533  8253 raft_consensus.cc:359] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.815049  8253 raft_consensus.cc:385] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.815395  8253 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.817444  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.817875  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:30.818821  8264 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.819178  8253 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.821134  8264 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.822464  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:30.822813  8253 ts_tablet_manager.cc:1434] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 13:12:30.823022  8208 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.823858  8253 tablet_bootstrap.cc:492] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:30.799089  8211 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:30.829231  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:30.829802  8253 tablet_bootstrap.cc:654] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.829921  8208 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
W20260709 13:12:30.830408  8026 leader_election.cc:343] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.831269  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:30.831806  8059 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.832693  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:30.833213  8212 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.835302  8258 raft_consensus.cc:359] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.835932  8258 raft_consensus.cc:385] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.836189  8258 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.836803  8258 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.838838  8258 ts_tablet_manager.cc:1434] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.002s
I20260709 13:12:30.839833  8258 tablet_bootstrap.cc:492] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:30.839720  8212 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:30.843802  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.844980  8104 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:30.848238  8500 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.848917  8500 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.833222  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:12:30.852092  8175 leader_election.cc:343] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.853209  8261 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.853636  8261 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:30.854399  8261 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.857362  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:30.858814  8264 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.859346  8264 raft_consensus.cc:697] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:30.859974  8264 consensus_queue.cc:237] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.861016  8258 tablet_bootstrap.cc:654] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.862519  8504 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.862994  8504 raft_consensus.cc:697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:12:30.863273  8175 leader_election.cc:343] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.863646  8175 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:30.863762  8504 consensus_queue.cc:237] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.865806  8481 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:30.868947  8500 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:12:30.869753  8026 leader_election.cc:343] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.871511  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:30.875448  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.876866  8252 tablet_bootstrap.cc:654] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.874186  8261 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.877398  8261 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.879864  8261 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:30.881114  8026 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:30.882478  8417 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:30.882884  8059 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:30.885468  8292 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:30.883553  8504 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:30.884557  8062 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.900113  7961 catalog_manager.cc:5697] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 0 to 1, leader changed from <none> to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:30.901221  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:30.902856  7958 catalog_manager.cc:5697] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:30.903890  8175 leader_election.cc:343] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.911041  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.914239  8130 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:30.915282  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:30.916215  8481 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.916563  8481 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.916908  8481 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.921633  8253 tablet_bootstrap.cc:492] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:30.922077  8253 ts_tablet_manager.cc:1403] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.098s	user 0.024s	sys 0.031s
I20260709 13:12:30.923447  8481 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.924762  8253 raft_consensus.cc:359] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.925437  8252 tablet_bootstrap.cc:492] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:30.925892  8252 ts_tablet_manager.cc:1403] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.117s	user 0.007s	sys 0.012s
I20260709 13:12:30.926079  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:12:30.927317  8175 leader_election.cc:343] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.928233  8252 raft_consensus.cc:359] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.928853  8252 raft_consensus.cc:385] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.929003  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:30.929602  8133 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.930977  8258 tablet_bootstrap.cc:492] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:30.931421  8258 ts_tablet_manager.cc:1403] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.092s	user 0.012s	sys 0.006s
I20260709 13:12:30.935575  8292 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:30.938472  8258 raft_consensus.cc:359] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.939121  8258 raft_consensus.cc:385] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.939441  8258 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.940150  8258 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.925421  8253 raft_consensus.cc:385] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.941505  8261 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:30.941954  8253 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.929102  8252 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.942703  8253 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.944149  8252 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.950892  8253 ts_tablet_manager.cc:1434] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:12:30.951015  8252 ts_tablet_manager.cc:1434] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.025s	user 0.001s	sys 0.005s
I20260709 13:12:30.951781  8253 tablet_bootstrap.cc:492] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:30.952099  8481 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:30.951813  8252 tablet_bootstrap.cc:492] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:30.963106  8258 ts_tablet_manager.cc:1434] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.004s
I20260709 13:12:30.971993  8258 tablet_bootstrap.cc:492] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:30.970950  7959 catalog_manager.cc:5697] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.978318  8133 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:30.980114  8417 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.980576  8417 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:30.983758  8417 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:30.984349  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:30.980973  8261 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:12:30.988682  8253 tablet_bootstrap.cc:654] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.994637  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:30.995204  8261 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.995651  8261 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:12:30.989308  8026 leader_election.cc:343] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.996497  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:30.996430  8261 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:30.997982  8029 leader_election.cc:343] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:30.998385  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:30.999318  8417 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:31.000164  8264 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:12:31.004463  8252 tablet_bootstrap.cc:654] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.021214  8258 tablet_bootstrap.cc:654] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.039700  7959 catalog_manager.cc:5697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:31.052178  8292 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.052614  8292 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.059530  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:31.062057  8253 tablet_bootstrap.cc:492] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:31.062562  8253 ts_tablet_manager.cc:1403] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.111s	user 0.010s	sys 0.005s
I20260709 13:12:31.050155  8261 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.061474  8292 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:31.065778  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:31.066013  8261 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.069031  8261 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:31.070255  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:31.071790  8292 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.072717  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:31.074290  8292 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:31.071096  8133 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:31.078060  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:31.080487  8253 raft_consensus.cc:359] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.081672  8253 raft_consensus.cc:385] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.082003  8253 raft_consensus.cc:740] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
W20260709 13:12:31.082870  8175 leader_election.cc:343] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.083549  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:31.083144  8253 consensus_queue.cc:260] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.095072  7958 catalog_manager.cc:5697] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 0 to 1, leader changed from <none> to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:31.094028  8261 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:31.085829  8133 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.099299  8261 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.106834  8253 ts_tablet_manager.cc:1434] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.044s	user 0.004s	sys 0.003s
I20260709 13:12:31.112257  8261 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.132045  8253 tablet_bootstrap.cc:492] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:31.134835  8252 tablet_bootstrap.cc:492] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:31.135285  8252 ts_tablet_manager.cc:1403] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.184s	user 0.020s	sys 0.004s
I20260709 13:12:31.138500  8504 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:12:31.146816  8252 raft_consensus.cc:359] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.147500  8252 raft_consensus.cc:385] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.147758  8252 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.148389  8252 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.150866  8261 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.151573  8495 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.153050  8261 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:31.153831  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:31.153949  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:31.154556  8130 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:31.155193  8175 leader_election.cc:343] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.155575  8258 tablet_bootstrap.cc:492] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:31.155687  8264 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.156016  8258 ts_tablet_manager.cc:1403] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.184s	user 0.029s	sys 0.037s
I20260709 13:12:31.159024  8258 raft_consensus.cc:359] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.159859  8258 raft_consensus.cc:385] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.160185  8258 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.166672  8258 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.172282  8130 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:31.173027  8252 ts_tablet_manager.cc:1434] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:12:31.173599  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:31.174531  8252 tablet_bootstrap.cc:492] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:31.174696  8532 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:31.175174  8532 raft_consensus.cc:697] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:31.175978  8532 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.181945  8253 tablet_bootstrap.cc:654] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.188999  8252 tablet_bootstrap.cc:654] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.192395  8258 ts_tablet_manager.cc:1434] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 13:12:31.193439  8258 tablet_bootstrap.cc:492] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:31.206887  8253 tablet_bootstrap.cc:492] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:31.207374  8253 ts_tablet_manager.cc:1403] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.001s
I20260709 13:12:31.210577  8253 raft_consensus.cc:359] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.211417  8253 raft_consensus.cc:385] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.211741  8253 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.223477  8258 tablet_bootstrap.cc:654] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.242385  8504 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.247079  8252 tablet_bootstrap.cc:492] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:31.247530  8252 ts_tablet_manager.cc:1403] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.074s	user 0.011s	sys 0.003s
I20260709 13:12:31.248555  7960 catalog_manager.cc:5697] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 0 to 1, leader changed from <none> to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:31.250627  8252 raft_consensus.cc:359] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.251405  8252 raft_consensus.cc:385] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.251730  8252 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.212491  8253 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.258302  8252 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.254227  8253 ts_tablet_manager.cc:1434] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.047s	user 0.004s	sys 0.003s
I20260709 13:12:31.264382  8253 tablet_bootstrap.cc:492] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:31.269989  8253 tablet_bootstrap.cc:654] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.270514  8252 ts_tablet_manager.cc:1434] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:12:31.271548  8252 tablet_bootstrap.cc:492] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:31.283721  8252 tablet_bootstrap.cc:654] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.287174  8264 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.287581  8264 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:31.288902  8009 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:31.289309  8264 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:31.290793  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:31.293650  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:31.308789  8504 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:31.314465  8133 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:31.315169  8208 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:31.316329  8029 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:31.317705  8264 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:31.318009  8264 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.319434  8264 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.332554  8253 tablet_bootstrap.cc:492] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:31.333789  8253 ts_tablet_manager.cc:1403] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.070s	user 0.024s	sys 0.008s
I20260709 13:12:31.336862  8264 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.337852  8252 tablet_bootstrap.cc:492] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:31.337905  8253 raft_consensus.cc:359] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.341106  8253 raft_consensus.cc:385] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.341216  8252 ts_tablet_manager.cc:1403] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.070s	user 0.018s	sys 0.012s
I20260709 13:12:31.341404  8253 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.342595  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:31.348006  8133 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.348944  8252 raft_consensus.cc:359] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.349551  8252 raft_consensus.cc:385] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.349834  8252 raft_consensus.cc:740] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.343278  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:31.350770  8208 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.351679  8252 consensus_queue.cc:260] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.353688  8252 ts_tablet_manager.cc:1434] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.002s
I20260709 13:12:31.354509  8252 tablet_bootstrap.cc:492] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:31.341601  8258 tablet_bootstrap.cc:492] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:31.359051  8258 ts_tablet_manager.cc:1403] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.166s	user 0.043s	sys 0.029s
I20260709 13:12:31.359254  8133 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:31.342135  8264 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:31.359825  8273 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.361397  8208 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:31.362440  8029 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:31.363724  8264 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:31.363920  8532 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.364351  8532 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.364727  8264 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:31.367028  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:31.367547  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:31.368530  8175 leader_election.cc:343] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.366150  8532 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:12:31.369158  8175 leader_election.cc:343] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.369518  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:31.365685  8264 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.370314  8532 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:31.373592  8253 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.375406  8253 ts_tablet_manager.cc:1434] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.000s
I20260709 13:12:31.377055  8258 raft_consensus.cc:359] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.377702  8258 raft_consensus.cc:385] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.377981  8258 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.378609  8258 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.379312  8500 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.382418  8253 tablet_bootstrap.cc:492] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:31.384183  8252 tablet_bootstrap.cc:654] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.394742  7958 catalog_manager.cc:5697] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:31.408764  8261 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.409943  8258 ts_tablet_manager.cc:1434] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.051s	user 0.005s	sys 0.003s
I20260709 13:12:31.412628  8253 tablet_bootstrap.cc:654] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.413210  8258 tablet_bootstrap.cc:492] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:31.422624  8504 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.423969  8504 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:31.425294  8026 consensus_peers.cc:597] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:31.426796  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:31.428596  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:31.427322  8211 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
W20260709 13:12:31.430246  8101 leader_election.cc:343] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.430804  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:31.427731  8504 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:31.431752  8504 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:31.433259  8504 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.433841  8258 tablet_bootstrap.cc:654] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.434808  8500 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.433712  8504 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.435246  8500 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.437789  8500 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:31.437986  8532 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:31.439461  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:12:31.440618  8026 leader_election.cc:343] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.443935  8057 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.448148  8253 tablet_bootstrap.cc:492] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:31.448621  8253 ts_tablet_manager.cc:1403] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.067s	user 0.022s	sys 0.000s
I20260709 13:12:31.455004  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:31.455554  8211 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:31.459304  8253 raft_consensus.cc:359] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.459945  8253 raft_consensus.cc:385] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.460191  8253 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.460256  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:31.470927  8500 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:31.471282  8500 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.471698  8500 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.479821  8500 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.483582  8532 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.490125  8504 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.498788  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:31.499264  8211 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.502802  8532 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.503312  8532 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.505174  8532 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:31.508833  8211 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:31.509835  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:31.509863  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:31.493208  8500 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:31.495814  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:31.511052  8207 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.511323  8292 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:31.511817  8292 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:31.512543  8292 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:31.515861  8175 leader_election.cc:343] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.516697  8258 tablet_bootstrap.cc:492] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:31.517270  8258 ts_tablet_manager.cc:1403] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.104s	user 0.016s	sys 0.000s
I20260709 13:12:31.469061  8253 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.519335  8253 ts_tablet_manager.cc:1434] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.070s	user 0.006s	sys 0.000s
I20260709 13:12:31.524112  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:31.522301  8207 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:31.524996  8532 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.526453  8059 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.529361  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:31.525533  8504 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:12:31.532467  8026 leader_election.cc:343] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.533165  8253 tablet_bootstrap.cc:492] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:31.539089  8261 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.539538  8261 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.541102  8059 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:31.541381  8261 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:31.541641  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:31.523537  8258 raft_consensus.cc:359] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.542610  8258 raft_consensus.cc:385] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.542846  8258 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.543632  8258 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.545110  8258 ts_tablet_manager.cc:1434] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:12:31.547765  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:31.557523  8258 tablet_bootstrap.cc:492] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:31.557653  8253 tablet_bootstrap.cc:654] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.550544  8500 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:12:31.558485  8175 leader_election.cc:343] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.558987  8500 raft_consensus.cc:697] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:31.559288  7960 catalog_manager.cc:5697] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:31.565798  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:31.566396  8057 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:31.570580  8500 consensus_queue.cc:237] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.579272  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:31.582110  8261 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:31.582508  8261 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.582839  8261 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.587378  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:31.591715  8261 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.593561  8261 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:12:31.595363  8175 leader_election.cc:343] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.595748  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:31.597957  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:31.598536  8059 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.600031  8252 tablet_bootstrap.cc:492] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:31.600442  8252 ts_tablet_manager.cc:1403] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.246s	user 0.032s	sys 0.059s
I20260709 13:12:31.600883  8261 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:31.597072  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:31.603863  8130 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.610296  8258 tablet_bootstrap.cc:654] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.612512  8252 raft_consensus.cc:359] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.613158  8252 raft_consensus.cc:385] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.613425  8252 raft_consensus.cc:740] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.615298  8130 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:31.617188  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:31.617373  8252 consensus_queue.cc:260] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.618368  8532 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:31.618872  8532 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:31.619522  8252 ts_tablet_manager.cc:1434] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.004s
I20260709 13:12:31.619874  8059 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:31.620189  8252 tablet_bootstrap.cc:492] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e: Bootstrap starting.
I20260709 13:12:31.619678  8532 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.627405  8253 tablet_bootstrap.cc:492] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:31.627849  8253 ts_tablet_manager.cc:1403] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.095s	user 0.015s	sys 0.000s
I20260709 13:12:31.635934  8575 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.643471  8253 raft_consensus.cc:359] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.644378  8253 raft_consensus.cc:385] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.644636  8253 raft_consensus.cc:740] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.648262  8575 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.648734  8575 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.648636  7958 catalog_manager.cc:5697] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:31.650633  8575 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:31.648185  8253 consensus_queue.cc:260] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.651978  8253 ts_tablet_manager.cc:1434] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:12:31.652864  8253 tablet_bootstrap.cc:492] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:31.653990  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:31.655308  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:31.661150  8258 tablet_bootstrap.cc:492] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
W20260709 13:12:31.661455  8175 leader_election.cc:343] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.661578  8258 ts_tablet_manager.cc:1403] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.104s	user 0.014s	sys 0.007s
W20260709 13:12:31.662029  8175 leader_election.cc:343] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.662418  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:31.662894  8252 tablet_bootstrap.cc:654] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.663339  8532 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:31.664845  8258 raft_consensus.cc:359] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.665570  8258 raft_consensus.cc:385] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.665834  8258 raft_consensus.cc:740] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.666473  7960 catalog_manager.cc:5697] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 0 to 1, leader changed from <none> to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:31.658244  8253 tablet_bootstrap.cc:654] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.676282  8059 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.666484  8258 consensus_queue.cc:260] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.690440  8258 ts_tablet_manager.cc:1434] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:12:31.691618  8258 tablet_bootstrap.cc:492] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:31.696605  8258 tablet_bootstrap.cc:654] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.703943  8261 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.704427  8261 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.708061  8261 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:31.709192  8252 tablet_bootstrap.cc:492] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e: No bootstrap required, opened a new log
I20260709 13:12:31.709601  8252 ts_tablet_manager.cc:1403] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e: Time spent bootstrapping tablet: real 0.090s	user 0.010s	sys 0.017s
I20260709 13:12:31.710659  8258 tablet_bootstrap.cc:492] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:31.711082  8258 ts_tablet_manager.cc:1403] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.020s	user 0.004s	sys 0.005s
I20260709 13:12:31.700322  8253 tablet_bootstrap.cc:492] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:31.711797  8253 ts_tablet_manager.cc:1403] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.059s	user 0.014s	sys 0.007s
I20260709 13:12:31.712504  8252 raft_consensus.cc:359] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.713522  8252 raft_consensus.cc:385] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.713804  8252 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.713665  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:31.714359  8059 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 0.
I20260709 13:12:31.714962  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:31.715421  8252 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.715649  8253 raft_consensus.cc:359] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.716476  8253 raft_consensus.cc:385] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.716814  8253 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.717444  8253 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:31.717646  8175 leader_election.cc:343] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.718775  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:31.719465  8575 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:31.719574  8253 ts_tablet_manager.cc:1434] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:12:31.719826  8575 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.720184  8575 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.721014  8253 tablet_bootstrap.cc:492] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:31.722460  8252 ts_tablet_manager.cc:1434] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:12:31.726790  8253 tablet_bootstrap.cc:654] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.726580  8575 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.728504  8575 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:31.730084  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:31.730638  8059 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.731462  8253 tablet_bootstrap.cc:492] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:31.731815  8253 ts_tablet_manager.cc:1403] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:12:31.733577  8253 raft_consensus.cc:359] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.734068  8253 raft_consensus.cc:385] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.734290  8253 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.734977  8253 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.736590  8059 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:31.736862  8253 ts_tablet_manager.cc:1434] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:12:31.717657  8258 raft_consensus.cc:359] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.737574  8258 raft_consensus.cc:385] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.737823  8258 raft_consensus.cc:740] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.738574  8253 tablet_bootstrap.cc:492] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:31.740715  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:31.701313  8532 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:31.742221  8261 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:31.742710  8261 raft_consensus.cc:697] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:31.743418  8261 consensus_queue.cc:237] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.742863  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:12:31.750063  8175 leader_election.cc:343] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.738415  8258 consensus_queue.cc:260] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.758459  8258 ts_tablet_manager.cc:1434] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 13:12:31.765030  8500 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.765676  8500 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.767521  8258 tablet_bootstrap.cc:492] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:31.768280  8500 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:31.769852  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:31.770442  8207 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:31.751982  8253 tablet_bootstrap.cc:654] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.775475  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:31.776314  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:31.777122  8500 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:31.777436  8622 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.777920  8622 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.779959  8258 tablet_bootstrap.cc:654] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:31.780434  8026 leader_election.cc:343] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.777480  8500 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.781031  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:31.781188  8500 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.781563  8207 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:31.782155  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:31.782728  8104 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:31.783490  8495 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:31.783823  8495 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.784132  8495 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:31.784806  8101 leader_election.cc:343] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.779978  8622 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:31.788654  8500 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.794256  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:12:31.796154  8026 leader_election.cc:343] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.797619  8495 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.801857  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:31.801417  7958 catalog_manager.cc:5697] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 0 to 1, leader changed from <none> to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:31.805721  8258 tablet_bootstrap.cc:492] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:31.805805  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:31.806408  8212 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.806195  8258 ts_tablet_manager.cc:1403] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.039s	user 0.004s	sys 0.014s
I20260709 13:12:31.810174  8258 raft_consensus.cc:359] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.810809  8258 raft_consensus.cc:385] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.811054  8258 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.802315  8207 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.811640  8258 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.813238  8212 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:31.814952  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:31.816080  8264 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:31.816524  8264 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:31.817171  8264 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.802681  8495 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:31.803344  8500 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:31.818148  8207 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
W20260709 13:12:31.818608  8164 tablet.cc:2406] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:12:31.819213  8104 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:31.819836  8495 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:31.820572  8253 tablet_bootstrap.cc:492] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:31.820958  8253 ts_tablet_manager.cc:1403] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.083s	user 0.012s	sys 0.012s
I20260709 13:12:31.821208  8504 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.821637  8504 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.803565  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:12:31.823355  8101 leader_election.cc:343] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:31.823431  8253 raft_consensus.cc:359] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.823931  8504 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:31.824039  8253 raft_consensus.cc:385] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.824402  8253 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.824954  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:31.825011  8253 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.826011  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:31.826537  8495 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:31.827128  8253 ts_tablet_manager.cc:1434] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:12:31.827381  8104 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:31.828038  8253 tablet_bootstrap.cc:492] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:31.828222  8622 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:31.827374  8495 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.831175  8258 ts_tablet_manager.cc:1434] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 13:12:31.832216  8258 tablet_bootstrap.cc:492] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:31.837718  8258 tablet_bootstrap.cc:654] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.841735  8253 tablet_bootstrap.cc:654] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.860692  8253 tablet_bootstrap.cc:492] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:31.861114  8253 ts_tablet_manager.cc:1403] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.033s	user 0.008s	sys 0.004s
I20260709 13:12:31.863565  8253 raft_consensus.cc:359] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.864192  8253 raft_consensus.cc:385] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.864478  8253 raft_consensus.cc:740] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.865063  8253 consensus_queue.cc:260] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.867733  8258 tablet_bootstrap.cc:492] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:31.868232  8258 ts_tablet_manager.cc:1403] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.036s	user 0.005s	sys 0.005s
I20260709 13:12:31.870997  8258 raft_consensus.cc:359] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.871609  8258 raft_consensus.cc:385] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.871866  8258 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.872771  8258 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.874997  8258 ts_tablet_manager.cc:1434] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:12:31.875124  8253 ts_tablet_manager.cc:1434] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:12:31.876052  8253 tablet_bootstrap.cc:492] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:31.875735  8623 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:31.875936  8258 tablet_bootstrap.cc:492] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:31.888788  8258 tablet_bootstrap.cc:654] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.890175  8253 tablet_bootstrap.cc:654] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.894055  7958 catalog_manager.cc:5697] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:31.929417  8417 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.933560  8258 tablet_bootstrap.cc:492] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:31.933995  8258 ts_tablet_manager.cc:1403] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.058s	user 0.017s	sys 0.000s
I20260709 13:12:31.934379  8623 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.934828  8623 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.936864  8623 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:31.936842  8258 raft_consensus.cc:359] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.937511  8258 raft_consensus.cc:385] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.937754  8258 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.938563  8622 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.938499  8258 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.941087  8258 ts_tablet_manager.cc:1434] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:12:31.942286  8258 tablet_bootstrap.cc:492] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:31.942798  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:31.943390  8211 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:31.944402  8029 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:31.945299  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:31.945940  8130 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:31.947844  8253 tablet_bootstrap.cc:492] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:31.948498  8253 ts_tablet_manager.cc:1403] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.073s	user 0.014s	sys 0.029s
I20260709 13:12:31.949994  8258 tablet_bootstrap.cc:654] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.950750  8417 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:31.954566  8417 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:31.955231  8417 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.950920  8253 raft_consensus.cc:359] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.955937  8253 raft_consensus.cc:385] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.955973  7961 catalog_manager.cc:5697] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:31.956326  8253 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.958520  8253 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.959960  8253 ts_tablet_manager.cc:1434] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 13:12:31.960816  8253 tablet_bootstrap.cc:492] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:31.964695  8258 tablet_bootstrap.cc:492] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:31.965183  8258 ts_tablet_manager.cc:1403] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.005s
I20260709 13:12:31.968014  8258 raft_consensus.cc:359] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.968366  8504 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.968761  8258 raft_consensus.cc:385] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.969136  8258 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:31.969795  8258 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.972081  8258 ts_tablet_manager.cc:1434] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:12:31.973071  8258 tablet_bootstrap.cc:492] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:31.974248  8253 tablet_bootstrap.cc:654] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.979977  8258 tablet_bootstrap.cc:654] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:31.984297  8417 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.987628  8130 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.989285  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:31.989895  8212 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:31.990453  8253 tablet_bootstrap.cc:492] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:31.991027  8253 ts_tablet_manager.cc:1403] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.002s
I20260709 13:12:31.995076  8258 tablet_bootstrap.cc:492] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:31.995466  8258 ts_tablet_manager.cc:1403] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.003s
I20260709 13:12:31.996255  8212 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:31.997272  8029 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:31.998018  8258 raft_consensus.cc:359] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.998750  8258 raft_consensus.cc:385] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:31.998733  8500 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:31.999383  8500 raft_consensus.cc:697] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:31.998065  8253 raft_consensus.cc:359] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:31.999972  8504 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:31.999986  8253 raft_consensus.cc:385] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:32.000174  8500 consensus_queue.cc:237] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.000455  8504 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.000742  8253 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:32.002418  8504 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:32.003733  8133 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.005729  8417 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:32.007237  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:32.007798  8137 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.005157  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:32.009801  8212 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 1: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:12:32.010557  8253 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.012880  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:32.015376  8253 ts_tablet_manager.cc:1434] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:12:32.015853  8101 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:32.016650  8622 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:32.017630  8258 raft_consensus.cc:740] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:32.021097  8253 tablet_bootstrap.cc:492] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Bootstrap starting.
I20260709 13:12:32.018317  8258 consensus_queue.cc:260] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.020339  8137 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:32.026798  8261 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.028498  8258 ts_tablet_manager.cc:1434] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.002s
I20260709 13:12:32.029400  8258 tablet_bootstrap.cc:492] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:32.031436  8264 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.081426  8253 tablet_bootstrap.cc:654] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:32.085841  8623 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:32.089560  8623 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.093734  8623 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.095499  8261 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.097246  8623 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:32.099437  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:32.099989  8132 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:32.101608  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:32.102332  8211 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:32.103633  8029 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:32.106998  8653 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:32.108729  8653 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:32.109074  8653 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.112640  7961 catalog_manager.cc:5697] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:32.124537  8258 tablet_bootstrap.cc:654] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:32.135098  8654 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.096s	user 0.003s	sys 0.000s
I20260709 13:12:32.139304  8253 tablet_bootstrap.cc:492] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8: No bootstrap required, opened a new log
I20260709 13:12:32.141625  8653 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.143290  8253 ts_tablet_manager.cc:1403] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent bootstrapping tablet: real 0.122s	user 0.009s	sys 0.004s
I20260709 13:12:32.160326  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:32.160980  8137 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.165130  8253 raft_consensus.cc:359] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.166039  8253 raft_consensus.cc:385] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:32.166388  8253 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Initialized, Role: FOLLOWER
I20260709 13:12:32.171876  8653 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:32.172391  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:32.172935  8212 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.176044  8253 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.177517  8212 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:32.191570  8258 tablet_bootstrap.cc:492] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:32.191648  8029 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:32.192085  8258 ts_tablet_manager.cc:1403] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.163s	user 0.006s	sys 0.043s
I20260709 13:12:32.193131  8253 ts_tablet_manager.cc:1434] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent starting tablet: real 0.046s	user 0.005s	sys 0.009s
I20260709 13:12:32.195292  8653 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:32.194312  8258 raft_consensus.cc:359] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.182600  8137 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:32.195848  8258 raft_consensus.cc:385] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:32.196215  8258 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:32.195789  8653 raft_consensus.cc:697] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:32.197230  8653 consensus_queue.cc:237] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.197292  8258 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.199291  8258 ts_tablet_manager.cc:1434] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:12:32.200404  8258 tablet_bootstrap.cc:492] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Bootstrap starting.
I20260709 13:12:32.205964  8258 tablet_bootstrap.cc:654] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:32.217108  7960 catalog_manager.cc:5697] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:32.228659  8258 tablet_bootstrap.cc:492] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: No bootstrap required, opened a new log
I20260709 13:12:32.229167  8258 ts_tablet_manager.cc:1403] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.008s
I20260709 13:12:32.231961  8258 raft_consensus.cc:359] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.232554  8258 raft_consensus.cc:385] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:32.232842  8258 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Initialized, Role: FOLLOWER
I20260709 13:12:32.233445  8258 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.235479  8258 ts_tablet_manager.cc:1434] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:12:32.238193  8137 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.243824  8261 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.245879  8417 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.246011  8057 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.246425  8417 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.249049  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:32.249315  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:32.249627  8130 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:32.249898  8212 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 0.
I20260709 13:12:32.250813  8026 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:32.251551  8653 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:32.251904  8653 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:32.252235  8653 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.255324  8622 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.258234  8653 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.259915  8622 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.260365  8622 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.260650  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:32.261206  8137 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.261536  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:32.262415  8417 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:32.262679  8212 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.262454  8261 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:32.270299  8212 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:32.274086  8029 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:32.276085  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:32.276736  8212 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 1: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:12:32.273134  8622 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:32.273512  8653 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 1 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:32.278949  8417 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:32.288861  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:32.289217  8417 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:32.290012  8417 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.299469  8137 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:12:32.302109  8101 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:32.303457  8667 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.306948  8495 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.313359  8653 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.315207  8667 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:32.328847  8623 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.337508  7960 catalog_manager.cc:5697] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:32.367615  8667 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.368073  8667 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.370244  8667 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:32.371805  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:32.372442  8057 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:32.375464  8101 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:32.376058  8667 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:12:32.373342  8009 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:32.371233  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:32.376478  8667 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:32.376907  8212 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:32.377174  8667 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.384687  8667 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.386505  8667 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:32.387072  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:32.387619  8057 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.387665  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:32.388263  8211 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.393644  8057 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:32.393759  8211 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:32.394635  8101 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:32.395315  8667 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:32.395686  8667 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:32.396448  8667 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.406199  7961 catalog_manager.cc:5697] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:32.453408  8667 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:32.453925  8667 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.455888  8667 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:32.457326  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:32.457621  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:32.457964  8059 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:32.458199  8211 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 0.
I20260709 13:12:32.458990  8101 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:32.459656  8622 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:32.459956  8622 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:32.460222  8622 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.464635  8622 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.465988  8622 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:32.466792  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:32.466881  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:32.467353  8059 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.467427  8211 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.472476  8211 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:32.472503  8059 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:12:32.473407  8104 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:32.474004  8622 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:32.474474  8622 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:32.475175  8622 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:32.483798  7961 catalog_manager.cc:5697] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:32.541901  8623 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.552066  8653 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.634618  8653 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.647280  8500 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.690723  8500 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.701138  8417 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.876569  8504 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.886147  8504 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.917675  8667 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:32.927309  8497 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:33.866701  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:33.897305  8685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:33.898166  8687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:33.899124  6937 server_base.cc:1061] running on GCE node
W20260709 13:12:33.897977  8684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:33.902778  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:33.903206  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:33.903421  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602753903404 us; error 0 us; skew 500 ppm
I20260709 13:12:33.904059  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:33.908042  6937 webserver.cc:533] Webserver started at http://127.6.198.68:38495/ using document root <none> and password file <none>
I20260709 13:12:33.909622  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:33.910004  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:33.910383  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:33.911803  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-3-root/instance:
uuid: "965e59f272f143449b18f65db4e61d98"
format_stamp: "Formatted at 2026-07-09 13:12:33 on dist-test-slave-tqzw"
I20260709 13:12:33.917958  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:12:33.927743  8692 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:33.929991  6937 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:12:33.930420  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-3-root
uuid: "965e59f272f143449b18f65db4e61d98"
format_stamp: "Formatted at 2026-07-09 13:12:33 on dist-test-slave-tqzw"
I20260709 13:12:33.930822  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:33.949234  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:33.951049  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:33.953343  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:33.955943  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:33.956176  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:33.956470  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:33.956688  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:34.065037  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.68:44573
I20260709 13:12:34.065567  8754 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.68:44573 every 8 connection(s)
I20260709 13:12:34.079790  6937 auto_leader_rebalancer-test.cc:384] add a tserver: 965e59f272f143449b18f65db4e61d98
I20260709 13:12:34.094738  8755 heartbeater.cc:344] Connected to a master server at 127.6.198.126:38543
I20260709 13:12:34.095120  8755 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:34.095772  8755 heartbeater.cc:507] Master 127.6.198.126:38543 requested a full tablet report, sending...
I20260709 13:12:34.097417  7961 ts_manager.cc:194] Registered new tserver with Master: 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:12:34.099048  7961 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37774
I20260709 13:12:34.509938  8137 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:34.518456  8211 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:34.519896  8761 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:34.521482  8059 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:34.523605  8504 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:34.553836  8504 raft_consensus.cc:2955] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:34.555763  8212 raft_consensus.cc:2955] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:34.576964  7961 catalog_manager.cc:5697] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
W20260709 13:12:34.604542  8102 consensus_peers.cc:597] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d013f25ff51143beb65d31cc33e5a248. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:34.606137  8212 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:34.612125  8059 raft_consensus.cc:2955] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:34.621775  8137 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:34.625221  8057 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:34.626797  8759 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:34.628455  8759 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:34.649988  8176 consensus_peers.cc:597] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9fbc44c6e33e41698020173171d4ff21. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:34.650979  8532 raft_consensus.cc:2955] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:34.655999  8057 raft_consensus.cc:2955] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:34.658779  8137 raft_consensus.cc:2955] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:34.687485  7958 catalog_manager.cc:5697] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:34.696717  8137 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
W20260709 13:12:34.728871  8102 consensus_peers.cc:597] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a3e2b86010604477955f19a56daf6d04. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:34.730054  8207 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:34.731858  8059 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:34.732331  8761 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:34.733170  8622 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:34.742067  8504 raft_consensus.cc:2955] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:34.744187  8059 raft_consensus.cc:2955] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:34.743929  8207 raft_consensus.cc:2955] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:34.754300  8137 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:34.760938  7961 catalog_manager.cc:5697] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:34.765852  8207 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:34.767143  8622 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:34.770331  8102 consensus_peers.cc:597] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bd0294d309cb4f8696e693285a7b5b04. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:34.775524  8059 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:34.779443  8504 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:34.791919  8760 raft_consensus.cc:2955] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:34.793861  8207 raft_consensus.cc:2955] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:34.798938  8057 raft_consensus.cc:2955] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:34.808679  7960 catalog_manager.cc:5697] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:34.923589  8788 ts_tablet_manager.cc:933] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:34.926563  8788 tablet_copy_client.cc:323] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:12:34.938647  8148 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet d013f25ff51143beb65d31cc33e5a248 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:12:34.939220  8148 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet d013f25ff51143beb65d31cc33e5a248 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-d013f25ff51143beb65d31cc33e5a248
I20260709 13:12:34.946038  8148 tablet_copy_source_session.cc:215] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:34.950096  8788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d013f25ff51143beb65d31cc33e5a248. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:34.965433  8788 tablet_copy_client.cc:806] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:34.966080  8788 tablet_copy_client.cc:670] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:34.970227  8788 tablet_copy_client.cc:538] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:34.978484  8788 tablet_bootstrap.cc:492] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:34.999001  8788 tablet_bootstrap.cc:492] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:34.999855  8788 tablet_bootstrap.cc:492] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:35.000365  8788 ts_tablet_manager.cc:1403] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.005s
I20260709 13:12:35.002821  8788 raft_consensus.cc:359] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:35.003437  8788 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:35.003926  8788 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:35.006891  8755 heartbeater.cc:499] Master 127.6.198.126:38543 was elected leader, sending a full tablet report...
I20260709 13:12:35.007299  8788 ts_tablet_manager.cc:1434] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:12:35.009647  8148 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-d013f25ff51143beb65d31cc33e5a248 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:12:35.010032  8148 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-d013f25ff51143beb65d31cc33e5a248 on tablet d013f25ff51143beb65d31cc33e5a248 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:35.054733  8788 ts_tablet_manager.cc:933] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:35.056314  8788 tablet_copy_client.cc:323] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.67:35745
I20260709 13:12:35.065836  8222 tablet_copy_service.cc:140] P d9cd2999aa314b39997b423ec1ae509e: Received BeginTabletCopySession request for tablet 9fbc44c6e33e41698020173171d4ff21 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:55942)
I20260709 13:12:35.066407  8222 tablet_copy_service.cc:161] P d9cd2999aa314b39997b423ec1ae509e: Beginning new tablet copy session on tablet 9fbc44c6e33e41698020173171d4ff21 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:55942: session id = 965e59f272f143449b18f65db4e61d98-9fbc44c6e33e41698020173171d4ff21
I20260709 13:12:35.071412  8222 tablet_copy_source_session.cc:215] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:35.073787  8788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fbc44c6e33e41698020173171d4ff21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:35.084936  8788 tablet_copy_client.cc:806] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:35.085443  8788 tablet_copy_client.cc:670] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:35.088590  8788 tablet_copy_client.cc:538] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:35.095129  8788 tablet_bootstrap.cc:492] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:35.126752  8788 tablet_bootstrap.cc:492] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:35.127591  8788 tablet_bootstrap.cc:492] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:35.128252  8788 ts_tablet_manager.cc:1403] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.033s	user 0.028s	sys 0.004s
I20260709 13:12:35.132336  8788 raft_consensus.cc:359] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:35.133090  8788 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:35.134645  8788 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:35.136772  8788 ts_tablet_manager.cc:1434] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.004s
I20260709 13:12:35.138446  8222 tablet_copy_service.cc:342] P d9cd2999aa314b39997b423ec1ae509e: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-9fbc44c6e33e41698020173171d4ff21 received from {username='slave'} at 127.0.0.1:55942
I20260709 13:12:35.138882  8222 tablet_copy_service.cc:434] P d9cd2999aa314b39997b423ec1ae509e: ending tablet copy session 965e59f272f143449b18f65db4e61d98-9fbc44c6e33e41698020173171d4ff21 on tablet 9fbc44c6e33e41698020173171d4ff21 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:35.212867  8788 ts_tablet_manager.cc:933] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:35.214996  8788 tablet_copy_client.cc:323] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:12:35.217898  8148 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet bd0294d309cb4f8696e693285a7b5b04 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:12:35.218444  8148 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet bd0294d309cb4f8696e693285a7b5b04 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-bd0294d309cb4f8696e693285a7b5b04
I20260709 13:12:35.229177  8148 tablet_copy_source_session.cc:215] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:35.231980  8788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd0294d309cb4f8696e693285a7b5b04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:35.242961  8788 tablet_copy_client.cc:806] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:35.243434  8788 tablet_copy_client.cc:670] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:35.247833  8788 tablet_copy_client.cc:538] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:35.252243  8788 tablet_bootstrap.cc:492] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:35.268555  8788 tablet_bootstrap.cc:492] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:35.269162  8788 tablet_bootstrap.cc:492] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:35.269641  8788 ts_tablet_manager.cc:1403] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.004s
I20260709 13:12:35.271423  8788 raft_consensus.cc:359] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:35.271930  8788 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:35.272711  8788 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:35.274513  8788 ts_tablet_manager.cc:1434] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:12:35.275894  8148 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-bd0294d309cb4f8696e693285a7b5b04 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:12:35.276234  8148 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-bd0294d309cb4f8696e693285a7b5b04 on tablet bd0294d309cb4f8696e693285a7b5b04 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:35.294847  8788 ts_tablet_manager.cc:933] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:35.296311  8788 tablet_copy_client.cc:323] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:12:35.297775  8148 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet a3e2b86010604477955f19a56daf6d04 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:12:35.298269  8148 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet a3e2b86010604477955f19a56daf6d04 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-a3e2b86010604477955f19a56daf6d04
I20260709 13:12:35.306411  8148 tablet_copy_source_session.cc:215] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:35.308736  8788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3e2b86010604477955f19a56daf6d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:35.320099  8788 tablet_copy_client.cc:806] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:35.320502  8788 tablet_copy_client.cc:670] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:35.323694  8788 tablet_copy_client.cc:538] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:35.328740  8788 tablet_bootstrap.cc:492] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:35.349933  8788 tablet_bootstrap.cc:492] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:35.350620  8788 tablet_bootstrap.cc:492] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:35.351073  8788 ts_tablet_manager.cc:1403] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.012s
I20260709 13:12:35.353175  8788 raft_consensus.cc:359] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:35.353741  8788 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:35.354285  8788 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:35.358680  8788 ts_tablet_manager.cc:1434] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:12:35.359997  8148 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-a3e2b86010604477955f19a56daf6d04 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:12:35.360385  8148 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-a3e2b86010604477955f19a56daf6d04 on tablet a3e2b86010604477955f19a56daf6d04 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:35.369990  8730 raft_consensus.cc:1217] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:35.451676  8730 raft_consensus.cc:1217] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:35.775321  8730 raft_consensus.cc:1217] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:35.852536  8730 raft_consensus.cc:1217] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:35.907572  8802 raft_consensus.cc:1064] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:35.910104  8802 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:35.916558  8730 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:35.916908  8211 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:35.918260  8802 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:12:35.920820  8805 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:35.923153  8061 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:35.927531  8799 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:35.933576  8779 raft_consensus.cc:1064] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:35.937574  8779 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:35.965673  8137 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:35.967108  8532 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:36.019678  8730 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:36.022424  8057 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:36.022596  8805 raft_consensus.cc:2955] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.028105  8803 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:36.030601  8803 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:36.026151  8061 raft_consensus.cc:2955] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.044750  8207 raft_consensus.cc:2955] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.053445  8729 raft_consensus.cc:2955] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.054257  8532 raft_consensus.cc:2955] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.061336  8728 raft_consensus.cc:2955] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.068470  8137 raft_consensus.cc:2955] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.080951  7961 catalog_manager.cc:5697] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:36.082614  7958 catalog_manager.cc:5697] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.084982  8059 raft_consensus.cc:2955] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.133198  8137 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:36.139730  8729 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:36.141287  8814 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:36.143986  8212 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:36.147365  8061 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:36.148663  8814 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:36.152134  8061 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:36.151906  8799 raft_consensus.cc:2955] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.153957  8803 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:36.155416  8729 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:36.154834  8728 raft_consensus.cc:2955] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.157992  8803 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:36.161113  8061 raft_consensus.cc:2955] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.166024  8779 raft_consensus.cc:2955] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.173915  8061 raft_consensus.cc:2955] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.177146  7959 catalog_manager.cc:5697] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.178198  8728 raft_consensus.cc:2955] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.181221  7948 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9fbc44c6e33e41698020173171d4ff21 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:36.188758  8198 tablet_service.cc:1558] Processing DeleteTablet for tablet d013f25ff51143beb65d31cc33e5a248 with delete_type TABLET_DATA_TOMBSTONED (TS d9cd2999aa314b39997b423ec1ae509e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59846
I20260709 13:12:36.195369  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d013f25ff51143beb65d31cc33e5a248 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:36.200966  8819 tablet_replica.cc:333] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:12:36.202553  8819 raft_consensus.cc:2243] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:36.203168  8819 raft_consensus.cc:2272] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:36.206518  8819 ts_tablet_manager.cc:1916] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:36.220953  8819 ts_tablet_manager.cc:1929] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:36.221350  8819 log.cc:1199] T d013f25ff51143beb65d31cc33e5a248 P d9cd2999aa314b39997b423ec1ae509e: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/wals/d013f25ff51143beb65d31cc33e5a248
I20260709 13:12:36.222944  7948 catalog_manager.cc:5028] TS d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): tablet d013f25ff51143beb65d31cc33e5a248 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:36.223001  7960 catalog_manager.cc:5697] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:36.240891  8123 tablet_service.cc:1558] Processing DeleteTablet for tablet 9fbc44c6e33e41698020173171d4ff21 with delete_type TABLET_DATA_TOMBSTONED (TS 8a968cada1e4446ebb2a5e167176e649 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36038
I20260709 13:12:36.244099  8822 tablet_replica.cc:333] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:12:36.245115  8822 raft_consensus.cc:2243] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:36.245776  8822 raft_consensus.cc:2272] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:36.249337  8822 ts_tablet_manager.cc:1916] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:36.259336  8822 ts_tablet_manager.cc:1929] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:36.259658  8822 log.cc:1199] T 9fbc44c6e33e41698020173171d4ff21 P 8a968cada1e4446ebb2a5e167176e649: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/wals/9fbc44c6e33e41698020173171d4ff21
I20260709 13:12:36.261051  7945 catalog_manager.cc:5028] TS 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): tablet 9fbc44c6e33e41698020173171d4ff21 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:36.262063  8799 raft_consensus.cc:1064] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:36.263815  8799 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:36.270697  8061 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:36.271729  8814 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:36.277549  8728 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:36.280807  8799 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:36.287292  8212 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:36.289595  8805 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:36.300278  8802 raft_consensus.cc:2955] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.316852  8207 raft_consensus.cc:2955] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.324786  8061 raft_consensus.cc:2955] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.333006  8728 raft_consensus.cc:2955] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.343321  8805 raft_consensus.cc:1064] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:36.342597  7958 catalog_manager.cc:5697] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:36.346208  8805 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:36.369911  8059 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:36.371330  8805 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:36.372552  8728 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:36.373927  8802 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:36.376377  8207 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:36.378674  8814 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:36.378842  8137 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:36.386018  8729 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:36.386560  8207 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:36.387399  8805 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:36.390748  8805 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:36.396935  8800 raft_consensus.cc:2955] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.408337  8729 raft_consensus.cc:2955] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.410969  8208 raft_consensus.cc:2955] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.419900  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bd0294d309cb4f8696e693285a7b5b04 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:36.427712  8800 raft_consensus.cc:2955] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.437381  7958 catalog_manager.cc:5697] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.441536  8208 raft_consensus.cc:2955] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.455368  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet bd0294d309cb4f8696e693285a7b5b04 with delete_type TABLET_DATA_TOMBSTONED (TS 84d775d1767e4cc4aa7b9c627d8f0fb8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49202
I20260709 13:12:36.473039  8831 tablet_replica.cc:333] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:12:36.474054  8831 raft_consensus.cc:2243] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:36.474732  8831 raft_consensus.cc:2272] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:36.478116  8831 ts_tablet_manager.cc:1916] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:36.477720  8059 raft_consensus.cc:2955] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.489571  7958 catalog_manager.cc:5697] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.493506  8729 raft_consensus.cc:2955] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.492604  8831 ts_tablet_manager.cc:1929] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:36.507925  8831 log.cc:1199] T bd0294d309cb4f8696e693285a7b5b04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/wals/bd0294d309cb4f8696e693285a7b5b04
I20260709 13:12:36.510293  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet bd0294d309cb4f8696e693285a7b5b04 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:36.538440  8137 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:36.546164  8729 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:36.548771  8805 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:36.551119  8208 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:36.552559  8833 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:36.554926  8805 raft_consensus.cc:2955] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.559581  8207 raft_consensus.cc:2955] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.562960  8729 raft_consensus.cc:2955] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.567584  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a3e2b86010604477955f19a56daf6d04 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:36.575628  7961 catalog_manager.cc:5697] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:36.588270  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet a3e2b86010604477955f19a56daf6d04 with delete_type TABLET_DATA_TOMBSTONED (TS 84d775d1767e4cc4aa7b9c627d8f0fb8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49202
I20260709 13:12:36.589152  8831 tablet_replica.cc:333] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:12:36.604462  8831 raft_consensus.cc:2243] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:36.604990  8831 raft_consensus.cc:2272] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:36.607229  8831 ts_tablet_manager.cc:1916] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:36.616617  8831 ts_tablet_manager.cc:1929] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:36.616927  8831 log.cc:1199] T a3e2b86010604477955f19a56daf6d04 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/wals/a3e2b86010604477955f19a56daf6d04
I20260709 13:12:36.617956  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet a3e2b86010604477955f19a56daf6d04 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:37.655014  8207 consensus_queue.cc:237] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:37.659917  8130 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:37.661177  8176 consensus_peers.cc:597] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c97c3defa48b473aaf762b3a27c0cba0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:37.661580  8803 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:37.662067  8061 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:37.663165  8779 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:37.669723  8803 raft_consensus.cc:2955] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:37.678360  8137 raft_consensus.cc:2955] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:37.688987  8059 raft_consensus.cc:2955] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:37.691310  7960 catalog_manager.cc:5697] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:37.716236  8059 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:37.726114  8137 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:37.727478  8207 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:37.729982  8820 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:37.732455  8821 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:37.745215  8837 raft_consensus.cc:2955] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
W20260709 13:12:37.748759  8027 consensus_peers.cc:597] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 51726a5d5cad44fd8e9b6803da998cb2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:37.749707  8207 raft_consensus.cc:2955] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:37.755360  8137 raft_consensus.cc:2955] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:37.760808  8211 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:37.768689  7958 catalog_manager.cc:5697] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:37.771515  8130 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:37.772383  8176 consensus_peers.cc:597] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bc4cbdf31b174089be94b5f87a22f629. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:37.773468  8841 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:37.774765  8059 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:37.776598  8841 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:37.789284  8843 raft_consensus.cc:2955] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:37.794855  8137 raft_consensus.cc:2955] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:37.794903  8059 raft_consensus.cc:2955] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:37.802804  8130 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
W20260709 13:12:37.811443  8102 consensus_peers.cc:597] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 481500a64cdb4a7fb94658807295dcbe. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:37.813143  8211 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:37.813627  7961 catalog_manager.cc:5697] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:37.814532  8835 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:37.823688  8059 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:37.826530  8832 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:37.845826  8835 raft_consensus.cc:2955] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:37.847425  8207 raft_consensus.cc:2955] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:37.850829  8059 raft_consensus.cc:2955] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:37.867841  7958 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:38.150055  8862 ts_tablet_manager.cc:933] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:38.151973  8863 ts_tablet_manager.cc:933] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
I20260709 13:12:38.162300  8863 tablet_copy_client.cc:323] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.65:45919
I20260709 13:12:38.165253  8862 tablet_copy_client.cc:323] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.67:35745
I20260709 13:12:38.167600  8222 tablet_copy_service.cc:140] P d9cd2999aa314b39997b423ec1ae509e: Received BeginTabletCopySession request for tablet c97c3defa48b473aaf762b3a27c0cba0 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:55942)
I20260709 13:12:38.168108  8222 tablet_copy_service.cc:161] P d9cd2999aa314b39997b423ec1ae509e: Beginning new tablet copy session on tablet c97c3defa48b473aaf762b3a27c0cba0 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:55942: session id = 965e59f272f143449b18f65db4e61d98-c97c3defa48b473aaf762b3a27c0cba0
I20260709 13:12:38.175524  8222 tablet_copy_source_session.cc:215] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:38.178541  8862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97c3defa48b473aaf762b3a27c0cba0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:38.200268  8073 tablet_copy_service.cc:140] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Received BeginTabletCopySession request for tablet 51726a5d5cad44fd8e9b6803da998cb2 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:37156)
I20260709 13:12:38.200904  8073 tablet_copy_service.cc:161] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Beginning new tablet copy session on tablet 51726a5d5cad44fd8e9b6803da998cb2 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:37156: session id = 965e59f272f143449b18f65db4e61d98-51726a5d5cad44fd8e9b6803da998cb2
I20260709 13:12:38.209348  8862 tablet_copy_client.cc:806] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:38.209918  8862 tablet_copy_client.cc:670] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:38.217532  8073 tablet_copy_source_session.cc:215] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:38.220549  8863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51726a5d5cad44fd8e9b6803da998cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:38.221138  8862 tablet_copy_client.cc:538] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:38.229151  8867 ts_tablet_manager.cc:933] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:38.229667  8862 tablet_bootstrap.cc:492] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:38.236799  8863 tablet_copy_client.cc:806] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:38.237350  8863 tablet_copy_client.cc:670] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:38.240880  8863 tablet_copy_client.cc:538] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:38.249858  8863 tablet_bootstrap.cc:492] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:38.250449  8867 tablet_copy_client.cc:323] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.67:35745
I20260709 13:12:38.255937  8222 tablet_copy_service.cc:140] P d9cd2999aa314b39997b423ec1ae509e: Received BeginTabletCopySession request for tablet bc4cbdf31b174089be94b5f87a22f629 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:55942)
I20260709 13:12:38.256443  8222 tablet_copy_service.cc:161] P d9cd2999aa314b39997b423ec1ae509e: Beginning new tablet copy session on tablet bc4cbdf31b174089be94b5f87a22f629 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:55942: session id = 965e59f272f143449b18f65db4e61d98-bc4cbdf31b174089be94b5f87a22f629
I20260709 13:12:38.261456  8862 tablet_bootstrap.cc:492] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:38.262056  8222 tablet_copy_source_session.cc:215] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:38.262245  8862 tablet_bootstrap.cc:492] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:38.262995  8862 ts_tablet_manager.cc:1403] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.034s	user 0.025s	sys 0.005s
I20260709 13:12:38.265973  8862 raft_consensus.cc:359] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:38.267148  8862 raft_consensus.cc:740] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:38.267812  8862 consensus_queue.cc:260] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:38.277922  8867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc4cbdf31b174089be94b5f87a22f629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:38.279078  8862 ts_tablet_manager.cc:1434] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.004s
I20260709 13:12:38.282222  8222 tablet_copy_service.cc:342] P d9cd2999aa314b39997b423ec1ae509e: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-c97c3defa48b473aaf762b3a27c0cba0 received from {username='slave'} at 127.0.0.1:55942
I20260709 13:12:38.282629  8222 tablet_copy_service.cc:434] P d9cd2999aa314b39997b423ec1ae509e: ending tablet copy session 965e59f272f143449b18f65db4e61d98-c97c3defa48b473aaf762b3a27c0cba0 on tablet c97c3defa48b473aaf762b3a27c0cba0 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:38.292860  8867 tablet_copy_client.cc:806] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:38.293514  8867 tablet_copy_client.cc:670] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:38.296861  8867 tablet_copy_client.cc:538] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:38.298425  8863 tablet_bootstrap.cc:492] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:38.299270  8863 tablet_bootstrap.cc:492] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:38.299881  8863 ts_tablet_manager.cc:1403] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.011s
I20260709 13:12:38.302819  8863 raft_consensus.cc:359] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:38.303429  8867 tablet_bootstrap.cc:492] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:38.303663  8863 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:38.304378  8863 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:38.306555  8863 ts_tablet_manager.cc:1434] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.002s
I20260709 13:12:38.308131  8073 tablet_copy_service.cc:342] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-51726a5d5cad44fd8e9b6803da998cb2 received from {username='slave'} at 127.0.0.1:37156
I20260709 13:12:38.308537  8073 tablet_copy_service.cc:434] P 84d775d1767e4cc4aa7b9c627d8f0fb8: ending tablet copy session 965e59f272f143449b18f65db4e61d98-51726a5d5cad44fd8e9b6803da998cb2 on tablet 51726a5d5cad44fd8e9b6803da998cb2 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:38.330266  8867 tablet_bootstrap.cc:492] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:38.331146  8867 tablet_bootstrap.cc:492] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:38.331792  8867 ts_tablet_manager.cc:1403] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.009s
I20260709 13:12:38.334609  8867 raft_consensus.cc:359] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:38.335431  8867 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:38.336056  8867 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:38.338605  8867 ts_tablet_manager.cc:1434] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:12:38.340302  8222 tablet_copy_service.cc:342] P d9cd2999aa314b39997b423ec1ae509e: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-bc4cbdf31b174089be94b5f87a22f629 received from {username='slave'} at 127.0.0.1:55942
I20260709 13:12:38.340688  8222 tablet_copy_service.cc:434] P d9cd2999aa314b39997b423ec1ae509e: ending tablet copy session 965e59f272f143449b18f65db4e61d98-bc4cbdf31b174089be94b5f87a22f629 on tablet bc4cbdf31b174089be94b5f87a22f629 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:38.381263  8867 ts_tablet_manager.cc:933] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:38.382802  8867 tablet_copy_client.cc:323] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:12:38.384160  8148 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet 481500a64cdb4a7fb94658807295dcbe from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:12:38.384639  8148 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet 481500a64cdb4a7fb94658807295dcbe from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-481500a64cdb4a7fb94658807295dcbe
I20260709 13:12:38.390553  8148 tablet_copy_source_session.cc:215] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:38.393002  8867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 481500a64cdb4a7fb94658807295dcbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:38.403301  8867 tablet_copy_client.cc:806] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:38.403764  8867 tablet_copy_client.cc:670] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:38.406944  8867 tablet_copy_client.cc:538] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:38.413444  8867 tablet_bootstrap.cc:492] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:38.429854  8867 tablet_bootstrap.cc:492] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:38.430682  8867 tablet_bootstrap.cc:492] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:38.431253  8867 ts_tablet_manager.cc:1403] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.018s	user 0.018s	sys 0.001s
I20260709 13:12:38.433774  8867 raft_consensus.cc:359] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:38.434562  8867 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:38.435061  8867 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:38.436710  8867 ts_tablet_manager.cc:1434] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:12:38.438951  8148 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-481500a64cdb4a7fb94658807295dcbe received from {username='slave'} at 127.0.0.1:46944
I20260709 13:12:38.439330  8148 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-481500a64cdb4a7fb94658807295dcbe on tablet 481500a64cdb4a7fb94658807295dcbe with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:38.558763  8728 raft_consensus.cc:1217] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:38.578876  8728 raft_consensus.cc:1217] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:38.782402  8728 raft_consensus.cc:1217] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:38.824190  8728 raft_consensus.cc:1217] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:39.096057  8837 raft_consensus.cc:1064] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:39.098559  8837 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:39.106771  8212 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.107451  8728 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.109831  8821 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:39.110599  8877 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:12:39.127171  8137 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.129248  8880 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:12:39.167179  8881 raft_consensus.cc:2955] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:39.170010  8843 raft_consensus.cc:1064] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:39.169450  8132 raft_consensus.cc:2955] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:39.171064  8729 raft_consensus.cc:2955] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:39.179689  8842 raft_consensus.cc:1064] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:39.172466  8843 consensus_queue.cc:237] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:39.188822  8842 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:39.196991  7961 catalog_manager.cc:5697] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:39.205780  8207 raft_consensus.cc:2955] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:39.214612  8729 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.220177  8137 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.220320  8875 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:39.220849  8729 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.222630  8132 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.225922  8875 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:39.233000  8875 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:39.234418  8805 raft_consensus.cc:1064] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:39.235347  8875 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:39.239578  8805 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:39.258949  8059 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.275020  8875 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:39.288806  8062 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.288335  8892 raft_consensus.cc:2955] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:39.297855  8875 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:39.301724  8137 raft_consensus.cc:2955] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:39.306737  8729 raft_consensus.cc:2955] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:39.399616  8061 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:12:39.403656  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 47231.8us
I20260709 13:12:39.424957  8207 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.430260  8835 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:12:39.449414  7961 catalog_manager.cc:5697] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:39.452603  8730 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:39.489799  8879 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:39.492362  8062 raft_consensus.cc:2955] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:39.495152  8132 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:39.509954  8878 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
W20260709 13:12:39.585644  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 81239.7us
I20260709 13:12:39.644408  8881 raft_consensus.cc:2955] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:39.648064  8057 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.661232  8892 raft_consensus.cc:2955] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:39.677227  8730 raft_consensus.cc:2955] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:12:39.709954  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 191929us
I20260709 13:12:39.761268  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 51726a5d5cad44fd8e9b6803da998cb2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:12:39.768486  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 142109us
I20260709 13:12:39.816216  8730 raft_consensus.cc:2955] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:39.854475  8728 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:39.870576  7961 catalog_manager.cc:5697] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:39.882058  8132 raft_consensus.cc:2955] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:12:39.901897  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 105488us
I20260709 13:12:39.904695  8062 raft_consensus.cc:2955] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:39.927809  8902 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:12:39.928094  8198 tablet_service.cc:1558] Processing DeleteTablet for tablet 51726a5d5cad44fd8e9b6803da998cb2 with delete_type TABLET_DATA_TOMBSTONED (TS d9cd2999aa314b39997b423ec1ae509e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59846
I20260709 13:12:39.945779  7957 catalog_manager.cc:5697] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:40.000797  8130 raft_consensus.cc:2955] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:40.038132  8835 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:40.071844  8921 tablet_replica.cc:333] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:12:40.081537  8921 raft_consensus.cc:2243] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:40.082857  8921 raft_consensus.cc:2272] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:40.097812  8211 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:40.114686  8729 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:40.119277  8913 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:40.123978  8062 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:40.144176  8920 raft_consensus.cc:2955] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:40.157953  7948 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bc4cbdf31b174089be94b5f87a22f629 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:40.169607  8729 raft_consensus.cc:2955] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:40.175535  8207 consensus_queue.cc:237] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:40.183411  8902 raft_consensus.cc:2955] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:40.185010  8730 raft_consensus.cc:2955] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:40.203187  8206 raft_consensus.cc:2955] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:40.220496  8058 raft_consensus.cc:2955] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:40.222837  8920 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:40.224610  8921 ts_tablet_manager.cc:1916] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:40.232200  8062 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:40.238220  8730 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:40.243767  7960 catalog_manager.cc:5697] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:40.262899  8919 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:40.280426  8920 raft_consensus.cc:2955] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:40.292533  8123 tablet_service.cc:1558] Processing DeleteTablet for tablet bc4cbdf31b174089be94b5f87a22f629 with delete_type TABLET_DATA_TOMBSTONED (TS 8a968cada1e4446ebb2a5e167176e649 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36038
I20260709 13:12:40.298877  8913 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:40.285777  8730 raft_consensus.cc:2955] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:40.286791  7961 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:40.326565  8059 raft_consensus.cc:2955] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:40.354401  8062 raft_consensus.cc:2955] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:40.407022  8921 ts_tablet_manager.cc:1929] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:40.407719  8921 log.cc:1199] T 51726a5d5cad44fd8e9b6803da998cb2 P d9cd2999aa314b39997b423ec1ae509e: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/wals/51726a5d5cad44fd8e9b6803da998cb2
I20260709 13:12:40.409718  7948 catalog_manager.cc:5028] TS d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): tablet 51726a5d5cad44fd8e9b6803da998cb2 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:40.411422  8132 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:40.414254  8924 tablet_replica.cc:333] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:12:40.422111  7948 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c97c3defa48b473aaf762b3a27c0cba0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:40.423879  8924 raft_consensus.cc:2243] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:40.425035  8924 raft_consensus.cc:2272] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:40.428144  8730 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:40.430315  8859 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:40.430728  8924 ts_tablet_manager.cc:1916] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:40.438639  8832 raft_consensus.cc:2955] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:40.444013  8730 raft_consensus.cc:2955] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:40.451732  7960 catalog_manager.cc:5697] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:40.462947  8062 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:40.467897  7961 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:40.469368  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 481500a64cdb4a7fb94658807295dcbe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:40.476789  8123 tablet_service.cc:1558] Processing DeleteTablet for tablet c97c3defa48b473aaf762b3a27c0cba0 with delete_type TABLET_DATA_TOMBSTONED (TS 8a968cada1e4446ebb2a5e167176e649 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36038
I20260709 13:12:40.487689  8901 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:40.488325  8198 tablet_service.cc:1558] Processing DeleteTablet for tablet 481500a64cdb4a7fb94658807295dcbe with delete_type TABLET_DATA_TOMBSTONED (TS d9cd2999aa314b39997b423ec1ae509e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59846
I20260709 13:12:40.493507  8924 ts_tablet_manager.cc:1929] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:40.494006  8924 log.cc:1199] T bc4cbdf31b174089be94b5f87a22f629 P 8a968cada1e4446ebb2a5e167176e649: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/wals/bc4cbdf31b174089be94b5f87a22f629
I20260709 13:12:40.495852  8924 tablet_replica.cc:333] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:12:40.494928  8058 raft_consensus.cc:2955] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:40.496973  8924 raft_consensus.cc:2243] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:40.503808  8921 tablet_replica.cc:333] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:12:40.505966  8924 raft_consensus.cc:2272] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:40.504853  8921 raft_consensus.cc:2243] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:40.509111  8921 raft_consensus.cc:2272] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:40.509249  8924 ts_tablet_manager.cc:1916] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:40.514526  8921 ts_tablet_manager.cc:1916] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:40.516242  7945 catalog_manager.cc:5028] TS 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): tablet bc4cbdf31b174089be94b5f87a22f629 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:40.538010  8924 ts_tablet_manager.cc:1929] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:40.538582  8924 log.cc:1199] T c97c3defa48b473aaf762b3a27c0cba0 P 8a968cada1e4446ebb2a5e167176e649: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/wals/c97c3defa48b473aaf762b3a27c0cba0
I20260709 13:12:40.540128  7945 catalog_manager.cc:5028] TS 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): tablet c97c3defa48b473aaf762b3a27c0cba0 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:40.541275  8921 ts_tablet_manager.cc:1929] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:40.541644  8921 log.cc:1199] T 481500a64cdb4a7fb94658807295dcbe P d9cd2999aa314b39997b423ec1ae509e: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/wals/481500a64cdb4a7fb94658807295dcbe
I20260709 13:12:40.543133  7948 catalog_manager.cc:5028] TS d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): tablet 481500a64cdb4a7fb94658807295dcbe (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:41.598788  8137 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:41.603859  8211 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:41.604729  8102 consensus_peers.cc:597] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b9bd4708595b40438af51aba853a0f94. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:41.605172  8058 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:41.605530  8832 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:41.606528  8925 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:41.612514  8832 raft_consensus.cc:2955] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:41.613981  8211 raft_consensus.cc:2955] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:41.616812  8062 raft_consensus.cc:2955] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:41.632380  8137 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:41.628609  7960 catalog_manager.cc:5697] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:41.651710  8207 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:41.653308  8929 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:41.656589  8102 consensus_peers.cc:597] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6771ba07746943f6a958ca7be1990296. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:41.658941  8062 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:41.660444  8925 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:41.679590  8901 raft_consensus.cc:2955] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:41.692114  8207 raft_consensus.cc:2955] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:41.694896  8058 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:41.709874  8062 raft_consensus.cc:2955] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:41.712797  7961 catalog_manager.cc:5697] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
W20260709 13:12:41.726536  8027 consensus_peers.cc:597] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8dd7949df539418cbc00001111885883. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:41.746047  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 54620us
I20260709 13:12:41.749624  8211 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:41.750392  8137 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:41.751013  8942 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:41.751940  8937 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:41.759895  8937 raft_consensus.cc:2955] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:41.761459  8211 raft_consensus.cc:2955] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:41.764518  8130 raft_consensus.cc:2955] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:41.775032  8207 consensus_queue.cc:237] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:41.776065  7960 catalog_manager.cc:5697] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
W20260709 13:12:41.795475  8176 consensus_peers.cc:597] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ed7f54a64a14435691eeef2ece07e5dc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:41.795532  8058 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:41.795953  8130 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:41.797515  8913 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:41.799772  8913 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:41.807078  8946 raft_consensus.cc:2955] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:41.812206  8062 raft_consensus.cc:2955] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:41.816339  8130 raft_consensus.cc:2955] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:41.832592  7960 catalog_manager.cc:5697] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:42.036541  8949 ts_tablet_manager.cc:933] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:42.038545  8949 tablet_copy_client.cc:323] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:12:42.040256  8148 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet 6771ba07746943f6a958ca7be1990296 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:12:42.040783  8148 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet 6771ba07746943f6a958ca7be1990296 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-6771ba07746943f6a958ca7be1990296
I20260709 13:12:42.046947  8148 tablet_copy_source_session.cc:215] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:42.049530  8949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6771ba07746943f6a958ca7be1990296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:42.061290  8949 tablet_copy_client.cc:806] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:42.061851  8949 tablet_copy_client.cc:670] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:42.065631  8949 tablet_copy_client.cc:538] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:42.074007  8949 tablet_bootstrap.cc:492] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:42.093724  8949 tablet_bootstrap.cc:492] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:42.094660  8949 tablet_bootstrap.cc:492] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:42.095291  8949 ts_tablet_manager.cc:1403] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.022s	user 0.020s	sys 0.004s
I20260709 13:12:42.098187  8949 raft_consensus.cc:359] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:42.099007  8949 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:42.099685  8949 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:42.103106  8949 ts_tablet_manager.cc:1434] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:12:42.105264  8148 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-6771ba07746943f6a958ca7be1990296 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:12:42.105701  8148 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-6771ba07746943f6a958ca7be1990296 on tablet 6771ba07746943f6a958ca7be1990296 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:42.192046  8949 ts_tablet_manager.cc:933] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:42.193576  8949 tablet_copy_client.cc:323] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:12:42.195137  8148 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet b9bd4708595b40438af51aba853a0f94 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:12:42.195571  8148 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet b9bd4708595b40438af51aba853a0f94 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-b9bd4708595b40438af51aba853a0f94
I20260709 13:12:42.201599  8148 tablet_copy_source_session.cc:215] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:42.206413  8949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9bd4708595b40438af51aba853a0f94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:42.226373  8949 tablet_copy_client.cc:806] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:42.226979  8949 tablet_copy_client.cc:670] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:42.236531  8949 tablet_copy_client.cc:538] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:42.245174  8949 tablet_bootstrap.cc:492] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:42.268422  8949 tablet_bootstrap.cc:492] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:42.269279  8949 tablet_bootstrap.cc:492] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:42.270058  8949 ts_tablet_manager.cc:1403] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.025s	user 0.020s	sys 0.004s
I20260709 13:12:42.272812  8949 raft_consensus.cc:359] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:42.273603  8949 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:42.274242  8949 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:42.276533  8949 ts_tablet_manager.cc:1434] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:12:42.278048  8148 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-b9bd4708595b40438af51aba853a0f94 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:12:42.278456  8148 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-b9bd4708595b40438af51aba853a0f94 on tablet b9bd4708595b40438af51aba853a0f94 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:42.297725  8949 ts_tablet_manager.cc:933] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:42.299361  8949 tablet_copy_client.cc:323] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.67:35745
I20260709 13:12:42.301007  8222 tablet_copy_service.cc:140] P d9cd2999aa314b39997b423ec1ae509e: Received BeginTabletCopySession request for tablet ed7f54a64a14435691eeef2ece07e5dc from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:55942)
I20260709 13:12:42.301463  8222 tablet_copy_service.cc:161] P d9cd2999aa314b39997b423ec1ae509e: Beginning new tablet copy session on tablet ed7f54a64a14435691eeef2ece07e5dc from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:55942: session id = 965e59f272f143449b18f65db4e61d98-ed7f54a64a14435691eeef2ece07e5dc
I20260709 13:12:42.307710  8956 ts_tablet_manager.cc:933] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
I20260709 13:12:42.310250  8222 tablet_copy_source_session.cc:215] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:42.313073  8956 tablet_copy_client.cc:323] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.65:45919
I20260709 13:12:42.314049  8949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed7f54a64a14435691eeef2ece07e5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:42.315008  8073 tablet_copy_service.cc:140] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Received BeginTabletCopySession request for tablet 8dd7949df539418cbc00001111885883 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:37156)
I20260709 13:12:42.315570  8073 tablet_copy_service.cc:161] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Beginning new tablet copy session on tablet 8dd7949df539418cbc00001111885883 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:37156: session id = 965e59f272f143449b18f65db4e61d98-8dd7949df539418cbc00001111885883
I20260709 13:12:42.324028  8073 tablet_copy_source_session.cc:215] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:42.327243  8956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dd7949df539418cbc00001111885883. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:42.328522  8949 tablet_copy_client.cc:806] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:42.329067  8949 tablet_copy_client.cc:670] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:42.335704  8949 tablet_copy_client.cc:538] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:42.342453  8956 tablet_copy_client.cc:806] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:42.343063  8956 tablet_copy_client.cc:670] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:42.343946  8949 tablet_bootstrap.cc:492] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:42.346621  8956 tablet_copy_client.cc:538] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:42.355576  8956 tablet_bootstrap.cc:492] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:42.364292  8949 tablet_bootstrap.cc:492] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:42.365156  8949 tablet_bootstrap.cc:492] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:42.365780  8949 ts_tablet_manager.cc:1403] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.004s
I20260709 13:12:42.368366  8949 raft_consensus.cc:359] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:42.369272  8949 raft_consensus.cc:740] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:42.370101  8949 consensus_queue.cc:260] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:42.373946  8949 ts_tablet_manager.cc:1434] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 13:12:42.378264  8222 tablet_copy_service.cc:342] P d9cd2999aa314b39997b423ec1ae509e: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-ed7f54a64a14435691eeef2ece07e5dc received from {username='slave'} at 127.0.0.1:55942
I20260709 13:12:42.378693  8222 tablet_copy_service.cc:434] P d9cd2999aa314b39997b423ec1ae509e: ending tablet copy session 965e59f272f143449b18f65db4e61d98-ed7f54a64a14435691eeef2ece07e5dc on tablet ed7f54a64a14435691eeef2ece07e5dc with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:42.383507  8956 tablet_bootstrap.cc:492] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:42.384459  8956 tablet_bootstrap.cc:492] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:42.385093  8956 ts_tablet_manager.cc:1403] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.008s
I20260709 13:12:42.387993  8956 raft_consensus.cc:359] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:42.388900  8956 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:42.389513  8956 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:42.392081  8956 ts_tablet_manager.cc:1434] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:12:42.393923  8073 tablet_copy_service.cc:342] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-8dd7949df539418cbc00001111885883 received from {username='slave'} at 127.0.0.1:37156
I20260709 13:12:42.394284  8073 tablet_copy_service.cc:434] P 84d775d1767e4cc4aa7b9c627d8f0fb8: ending tablet copy session 965e59f272f143449b18f65db4e61d98-8dd7949df539418cbc00001111885883 on tablet 8dd7949df539418cbc00001111885883 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:42.612344  8730 raft_consensus.cc:1217] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:42.751158  8730 raft_consensus.cc:1217] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:42.788127  8730 raft_consensus.cc:1217] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:42.823350  8730 raft_consensus.cc:1217] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:43.067862  8939 raft_consensus.cc:1064] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:43.070462  8939 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:43.076443  8207 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.076522  8058 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.078712  8730 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.078985  8961 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.082450  8901 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.085456  8901 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.103499  8938 raft_consensus.cc:2955] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.105996  8207 raft_consensus.cc:2955] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.112181  8730 raft_consensus.cc:2955] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.118608  8058 raft_consensus.cc:2955] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.120601  7957 catalog_manager.cc:5697] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.156965  8137 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:43.160746  8960 raft_consensus.cc:1064] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:43.163365  8960 consensus_queue.cc:237] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:43.166512  8730 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:43.167701  8901 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.171468  8207 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:43.172775  8901 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.176781  8939 raft_consensus.cc:2955] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.181522  8137 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.182798  8730 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.184052  8059 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.183377  8960 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:12:43.186693  8919 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.191181  8919 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.191054  8730 raft_consensus.cc:2955] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.201020  8207 raft_consensus.cc:2955] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.213284  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6771ba07746943f6a958ca7be1990296 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:43.212486  7960 catalog_manager.cc:5697] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.238142  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet 6771ba07746943f6a958ca7be1990296 with delete_type TABLET_DATA_TOMBSTONED (TS 84d775d1767e4cc4aa7b9c627d8f0fb8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49202
I20260709 13:12:43.251178  8946 raft_consensus.cc:2955] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.259716  8974 tablet_replica.cc:333] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:12:43.261188  8974 raft_consensus.cc:2243] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:43.261034  8058 raft_consensus.cc:2955] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.262256  8974 raft_consensus.cc:2272] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:43.272212  8729 raft_consensus.cc:2955] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.275123  8137 raft_consensus.cc:2955] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.280225  8939 raft_consensus.cc:1064] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:43.283676  8939 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:43.285581  7958 catalog_manager.cc:5697] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:43.300175  8974 ts_tablet_manager.cc:1916] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:43.323518  8207 consensus_queue.cc:237] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:43.331215  8058 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:43.333760  8919 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.352418  8207 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.353978  8961 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.362789  8730 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:43.364797  8729 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.366154  8901 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.366994  8946 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.371156  8919 raft_consensus.cc:2955] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.414479  8974 ts_tablet_manager.cc:1929] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:43.422837  8974 log.cc:1199] T 6771ba07746943f6a958ca7be1990296 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/wals/6771ba07746943f6a958ca7be1990296
I20260709 13:12:43.436558  8976 raft_consensus.cc:1064] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:43.436475  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet 6771ba07746943f6a958ca7be1990296 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:43.439369  8976 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:43.440218  8729 raft_consensus.cc:2955] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.451115  7948 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ed7f54a64a14435691eeef2ece07e5dc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:43.452371  8059 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.461318  7961 catalog_manager.cc:5697] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:43.460762  8961 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:12:43.474874  8123 tablet_service.cc:1558] Processing DeleteTablet for tablet ed7f54a64a14435691eeef2ece07e5dc with delete_type TABLET_DATA_TOMBSTONED (TS 8a968cada1e4446ebb2a5e167176e649 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36038
I20260709 13:12:43.483675  8058 raft_consensus.cc:2955] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.502079  8981 raft_consensus.cc:2955] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.504084  8730 raft_consensus.cc:2955] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.519445  8729 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.521011  7958 catalog_manager.cc:5697] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.523169  8941 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:43.526729  8207 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.538625  8941 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.541975  8207 raft_consensus.cc:2955] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.546324  8130 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:43.550899  8060 raft_consensus.cc:2955] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.560005  8986 tablet_replica.cc:333] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:12:43.563596  8976 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.570081  8137 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:43.576190  8730 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:43.577354  8207 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:43.577554  8981 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.583256  8901 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.587075  8901 raft_consensus.cc:2955] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.588094  8730 raft_consensus.cc:2955] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.601527  8207 raft_consensus.cc:2955] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.604552  8986 raft_consensus.cc:2243] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:43.606917  8986 raft_consensus.cc:2272] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:43.616101  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b9bd4708595b40438af51aba853a0f94 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:43.611274  7961 catalog_manager.cc:5697] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.624413  8986 ts_tablet_manager.cc:1916] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:43.644029  8986 ts_tablet_manager.cc:1929] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:43.644464  8986 log.cc:1199] T ed7f54a64a14435691eeef2ece07e5dc P 8a968cada1e4446ebb2a5e167176e649: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/wals/ed7f54a64a14435691eeef2ece07e5dc
I20260709 13:12:43.648234  8978 raft_consensus.cc:2955] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.650333  8210 raft_consensus.cc:2955] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.651856  8137 raft_consensus.cc:2955] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.651857  7945 catalog_manager.cc:5028] TS 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): tablet ed7f54a64a14435691eeef2ece07e5dc (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:43.660072  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet b9bd4708595b40438af51aba853a0f94 with delete_type TABLET_DATA_TOMBSTONED (TS 84d775d1767e4cc4aa7b9c627d8f0fb8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49202
I20260709 13:12:43.661002  8974 tablet_replica.cc:333] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:12:43.661952  8974 raft_consensus.cc:2243] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:43.662647  8974 raft_consensus.cc:2272] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:43.667080  8974 ts_tablet_manager.cc:1916] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:43.686538  7960 catalog_manager.cc:5697] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.697609  8730 raft_consensus.cc:2955] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.711903  8060 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:43.717034  8974 ts_tablet_manager.cc:1929] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:43.717516  8974 log.cc:1199] T b9bd4708595b40438af51aba853a0f94 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/wals/b9bd4708595b40438af51aba853a0f94
I20260709 13:12:43.719328  8730 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:43.720100  8137 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:43.720497  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet b9bd4708595b40438af51aba853a0f94 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:43.721253  8978 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.721902  8976 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.731487  8942 raft_consensus.cc:2955] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.737228  8730 raft_consensus.cc:2955] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.743850  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8dd7949df539418cbc00001111885883 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:43.744407  8132 raft_consensus.cc:2955] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:43.754019  7958 catalog_manager.cc:5697] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:43.770201  8198 tablet_service.cc:1558] Processing DeleteTablet for tablet 8dd7949df539418cbc00001111885883 with delete_type TABLET_DATA_TOMBSTONED (TS d9cd2999aa314b39997b423ec1ae509e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59846
I20260709 13:12:43.771692  8992 tablet_replica.cc:333] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:12:43.772401  8992 raft_consensus.cc:2243] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:43.772890  8992 raft_consensus.cc:2272] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:43.775175  8992 ts_tablet_manager.cc:1916] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:43.784829  8992 ts_tablet_manager.cc:1929] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:43.785135  8992 log.cc:1199] T 8dd7949df539418cbc00001111885883 P d9cd2999aa314b39997b423ec1ae509e: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/wals/8dd7949df539418cbc00001111885883
I20260709 13:12:43.786517  7948 catalog_manager.cc:5028] TS d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): tablet 8dd7949df539418cbc00001111885883 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:44.841756  8135 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:44.847884  8207 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:44.848210  8060 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:44.848630  8102 consensus_peers.cc:597] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f229a5eb4514abc9183538dd97073d2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:44.849025  8981 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:44.849821  8901 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:44.857016  8994 raft_consensus.cc:2955] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:44.858131  8060 raft_consensus.cc:2955] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:44.858609  8207 raft_consensus.cc:2955] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:44.873970  7958 catalog_manager.cc:5697] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:44.875885  8060 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
W20260709 13:12:44.886869  8027 consensus_peers.cc:597] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 542d60a75b684950a46a64c1d64b80a5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:44.887883  8207 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:44.889441  8941 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:44.890619  8135 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:44.891855  8941 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:44.921304  8941 raft_consensus.cc:2955] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:44.928196  8210 raft_consensus.cc:2955] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:44.936451  8135 raft_consensus.cc:2955] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:44.944021  8059 consensus_queue.cc:237] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:44.947794  7960 catalog_manager.cc:5697] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:44.954916  8207 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:44.955060  8130 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:44.955386  8027 consensus_peers.cc:597] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1dfd4edf9eab4bcc8b8aad94587349c3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:44.956395  8976 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:44.957756  8976 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:44.973178  8941 raft_consensus.cc:2955] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:44.978114  8130 raft_consensus.cc:2955] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:44.990571  8059 consensus_queue.cc:237] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:44.991390  7961 catalog_manager.cc:5697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:44.996016  8207 raft_consensus.cc:2955] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
W20260709 13:12:45.001216  8027 consensus_peers.cc:597] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a9aaa6e2a1c42a0a0db69d6406a3762. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:45.001375  8210 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:45.002581  8137 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:45.003980  8941 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:45.004674  9006 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:45.132560  9020 raft_consensus.cc:2955] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:45.147590  8137 raft_consensus.cc:2955] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:45.163726  8211 raft_consensus.cc:2955] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:45.180460  7958 catalog_manager.cc:5697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:45.259087  9021 ts_tablet_manager.cc:933] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:45.260752  9021 tablet_copy_client.cc:323] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:12:45.262158  8148 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet 1f229a5eb4514abc9183538dd97073d2 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:12:45.262588  8148 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet 1f229a5eb4514abc9183538dd97073d2 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-1f229a5eb4514abc9183538dd97073d2
I20260709 13:12:45.267256  8148 tablet_copy_source_session.cc:215] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:45.269887  9021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f229a5eb4514abc9183538dd97073d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.284960  9021 tablet_copy_client.cc:806] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:45.285509  9021 tablet_copy_client.cc:670] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:45.285543  9023 ts_tablet_manager.cc:933] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
I20260709 13:12:45.288671  9023 tablet_copy_client.cc:323] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.65:45919
I20260709 13:12:45.291236  9021 tablet_copy_client.cc:538] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:45.299219  9021 tablet_bootstrap.cc:492] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:45.300382  8073 tablet_copy_service.cc:140] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Received BeginTabletCopySession request for tablet 542d60a75b684950a46a64c1d64b80a5 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:37156)
I20260709 13:12:45.300969  8073 tablet_copy_service.cc:161] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Beginning new tablet copy session on tablet 542d60a75b684950a46a64c1d64b80a5 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:37156: session id = 965e59f272f143449b18f65db4e61d98-542d60a75b684950a46a64c1d64b80a5
I20260709 13:12:45.307616  8073 tablet_copy_source_session.cc:215] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:45.310598  9023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 542d60a75b684950a46a64c1d64b80a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.327411  9023 tablet_copy_client.cc:806] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:45.327997  9023 tablet_copy_client.cc:670] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:45.330089  9021 tablet_bootstrap.cc:492] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:45.330977  9021 tablet_bootstrap.cc:492] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:45.332058  9021 ts_tablet_manager.cc:1403] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.033s	user 0.018s	sys 0.004s
I20260709 13:12:45.332389  9023 tablet_copy_client.cc:538] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:45.334833  9021 raft_consensus.cc:359] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:45.335445  9021 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:45.335937  9021 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:45.338809  9021 ts_tablet_manager.cc:1434] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.005s
I20260709 13:12:45.340273  8148 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-1f229a5eb4514abc9183538dd97073d2 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:12:45.340673  8148 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-1f229a5eb4514abc9183538dd97073d2 on tablet 1f229a5eb4514abc9183538dd97073d2 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:45.342694  9023 tablet_bootstrap.cc:492] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:45.363413  9023 tablet_bootstrap.cc:492] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:45.364136  9023 tablet_bootstrap.cc:492] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:45.364976  9023 ts_tablet_manager.cc:1403] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.022s	user 0.004s	sys 0.013s
I20260709 13:12:45.367163  9023 raft_consensus.cc:359] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:45.367736  9023 raft_consensus.cc:740] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:45.368316  9023 consensus_queue.cc:260] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:45.369954  9023 ts_tablet_manager.cc:1434] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:12:45.372772  8073 tablet_copy_service.cc:342] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-542d60a75b684950a46a64c1d64b80a5 received from {username='slave'} at 127.0.0.1:37156
I20260709 13:12:45.373164  8073 tablet_copy_service.cc:434] P 84d775d1767e4cc4aa7b9c627d8f0fb8: ending tablet copy session 965e59f272f143449b18f65db4e61d98-542d60a75b684950a46a64c1d64b80a5 on tablet 542d60a75b684950a46a64c1d64b80a5 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:45.495381  9023 ts_tablet_manager.cc:933] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
I20260709 13:12:45.496917  9023 tablet_copy_client.cc:323] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.65:45919
I20260709 13:12:45.498139  8073 tablet_copy_service.cc:140] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Received BeginTabletCopySession request for tablet 1dfd4edf9eab4bcc8b8aad94587349c3 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:37156)
I20260709 13:12:45.498586  8073 tablet_copy_service.cc:161] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Beginning new tablet copy session on tablet 1dfd4edf9eab4bcc8b8aad94587349c3 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:37156: session id = 965e59f272f143449b18f65db4e61d98-1dfd4edf9eab4bcc8b8aad94587349c3
I20260709 13:12:45.504343  8073 tablet_copy_source_session.cc:215] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:45.507006  9023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1dfd4edf9eab4bcc8b8aad94587349c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.519136  9023 tablet_copy_client.cc:806] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:45.519598  9023 tablet_copy_client.cc:670] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:45.522617  9023 tablet_copy_client.cc:538] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:45.529711  9023 tablet_bootstrap.cc:492] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:45.548247  9023 tablet_bootstrap.cc:492] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:45.549193  9023 tablet_bootstrap.cc:492] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:45.549852  9023 ts_tablet_manager.cc:1403] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.004s
I20260709 13:12:45.552812  9023 raft_consensus.cc:359] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:45.553653  9023 raft_consensus.cc:740] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:45.555243  9023 consensus_queue.cc:260] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:45.560030  9023 ts_tablet_manager.cc:1434] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:12:45.564278  8073 tablet_copy_service.cc:342] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-1dfd4edf9eab4bcc8b8aad94587349c3 received from {username='slave'} at 127.0.0.1:37156
I20260709 13:12:45.564677  8073 tablet_copy_service.cc:434] P 84d775d1767e4cc4aa7b9c627d8f0fb8: ending tablet copy session 965e59f272f143449b18f65db4e61d98-1dfd4edf9eab4bcc8b8aad94587349c3 on tablet 1dfd4edf9eab4bcc8b8aad94587349c3 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:45.620182  9023 ts_tablet_manager.cc:933] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
I20260709 13:12:45.622664  9023 tablet_copy_client.cc:323] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.65:45919
I20260709 13:12:45.623993  8073 tablet_copy_service.cc:140] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Received BeginTabletCopySession request for tablet 7a9aaa6e2a1c42a0a0db69d6406a3762 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:37156)
I20260709 13:12:45.624490  8073 tablet_copy_service.cc:161] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Beginning new tablet copy session on tablet 7a9aaa6e2a1c42a0a0db69d6406a3762 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:37156: session id = 965e59f272f143449b18f65db4e61d98-7a9aaa6e2a1c42a0a0db69d6406a3762
I20260709 13:12:45.631685  8073 tablet_copy_source_session.cc:215] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:45.634280  9023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a9aaa6e2a1c42a0a0db69d6406a3762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.644351  9023 tablet_copy_client.cc:806] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:45.644866  9023 tablet_copy_client.cc:670] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:45.651289  9023 tablet_copy_client.cc:538] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:45.658687  9023 tablet_bootstrap.cc:492] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:45.676069  9023 tablet_bootstrap.cc:492] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:45.676961  9023 tablet_bootstrap.cc:492] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:45.677580  9023 ts_tablet_manager.cc:1403] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.004s
I20260709 13:12:45.680091  9023 raft_consensus.cc:359] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:45.680866  9023 raft_consensus.cc:740] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:45.681401  9023 consensus_queue.cc:260] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:45.683933  9023 ts_tablet_manager.cc:1434] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:12:45.685438  8073 tablet_copy_service.cc:342] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-7a9aaa6e2a1c42a0a0db69d6406a3762 received from {username='slave'} at 127.0.0.1:37156
I20260709 13:12:45.685811  8073 tablet_copy_service.cc:434] P 84d775d1767e4cc4aa7b9c627d8f0fb8: ending tablet copy session 965e59f272f143449b18f65db4e61d98-7a9aaa6e2a1c42a0a0db69d6406a3762 on tablet 7a9aaa6e2a1c42a0a0db69d6406a3762 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:45.837082  8729 raft_consensus.cc:1217] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:45.853284  8729 raft_consensus.cc:1217] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:46.069242  8729 raft_consensus.cc:1217] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:46.165225  8729 raft_consensus.cc:1217] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:46.377375  9020 raft_consensus.cc:1064] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:46.380239  9020 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:46.392758  8729 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.394095  8137 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.394325  8976 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.395490  8207 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.395792  9020 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.396947  8976 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.423221  9037 raft_consensus.cc:2955] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.424507  8729 raft_consensus.cc:2955] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.424326  8211 raft_consensus.cc:2955] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.441154  7961 catalog_manager.cc:5697] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:46.437176  8137 raft_consensus.cc:2955] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.462468  9014 raft_consensus.cc:1064] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:46.464942  9014 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:46.489640  8729 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.491413  8211 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.491520  8062 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.492667  9017 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.492695  8059 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:46.493754  8901 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.497100  8994 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.499941  8730 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.502542  9020 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:46.512529  8135 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.516471  9037 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:46.518193  9037 raft_consensus.cc:2955] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.523073  8730 raft_consensus.cc:2955] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.553072  7957 catalog_manager.cc:5697] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.564980  8135 raft_consensus.cc:2955] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.574513  8198 tablet_service.cc:1558] Processing DeleteTablet for tablet 542d60a75b684950a46a64c1d64b80a5 with delete_type TABLET_DATA_TOMBSTONED (TS d9cd2999aa314b39997b423ec1ae509e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59846
I20260709 13:12:46.581718  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 542d60a75b684950a46a64c1d64b80a5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:46.594731  8730 raft_consensus.cc:2955] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.592957  9014 raft_consensus.cc:2955] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.611204  9047 tablet_replica.cc:333] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:12:46.614362  9047 raft_consensus.cc:2243] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:46.617053  8976 raft_consensus.cc:1064] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:46.618978  7960 catalog_manager.cc:5697] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.619391  8976 consensus_queue.cc:237] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:46.615180  9047 raft_consensus.cc:2272] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:46.626899  9047 ts_tablet_manager.cc:1916] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:46.634642  8211 raft_consensus.cc:2955] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.636821  8729 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.638197  9037 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.651379  8210 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.653143  9032 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.626783  8062 raft_consensus.cc:2955] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.666175  8137 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.670130  9020 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.674079  8135 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:46.677243  9047 ts_tablet_manager.cc:1929] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:46.677698  9047 log.cc:1199] T 542d60a75b684950a46a64c1d64b80a5 P d9cd2999aa314b39997b423ec1ae509e: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/wals/542d60a75b684950a46a64c1d64b80a5
I20260709 13:12:46.679472  7948 catalog_manager.cc:5028] TS d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): tablet 542d60a75b684950a46a64c1d64b80a5 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:46.686203  9051 raft_consensus.cc:2955] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.687736  8211 raft_consensus.cc:2955] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.695089  8729 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.695721  8210 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.699683  9014 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:46.700464  9049 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:46.712922  9049 raft_consensus.cc:2955] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.713455  8137 raft_consensus.cc:2955] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.715559  8206 raft_consensus.cc:2955] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.730665  7960 catalog_manager.cc:5697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.730834  8729 raft_consensus.cc:2955] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.732249  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1f229a5eb4514abc9183538dd97073d2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:46.739593  7957 catalog_manager.cc:5697] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:46.759076  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet 1f229a5eb4514abc9183538dd97073d2 with delete_type TABLET_DATA_TOMBSTONED (TS 84d775d1767e4cc4aa7b9c627d8f0fb8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49202
I20260709 13:12:46.765242  8729 raft_consensus.cc:2955] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.784644  8062 consensus_queue.cc:237] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:46.786540  9055 tablet_replica.cc:333] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:12:46.789029  9020 raft_consensus.cc:1064] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:46.790261  8729 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.792829  8206 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.790930  9055 raft_consensus.cc:2243] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:46.791714  9020 consensus_queue.cc:237] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:46.795006  9051 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:46.795799  9037 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:46.797281  9055 raft_consensus.cc:2272] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:46.805997  9055 ts_tablet_manager.cc:1916] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:46.805855  9037 raft_consensus.cc:2955] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.808275  8730 raft_consensus.cc:2955] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.822734  8729 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.824013  9032 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.824764  7958 catalog_manager.cc:5697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.831527  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1dfd4edf9eab4bcc8b8aad94587349c3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:46.832597  8137 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.837664  9051 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.838330  8206 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:46.839032  8210 raft_consensus.cc:2955] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.844177  9055 ts_tablet_manager.cc:1929] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:46.844601  9055 log.cc:1199] T 1f229a5eb4514abc9183538dd97073d2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/wals/1f229a5eb4514abc9183538dd97073d2
I20260709 13:12:46.847087  9051 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:46.846110  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet 1f229a5eb4514abc9183538dd97073d2 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:46.857232  8123 tablet_service.cc:1558] Processing DeleteTablet for tablet 1dfd4edf9eab4bcc8b8aad94587349c3 with delete_type TABLET_DATA_TOMBSTONED (TS 8a968cada1e4446ebb2a5e167176e649 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36038
I20260709 13:12:46.877157  9051 raft_consensus.cc:2955] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.883086  9060 tablet_replica.cc:333] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:12:46.886179  9060 raft_consensus.cc:2243] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:46.887452  9060 raft_consensus.cc:2272] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:46.886886  8137 raft_consensus.cc:2955] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.891598  9060 ts_tablet_manager.cc:1916] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:46.896492  8207 raft_consensus.cc:2955] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.902168  8730 raft_consensus.cc:2955] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.918265  7961 catalog_manager.cc:5697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.928946  9060 ts_tablet_manager.cc:1929] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:46.929442  9060 log.cc:1199] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 8a968cada1e4446ebb2a5e167176e649: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/wals/1dfd4edf9eab4bcc8b8aad94587349c3
I20260709 13:12:46.931090  7945 catalog_manager.cc:5028] TS 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): tablet 1dfd4edf9eab4bcc8b8aad94587349c3 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:46.942322  8059 consensus_queue.cc:237] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:46.948784  8730 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.948742  8206 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:46.950286  9037 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:46.952044  9032 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:46.957122  9020 raft_consensus.cc:2955] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.963003  8206 raft_consensus.cc:2955] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.966296  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7a9aaa6e2a1c42a0a0db69d6406a3762 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:46.972186  8730 raft_consensus.cc:2955] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:46.975677  7958 catalog_manager.cc:5697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:46.992890  8123 tablet_service.cc:1558] Processing DeleteTablet for tablet 7a9aaa6e2a1c42a0a0db69d6406a3762 with delete_type TABLET_DATA_TOMBSTONED (TS 8a968cada1e4446ebb2a5e167176e649 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36038
I20260709 13:12:46.993749  9060 tablet_replica.cc:333] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:12:46.994688  9060 raft_consensus.cc:2243] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:46.995285  9060 raft_consensus.cc:2272] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:46.999182  9060 ts_tablet_manager.cc:1916] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:47.012913  9060 ts_tablet_manager.cc:1929] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:47.013279  9060 log.cc:1199] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 8a968cada1e4446ebb2a5e167176e649: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/wals/7a9aaa6e2a1c42a0a0db69d6406a3762
I20260709 13:12:47.014624  7945 catalog_manager.cc:5028] TS 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): tablet 7a9aaa6e2a1c42a0a0db69d6406a3762 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:48.062800  8137 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:48.067773  8210 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:48.068385  8102 consensus_peers.cc:597] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f4539f806feb4e37900025e737458841. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:48.068440  8059 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:48.068825  9049 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.069661  9054 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.078099  8210 raft_consensus.cc:2955] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.079521  8059 raft_consensus.cc:2955] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.076579  9054 raft_consensus.cc:2955] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.094295  7960 catalog_manager.cc:5697] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.100109  8059 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:48.107609  8210 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:48.109949  8027 consensus_peers.cc:597] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf628848d7464a21aa8f76f7eb56a408. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:48.110041  8137 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:48.111635  9051 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.112583  9020 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.123759  9020 raft_consensus.cc:2955] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.127561  8137 raft_consensus.cc:2955] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.138252  8210 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:48.146186  8132 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:48.146188  8176 consensus_peers.cc:597] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1c85566831fd4f4283bc20a68e5ff448. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:48.147928  9033 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.156255  8210 raft_consensus.cc:2955] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.158901  7957 catalog_manager.cc:5697] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.166033  9075 raft_consensus.cc:2955] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.167879  8137 raft_consensus.cc:2955] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.173550  8062 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:48.185992  8132 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:48.188395  9061 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.188467  7957 catalog_manager.cc:5697] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.199273  8059 raft_consensus.cc:2955] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.206708  8207 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:48.207440  8102 consensus_peers.cc:597] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ca98b6f039a45fb80c6e78ff2a0ae32. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:48.208731  8060 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:48.209506  9049 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.213459  9054 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.218955  8901 raft_consensus.cc:2955] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.225319  8207 raft_consensus.cc:2955] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.229499  8060 raft_consensus.cc:2955] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:48.240934  7960 catalog_manager.cc:5697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:48.544865  9082 ts_tablet_manager.cc:933] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:48.546886  9082 tablet_copy_client.cc:323] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:12:48.548372  8148 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet f4539f806feb4e37900025e737458841 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:12:48.548882  8148 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet f4539f806feb4e37900025e737458841 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-f4539f806feb4e37900025e737458841
I20260709 13:12:48.553565  8148 tablet_copy_source_session.cc:215] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:48.556272  9082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4539f806feb4e37900025e737458841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:48.567739  9082 tablet_copy_client.cc:806] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:48.568224  9082 tablet_copy_client.cc:670] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:48.571863  9082 tablet_copy_client.cc:538] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:48.578855  9082 tablet_bootstrap.cc:492] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:48.604254  9084 ts_tablet_manager.cc:933] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
I20260709 13:12:48.605713  9082 tablet_bootstrap.cc:492] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:48.606704  9082 tablet_bootstrap.cc:492] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:48.607414  9082 ts_tablet_manager.cc:1403] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.016s
I20260709 13:12:48.610291  9082 raft_consensus.cc:359] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:48.611159  9082 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:48.611848  9082 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:48.614462  9087 ts_tablet_manager.cc:933] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:48.616371  9084 tablet_copy_client.cc:323] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.65:45919
I20260709 13:12:48.617962  8073 tablet_copy_service.cc:140] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Received BeginTabletCopySession request for tablet bf628848d7464a21aa8f76f7eb56a408 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:37156)
I20260709 13:12:48.618465  8073 tablet_copy_service.cc:161] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Beginning new tablet copy session on tablet bf628848d7464a21aa8f76f7eb56a408 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:37156: session id = 965e59f272f143449b18f65db4e61d98-bf628848d7464a21aa8f76f7eb56a408
I20260709 13:12:48.621330  9082 ts_tablet_manager.cc:1434] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.001s
I20260709 13:12:48.622978  8148 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-f4539f806feb4e37900025e737458841 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:12:48.623366  8148 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-f4539f806feb4e37900025e737458841 on tablet f4539f806feb4e37900025e737458841 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:48.628664  8073 tablet_copy_source_session.cc:215] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:48.632619  9084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf628848d7464a21aa8f76f7eb56a408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:48.639282  9087 tablet_copy_client.cc:323] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:12:48.640770  8148 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet 3ca98b6f039a45fb80c6e78ff2a0ae32 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:12:48.641270  8148 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet 3ca98b6f039a45fb80c6e78ff2a0ae32 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-3ca98b6f039a45fb80c6e78ff2a0ae32
I20260709 13:12:48.648991  8148 tablet_copy_source_session.cc:215] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:48.650501  9084 tablet_copy_client.cc:806] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:48.651886  9087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ca98b6f039a45fb80c6e78ff2a0ae32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:48.655620  9084 tablet_copy_client.cc:670] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:48.658669  9084 tablet_copy_client.cc:538] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:48.666621  9084 tablet_bootstrap.cc:492] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:48.666687  9087 tablet_copy_client.cc:806] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:48.668398  9087 tablet_copy_client.cc:670] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:48.673004  9087 tablet_copy_client.cc:538] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:48.702040  9084 tablet_bootstrap.cc:492] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:48.703071  9084 tablet_bootstrap.cc:492] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:48.703841  9084 ts_tablet_manager.cc:1403] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.037s	user 0.020s	sys 0.008s
I20260709 13:12:48.707063  9084 raft_consensus.cc:359] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:48.707906  9084 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:48.708561  9084 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:48.713281  9084 ts_tablet_manager.cc:1434] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.005s
I20260709 13:12:48.714754  9082 ts_tablet_manager.cc:933] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:48.714888  8073 tablet_copy_service.cc:342] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-bf628848d7464a21aa8f76f7eb56a408 received from {username='slave'} at 127.0.0.1:37156
I20260709 13:12:48.715278  8073 tablet_copy_service.cc:434] P 84d775d1767e4cc4aa7b9c627d8f0fb8: ending tablet copy session 965e59f272f143449b18f65db4e61d98-bf628848d7464a21aa8f76f7eb56a408 on tablet bf628848d7464a21aa8f76f7eb56a408 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:48.719659  9087 tablet_bootstrap.cc:492] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:48.726771  9082 tablet_copy_client.cc:323] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.67:35745
I20260709 13:12:48.728730  8222 tablet_copy_service.cc:140] P d9cd2999aa314b39997b423ec1ae509e: Received BeginTabletCopySession request for tablet 1c85566831fd4f4283bc20a68e5ff448 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:55942)
I20260709 13:12:48.729246  8222 tablet_copy_service.cc:161] P d9cd2999aa314b39997b423ec1ae509e: Beginning new tablet copy session on tablet 1c85566831fd4f4283bc20a68e5ff448 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:55942: session id = 965e59f272f143449b18f65db4e61d98-1c85566831fd4f4283bc20a68e5ff448
I20260709 13:12:48.744983  8222 tablet_copy_source_session.cc:215] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:48.748426  9082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c85566831fd4f4283bc20a68e5ff448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:48.753315  9087 tablet_bootstrap.cc:492] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:48.754199  9087 tablet_bootstrap.cc:492] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:48.754884  9087 ts_tablet_manager.cc:1403] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.015s
I20260709 13:12:48.758693  9087 raft_consensus.cc:359] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:48.759339  9087 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:48.759891  9087 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:48.763907  9087 ts_tablet_manager.cc:1434] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.009s	user 0.001s	sys 0.004s
I20260709 13:12:48.765456  8148 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-3ca98b6f039a45fb80c6e78ff2a0ae32 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:12:48.765842  8148 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-3ca98b6f039a45fb80c6e78ff2a0ae32 on tablet 3ca98b6f039a45fb80c6e78ff2a0ae32 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:48.777812  9082 tablet_copy_client.cc:806] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:48.778378  9082 tablet_copy_client.cc:670] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:48.782361  9082 tablet_copy_client.cc:538] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:48.787787  9082 tablet_bootstrap.cc:492] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:48.806089  9082 tablet_bootstrap.cc:492] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:48.806964  9082 tablet_bootstrap.cc:492] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:48.807557  9082 ts_tablet_manager.cc:1403] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.020s	user 0.019s	sys 0.000s
I20260709 13:12:48.810196  9082 raft_consensus.cc:359] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:48.811044  9082 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:48.811621  9082 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:48.813973  9082 ts_tablet_manager.cc:1434] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:12:48.817392  8222 tablet_copy_service.cc:342] P d9cd2999aa314b39997b423ec1ae509e: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-1c85566831fd4f4283bc20a68e5ff448 received from {username='slave'} at 127.0.0.1:55942
I20260709 13:12:48.817795  8222 tablet_copy_service.cc:434] P d9cd2999aa314b39997b423ec1ae509e: ending tablet copy session 965e59f272f143449b18f65db4e61d98-1c85566831fd4f4283bc20a68e5ff448 on tablet 1c85566831fd4f4283bc20a68e5ff448 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:48.935216  8729 raft_consensus.cc:1217] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:49.096961  8729 raft_consensus.cc:1217] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:49.111526  8729 raft_consensus.cc:1217] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:49.285938  8730 raft_consensus.cc:1217] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:49.374884  8901 raft_consensus.cc:1064] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:49.377043  8901 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:49.386534  8211 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:49.388666  8059 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:49.390219  8730 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:49.390043  9054 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:49.391605  8901 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:49.395843  9049 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:49.402841  8901 raft_consensus.cc:2955] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.420300  8730 raft_consensus.cc:2955] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.428551  8211 raft_consensus.cc:2955] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.429092  8059 raft_consensus.cc:2955] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.433915  7957 catalog_manager.cc:5697] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:49.476526  8137 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:49.483880  8730 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:49.484009  8211 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:49.485167  8901 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:49.485978  9054 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:49.491577  9054 raft_consensus.cc:2955] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.498061  8211 raft_consensus.cc:2955] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.501609  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f4539f806feb4e37900025e737458841 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:49.503504  8730 raft_consensus.cc:2955] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.509745  7958 catalog_manager.cc:5697] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:49.518811  8901 raft_consensus.cc:1064] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:49.520849  8901 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:49.536960  8211 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:49.537674  8059 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:49.539438  8901 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:49.540043  8730 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:49.540998  8901 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:49.541270  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet f4539f806feb4e37900025e737458841 with delete_type TABLET_DATA_TOMBSTONED (TS 84d775d1767e4cc4aa7b9c627d8f0fb8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49202
I20260709 13:12:49.542007  9049 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:49.580276  8901 raft_consensus.cc:2955] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.587168  8211 raft_consensus.cc:2955] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.594795  8060 raft_consensus.cc:2955] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.596832  9106 tablet_replica.cc:333] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:12:49.596388  7958 catalog_manager.cc:5697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:49.599349  9106 raft_consensus.cc:2243] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:49.600001  9106 raft_consensus.cc:2272] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:49.603348  9106 ts_tablet_manager.cc:1916] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:49.608660  8730 raft_consensus.cc:2955] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.623914  8137 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:49.625470  9106 ts_tablet_manager.cc:1929] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:49.625937  9106 log.cc:1199] T f4539f806feb4e37900025e737458841 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/wals/f4539f806feb4e37900025e737458841
I20260709 13:12:49.629987  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet f4539f806feb4e37900025e737458841 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:49.637683  8060 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:49.637221  8730 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:49.639873  9049 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:49.641422  8901 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:49.646286  9049 raft_consensus.cc:2955] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.650911  8060 raft_consensus.cc:2955] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.657976  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3ca98b6f039a45fb80c6e78ff2a0ae32 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:49.663057  7961 catalog_manager.cc:5697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:49.670605  8730 raft_consensus.cc:2955] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.675087  8198 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ca98b6f039a45fb80c6e78ff2a0ae32 with delete_type TABLET_DATA_TOMBSTONED (TS d9cd2999aa314b39997b423ec1ae509e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59846
I20260709 13:12:49.691867  9110 tablet_replica.cc:333] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:12:49.692782  9110 raft_consensus.cc:2243] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:49.693585  9110 raft_consensus.cc:2272] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:49.697189  9110 ts_tablet_manager.cc:1916] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:49.711406  9110 ts_tablet_manager.cc:1929] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:49.711777  9110 log.cc:1199] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P d9cd2999aa314b39997b423ec1ae509e: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/wals/3ca98b6f039a45fb80c6e78ff2a0ae32
I20260709 13:12:49.713371  7948 catalog_manager.cc:5028] TS d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): tablet 3ca98b6f039a45fb80c6e78ff2a0ae32 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:49.726548  9020 raft_consensus.cc:1064] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:49.728233  9020 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:49.734557  8210 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:12:49.781937  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 45949.2us
I20260709 13:12:49.788122  9095 raft_consensus.cc:1064] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:49.791837  9095 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:49.793356  8730 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:49.782930  9112 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:49.817227  9020 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:49.827277  8137 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:49.828008  8728 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:49.828809  9095 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:49.830744  9115 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:49.831439  8135 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:49.839932  8060 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:49.851907  9075 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:49.856673  9113 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:49.874080  9113 raft_consensus.cc:2955] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.875902  9075 raft_consensus.cc:2955] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.881109  8130 raft_consensus.cc:2955] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.884954  8137 raft_consensus.cc:2955] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.896307  8730 raft_consensus.cc:2955] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.900861  8728 raft_consensus.cc:2955] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.904696  8211 raft_consensus.cc:2955] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.917142  8060 raft_consensus.cc:2955] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.913664  7961 catalog_manager.cc:5697] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:49.942858  8060 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:49.950312  7961 catalog_manager.cc:5697] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:49.967924  8730 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:49.979372  9113 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:49.982018  8137 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:49.983366  9113 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:49.988837  9116 raft_consensus.cc:2955] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.991911  8135 raft_consensus.cc:2955] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:49.995195  8730 raft_consensus.cc:2955] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:50.040078  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bf628848d7464a21aa8f76f7eb56a408 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:50.051294  8207 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:50.091248  7961 catalog_manager.cc:5697] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 3 to 4, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:50.102603  8730 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:50.106606  9120 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:12:50.122581  8060 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:50.124150  9120 raft_consensus.cc:2955] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:50.134620  9095 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:50.146313  8198 tablet_service.cc:1558] Processing DeleteTablet for tablet bf628848d7464a21aa8f76f7eb56a408 with delete_type TABLET_DATA_TOMBSTONED (TS d9cd2999aa314b39997b423ec1ae509e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59846
I20260709 13:12:50.148361  9110 tablet_replica.cc:333] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:12:50.146414  8730 raft_consensus.cc:2955] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:50.153007  8060 raft_consensus.cc:2955] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:50.177692  7948 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1c85566831fd4f4283bc20a68e5ff448 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:50.185398  9110 raft_consensus.cc:2243] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:50.186319  9110 raft_consensus.cc:2272] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:50.185544  7957 catalog_manager.cc:5697] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:50.196563  9110 ts_tablet_manager.cc:1916] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:50.218250  9110 ts_tablet_manager.cc:1929] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:50.218837  9110 log.cc:1199] T bf628848d7464a21aa8f76f7eb56a408 P d9cd2999aa314b39997b423ec1ae509e: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/wals/bf628848d7464a21aa8f76f7eb56a408
I20260709 13:12:50.219641  8123 tablet_service.cc:1558] Processing DeleteTablet for tablet 1c85566831fd4f4283bc20a68e5ff448 with delete_type TABLET_DATA_TOMBSTONED (TS 8a968cada1e4446ebb2a5e167176e649 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36038
I20260709 13:12:50.222023  7948 catalog_manager.cc:5028] TS d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): tablet bf628848d7464a21aa8f76f7eb56a408 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:50.224299  9130 tablet_replica.cc:333] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:12:50.226327  9130 raft_consensus.cc:2243] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:50.227510  9130 raft_consensus.cc:2272] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:50.233457  9130 ts_tablet_manager.cc:1916] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:50.251451  9130 ts_tablet_manager.cc:1929] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:50.251809  9130 log.cc:1199] T 1c85566831fd4f4283bc20a68e5ff448 P 8a968cada1e4446ebb2a5e167176e649: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/wals/1c85566831fd4f4283bc20a68e5ff448
I20260709 13:12:50.253980  7945 catalog_manager.cc:5028] TS 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): tablet 1c85566831fd4f4283bc20a68e5ff448 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:51.294288  8207 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:51.300844  8137 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:51.302455  9120 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:51.313400  8176 consensus_peers.cc:597] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7c4e1687a1db419c8863789bb37c260a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:51.315869  8060 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:51.318356  9095 raft_consensus.cc:2955] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:51.318797  9135 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:51.320686  8137 raft_consensus.cc:2955] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:51.325655  8060 raft_consensus.cc:2955] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:51.331776  8130 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:51.338629  7957 catalog_manager.cc:5697] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:51.342304  8211 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:51.344974  8060 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:51.348954  9131 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:12:51.351092  8102 consensus_peers.cc:597] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5d0e2cd74dac4358aba60edf5382befd. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:51.351452  9126 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:51.357823  9126 raft_consensus.cc:2955] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:51.360050  8207 raft_consensus.cc:2955] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:51.375609  8060 raft_consensus.cc:2955] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:51.376363  8137 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:51.390856  7957 catalog_manager.cc:5697] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:51.397766  8060 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:51.399441  9131 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:51.402199  8208 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:12:51.403470  8102 consensus_peers.cc:597] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5124ddc96f984ec2ae0ee97be6732a90. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:51.409327  9131 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:51.410729  9054 raft_consensus.cc:2955] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:51.412149  8060 raft_consensus.cc:2955] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:51.425277  8062 consensus_queue.cc:237] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:51.437255  8207 raft_consensus.cc:2955] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:51.441561  7960 catalog_manager.cc:5697] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:51.461076  8137 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:51.466764  9132 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:51.483886  8208 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:51.485562  9132 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:51.492739  8027 consensus_peers.cc:597] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b8fabe9526434f818c106b17c57b3b85. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:51.558174  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 70933.9us
I20260709 13:12:51.572897  9132 raft_consensus.cc:2955] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:51.575980  8137 raft_consensus.cc:2955] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:51.623363  8208 raft_consensus.cc:2955] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:51.627215  7961 catalog_manager.cc:5697] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:51.724537  9157 ts_tablet_manager.cc:933] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:51.726541  9157 tablet_copy_client.cc:323] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.67:35745
I20260709 13:12:51.729630  8222 tablet_copy_service.cc:140] P d9cd2999aa314b39997b423ec1ae509e: Received BeginTabletCopySession request for tablet 7c4e1687a1db419c8863789bb37c260a from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:55942)
I20260709 13:12:51.730160  8222 tablet_copy_service.cc:161] P d9cd2999aa314b39997b423ec1ae509e: Beginning new tablet copy session on tablet 7c4e1687a1db419c8863789bb37c260a from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:55942: session id = 965e59f272f143449b18f65db4e61d98-7c4e1687a1db419c8863789bb37c260a
I20260709 13:12:51.731572  9159 ts_tablet_manager.cc:933] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:51.733582  9159 tablet_copy_client.cc:323] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:12:51.735447  8148 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet 5d0e2cd74dac4358aba60edf5382befd from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:12:51.735950  8148 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet 5d0e2cd74dac4358aba60edf5382befd from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-5d0e2cd74dac4358aba60edf5382befd
I20260709 13:12:51.737130  8222 tablet_copy_source_session.cc:215] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:51.740684  9157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c4e1687a1db419c8863789bb37c260a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:51.743844  8148 tablet_copy_source_session.cc:215] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:51.748246  9159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0e2cd74dac4358aba60edf5382befd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:51.755940  9157 tablet_copy_client.cc:806] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:51.756551  9157 tablet_copy_client.cc:670] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:51.760224  9157 tablet_copy_client.cc:538] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:51.768420  9157 tablet_bootstrap.cc:492] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:51.770201  9159 tablet_copy_client.cc:806] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:51.770813  9159 tablet_copy_client.cc:670] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:51.779989  9159 tablet_copy_client.cc:538] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:51.791177  9159 tablet_bootstrap.cc:492] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:51.807473  9157 tablet_bootstrap.cc:492] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:51.808351  9157 tablet_bootstrap.cc:492] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:51.809324  9157 ts_tablet_manager.cc:1403] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.041s	user 0.022s	sys 0.012s
I20260709 13:12:51.811964  9157 raft_consensus.cc:359] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:51.812778  9157 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:51.813537  9157 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:51.814864  9159 tablet_bootstrap.cc:492] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:51.815775  9159 tablet_bootstrap.cc:492] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:51.816426  9159 ts_tablet_manager.cc:1403] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.008s
I20260709 13:12:51.819707  9159 raft_consensus.cc:359] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:51.820546  9159 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:51.820600  9157 ts_tablet_manager.cc:1434] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.003s
I20260709 13:12:51.821282  9159 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:51.822965  8222 tablet_copy_service.cc:342] P d9cd2999aa314b39997b423ec1ae509e: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-7c4e1687a1db419c8863789bb37c260a received from {username='slave'} at 127.0.0.1:55942
I20260709 13:12:51.823434  8222 tablet_copy_service.cc:434] P d9cd2999aa314b39997b423ec1ae509e: ending tablet copy session 965e59f272f143449b18f65db4e61d98-7c4e1687a1db419c8863789bb37c260a on tablet 7c4e1687a1db419c8863789bb37c260a with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:51.823890  9159 ts_tablet_manager.cc:1434] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.004s
I20260709 13:12:51.825723  8148 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-5d0e2cd74dac4358aba60edf5382befd received from {username='slave'} at 127.0.0.1:46944
I20260709 13:12:51.826129  8148 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-5d0e2cd74dac4358aba60edf5382befd on tablet 5d0e2cd74dac4358aba60edf5382befd with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:51.840766  9157 ts_tablet_manager.cc:933] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
I20260709 13:12:51.844208  9157 tablet_copy_client.cc:323] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.65:45919
I20260709 13:12:51.846171  8073 tablet_copy_service.cc:140] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Received BeginTabletCopySession request for tablet b8fabe9526434f818c106b17c57b3b85 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:37156)
I20260709 13:12:51.846702  8073 tablet_copy_service.cc:161] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Beginning new tablet copy session on tablet b8fabe9526434f818c106b17c57b3b85 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:37156: session id = 965e59f272f143449b18f65db4e61d98-b8fabe9526434f818c106b17c57b3b85
I20260709 13:12:51.854156  8073 tablet_copy_source_session.cc:215] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:51.857331  9157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8fabe9526434f818c106b17c57b3b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:51.873198  9157 tablet_copy_client.cc:806] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:51.873785  9157 tablet_copy_client.cc:670] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:51.877456  9157 tablet_copy_client.cc:538] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:51.885782  9157 tablet_bootstrap.cc:492] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:51.909699  9157 tablet_bootstrap.cc:492] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:51.910653  9157 tablet_bootstrap.cc:492] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:51.911799  9157 ts_tablet_manager.cc:1403] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.011s
I20260709 13:12:51.914613  9157 raft_consensus.cc:359] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:51.915462  9157 raft_consensus.cc:740] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:51.916074  9157 consensus_queue.cc:260] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:51.918320  9157 ts_tablet_manager.cc:1434] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:12:51.920037  8073 tablet_copy_service.cc:342] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-b8fabe9526434f818c106b17c57b3b85 received from {username='slave'} at 127.0.0.1:37156
I20260709 13:12:51.920441  8073 tablet_copy_service.cc:434] P 84d775d1767e4cc4aa7b9c627d8f0fb8: ending tablet copy session 965e59f272f143449b18f65db4e61d98-b8fabe9526434f818c106b17c57b3b85 on tablet b8fabe9526434f818c106b17c57b3b85 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:51.968194  9157 ts_tablet_manager.cc:933] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:51.970958  9157 tablet_copy_client.cc:323] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:12:51.972332  8148 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet 5124ddc96f984ec2ae0ee97be6732a90 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:12:51.972836  8148 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet 5124ddc96f984ec2ae0ee97be6732a90 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-5124ddc96f984ec2ae0ee97be6732a90
I20260709 13:12:51.980051  8148 tablet_copy_source_session.cc:215] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:51.982831  9157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5124ddc96f984ec2ae0ee97be6732a90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:51.997011  9157 tablet_copy_client.cc:806] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:51.997539  9157 tablet_copy_client.cc:670] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:52.002624  9157 tablet_copy_client.cc:538] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:52.007678  9157 tablet_bootstrap.cc:492] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:52.026787  9157 tablet_bootstrap.cc:492] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:52.027406  9157 tablet_bootstrap.cc:492] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:52.028015  9157 ts_tablet_manager.cc:1403] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.003s
I20260709 13:12:52.030323  9157 raft_consensus.cc:359] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:52.030889  9157 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:52.031422  9157 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:52.033304  9157 ts_tablet_manager.cc:1434] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.004s
I20260709 13:12:52.035488  8148 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-5124ddc96f984ec2ae0ee97be6732a90 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:12:52.035868  8148 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-5124ddc96f984ec2ae0ee97be6732a90 on tablet 5124ddc96f984ec2ae0ee97be6732a90 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:12:52.252105  8728 raft_consensus.cc:1217] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:52.266239  8728 raft_consensus.cc:1217] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:52.288918  8728 raft_consensus.cc:1217] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:52.349071  8728 raft_consensus.cc:1217] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:12:53.334227  9173 raft_consensus.cc:1064] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:53.337598  9173 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:53.518057  9172 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:53.519003  9172 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.521811  9172 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.585922  9172 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:53.586494  9172 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.589475  9172 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.598512  9134 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.600531  9134 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.603107  9134 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:53.621589  9177 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.622110  9177 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:53.625272  9177 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:53.626052  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:12:53.626758  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 296492us
I20260709 13:12:53.630914  9175 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.631528  9175 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.633818  9175 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.634231  9175 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.667771  9175 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.670162  9175 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.615119  9172 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:53.686815  9172 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:53.699514  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:53.701894  9181 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.701918  8058 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:53.702435  9181 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.689477  9179 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.704602  9179 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.706106  9172 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:12:53.697359  9168 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.706755  9168 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:53.707867  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 77638.6us
I20260709 13:12:53.718019  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:53.720477  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:53.721562  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:53.724145  9177 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.724696  9177 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:12:53.726527  8103 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.100s	user 0.020s	sys 0.022s
I20260709 13:12:53.726948  9177 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:12:53.727963  8103 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.102s	user 0.021s	sys 0.022s
I20260709 13:12:53.728032  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:53.732741  9129 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:53.733271  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.733273  9129 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:53.734881  9179 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:53.735977  9181 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.736526  9129 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:12:53.727151  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.738430  9170 raft_consensus.cc:1064] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:53.742991  9170 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:53.744555  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:12:53.744917  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:12:53.745424  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.748917  9126 raft_consensus.cc:1064] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:53.749145  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:12:53.750453  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.752197  9178 raft_consensus.cc:1064] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:12:53.752096  9173 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:53.752763  9173 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.738035  9168 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:53.754021  9172 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:53.754763  9173 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:53.754591  9172 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.757717  9172 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.754501  9178 consensus_queue.cc:237] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:53.751802  9126 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:53.758737  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.760351  8137 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:53.761309  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.762063  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.768647  9180 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:53.770125  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:53.770939  9177 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:53.771978  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:53.772688  9177 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:53.774761  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.775628  9172 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:53.776135  9172 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.778609  9172 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:12:53.779542  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.779500  8727 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:53.780978  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:53.785316  9131 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.782919  9175 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.785836  9131 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.788333  9175 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:53.788977  8727 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:53.790558  9132 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:53.791122  9175 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:12:53.791666  9168 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:53.791801  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:12:53.792454  9131 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.792852  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:53.792151  9168 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.797595  9132 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:53.800036  9153 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.800494  9132 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.800755  9153 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.802327  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.803525  8727 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:53.804003  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.804363  9132 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.804594  9153 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.806617  9168 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:53.806658  9153 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.807260  9153 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:53.809638  9153 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:12:53.810676  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:12:53.811406  9126 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:53.812270  9132 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:53.813932  9132 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.815496  9156 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:12:53.815861  9132 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.816025  9129 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:53.815984  9156 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:53.816677  9129 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.819228  9129 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.822815  9156 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.823139  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:53.821484  8101 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:53.828826  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.829437  8132 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:53.830366  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.826680  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:53.830927  8137 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:53.831307  9173 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:53.833133  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.833824  9173 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:53.833190  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:53.835968  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:53.836243  8208 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:53.836464  9173 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:12:53.836752  9181 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.839092  8101 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:53.840184  8727 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:53.843261  9180 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:12:53.847039  9180 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.842732  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.847895  9180 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.841490  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:53.847889  9131 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:53.841730  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.849737  9155 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:53.842803  9156 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.850438  9155 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.850528  9156 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.850952  9155 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.852052  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:53.852685  9156 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.840461  9175 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:53.852873  9155 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.841755  9153 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.872738  9153 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:53.874720  9153 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:12:53.852797  9131 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:53.875294  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.875720  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.876948  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.878736  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.879065  8210 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:53.840180  9173 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:12:53.879451  9180 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.879882  9173 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:53.877831  9184 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:12:53.881547  9180 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:53.840404  9181 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.854765  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.855444  9156 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:53.858062  8208 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:53.858459  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:53.886955  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.859416  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.860280  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.889772  9181 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.893007  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:12:53.893401  9173 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.896001  9173 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:53.897028  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.902200  9181 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:53.910123  9173 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:12:53.910231  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 51877.1us
I20260709 13:12:53.911649  9179 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:53.911904  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:53.912549  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:53.913475  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:53.913753  9179 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:53.913980  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:53.915103  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:53.915630  8135 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:53.916867  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:53.916168  9172 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:53.926643  9180 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:12:53.930089  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.932794  9178 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:53.932701  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.933308  9178 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.936214  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:12:53.941658  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:53.938483  9178 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.941712  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.937839  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.937986  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.941370  9131 raft_consensus.cc:2955] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:53.944289  9180 raft_consensus.cc:2955] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:53.954856  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.936264  8135 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:12:53.958174  8132 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:53.960098  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.962626  8137 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:53.962576  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:53.963249  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:53.963958  9173 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:53.964442  9173 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:53.950799  8727 raft_consensus.cc:2955] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:53.966868  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:53.971657  9173 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.969295  9153 raft_consensus.cc:2955] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:53.971403  8209 raft_consensus.cc:2955] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:53.977010  8208 raft_consensus.cc:2955] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:53.981662  9178 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:53.988891  9178 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:53.990788  9178 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:53.989909  9132 raft_consensus.cc:2955] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:53.980432  8057 raft_consensus.cc:2955] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:53.999954  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:54.007248  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:53.997254  8212 raft_consensus.cc:2955] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.010746  7961 catalog_manager.cc:5697] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:54.008247  8026 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.017159  9172 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.020777  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:54.022775  8059 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:54.023370  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.023976  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:54.024106  9129 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.025470  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:54.027431  8061 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:12:54.027690  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:53.995256  8133 raft_consensus.cc:2955] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.031329  8026 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:12:54.032202  9129 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.033456  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:54.027426  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:54.038524  8727 raft_consensus.cc:2955] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.041253  8026 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.042313  9178 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.043244  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:54.044535  8026 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.045835  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:54.050791  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:54.057044  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:12:54.061928  8026 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.075453  9172 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.078213  9171 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:12:54.081768  9193 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:12:54.082744  9169 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.096222  8729 raft_consensus.cc:2955] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.098799  7958 catalog_manager.cc:5697] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:54.104883  7960 catalog_manager.cc:5697] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 2, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:54.107844  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45992.6us
I20260709 13:12:54.111061  8727 raft_consensus.cc:2955] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.120913  9129 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:54.121450  8026 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.109006  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:12:54.125031  9129 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:54.125371  9129 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:54.126049  9129 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:54.131449  9129 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.132414  8026 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.133462  8101 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.132392  9178 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.141186  9169 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.141284  9199 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
W20260709 13:12:54.141207  8751 debug-util.cc:398] Leaking SignalData structure 0x7b0800248b80 after lost signal to thread 6940
I20260709 13:12:54.142519  8026 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:12:54.143945  8751 debug-util.cc:398] Leaking SignalData structure 0x7b08001a7340 after lost signal to thread 7995
I20260709 13:12:54.143599  9196 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.160440  9177 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:54.160982  9177 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:12:54.162637  8751 debug-util.cc:398] Leaking SignalData structure 0x7b0800311b80 after lost signal to thread 8087
I20260709 13:12:54.163198  9177 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:54.165107  9201 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.165958  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:54.170416  9134 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:54.170907  9134 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:54.171411  9180 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:12:54.171892  9180 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:54.172825  9134 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:12:54.164289  8751 debug-util.cc:398] Leaking SignalData structure 0x7b08002f5700 after lost signal to thread 8162
I20260709 13:12:54.173626  9180 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:54.164516  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:12:54.175091  8696 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.176311  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:54.176263  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:54.177479  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:54.177611  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:54.178748  9177 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.179159  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:54.180294  9126 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:12:54.180410  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 47052.8us
I20260709 13:12:54.180739  9134 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.181406  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:12:54.183113  9120 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:12:54.179872  8751 debug-util.cc:398] Leaking SignalData structure 0x7b08002f66a0 after lost signal to thread 8236
I20260709 13:12:54.190173  8208 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:54.191618  9196 consensus_queue.cc:1059] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:54.192634  9196 raft_consensus.cc:3055] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:54.195842  9196 raft_consensus.cc:740] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:54.196650  9196 consensus_queue.cc:260] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:54.190301  8751 debug-util.cc:398] Leaking SignalData structure 0x7b080030c240 after lost signal to thread 8754
I20260709 13:12:54.204101  9196 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.206278  9131 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:54.206408  8101 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.208446  9201 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.213172  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:12:54.213847  9201 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.215297  9201 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:54.215488  8101 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.218215  9131 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.221333  8101 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.222640  9201 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.223850  8101 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.258471  9201 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.259173  8101 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.259917  9201 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.260471  7958 catalog_manager.cc:5697] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.261929  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:12:54.262535  9201 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.262140  7958 catalog_manager.cc:5697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.264122  8026 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.264714  9196 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:54.278975  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:54.310189  8209 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:54.319777  8727 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:54.321462  9120 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:54.328434  8132 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:54.332180  9134 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:54.341920  8132 raft_consensus.cc:2955] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.341773  8059 raft_consensus.cc:2955] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.346638  9180 raft_consensus.cc:2955] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.352690  8727 raft_consensus.cc:2955] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.354440  8130 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:54.360044  8729 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:54.361116  8061 consensus_queue.cc:237] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:54.361337  9131 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:54.364593  7948 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7c4e1687a1db419c8863789bb37c260a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:54.366808  8730 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:54.368304  9129 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:54.374910  8209 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:54.374476  9182 raft_consensus.cc:2955] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.375058  9129 raft_consensus.cc:2955] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.375340  7960 catalog_manager.cc:5697] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New cstate: current_term: 1 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:54.376300  9153 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:54.377856  8730 raft_consensus.cc:2955] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.380111  8208 raft_consensus.cc:2955] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.387639  8060 raft_consensus.cc:2955] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.381197  8729 raft_consensus.cc:2955] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.383749  8212 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:12:54.391729  9196 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:54.395452  8212 raft_consensus.cc:2955] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.405234  7958 catalog_manager.cc:5697] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 3 to 4, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:54.413113  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet 7c4e1687a1db419c8863789bb37c260a with delete_type TABLET_DATA_TOMBSTONED (TS 84d775d1767e4cc4aa7b9c627d8f0fb8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49202
I20260709 13:12:54.417948  8137 raft_consensus.cc:2955] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.431429  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5124ddc96f984ec2ae0ee97be6732a90 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:54.446527  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b8fabe9526434f818c106b17c57b3b85 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:54.446895  8060 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:54.448199  9120 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:54.448772  9209 tablet_replica.cc:333] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:12:54.449786  9209 raft_consensus.cc:2243] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:54.456542  9209 raft_consensus.cc:2272] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:54.460934  9209 ts_tablet_manager.cc:1916] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:54.471768  7961 catalog_manager.cc:5697] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 3 to 4, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.473151  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet 5124ddc96f984ec2ae0ee97be6732a90 with delete_type TABLET_DATA_TOMBSTONED (TS 84d775d1767e4cc4aa7b9c627d8f0fb8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:49202
I20260709 13:12:54.493287  8130 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:54.496381  9120 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:54.516598  9209 ts_tablet_manager.cc:1929] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:54.529276  9209 log.cc:1199] T 7c4e1687a1db419c8863789bb37c260a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/wals/7c4e1687a1db419c8863789bb37c260a
I20260709 13:12:54.531100  9209 tablet_replica.cc:333] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:12:54.531411  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet 7c4e1687a1db419c8863789bb37c260a (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:54.531991  9209 raft_consensus.cc:2243] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:54.532639  9209 raft_consensus.cc:2272] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:54.541821  8123 tablet_service.cc:1558] Processing DeleteTablet for tablet b8fabe9526434f818c106b17c57b3b85 with delete_type TABLET_DATA_TOMBSTONED (TS 8a968cada1e4446ebb2a5e167176e649 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36038
I20260709 13:12:54.560758  8130 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b4358ace86545339036f21610735b47"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:54.561194  9216 tablet_replica.cc:333] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:12:54.561614  8130 raft_consensus.cc:606] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:54.564497  9216 raft_consensus.cc:2243] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:54.565281  9216 raft_consensus.cc:2272] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:54.566636  8130 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:54.567548  8130 raft_consensus.cc:606] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:12:54.570840  9209 ts_tablet_manager.cc:1916] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:54.573293  8137 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:54.573802  8137 raft_consensus.cc:606] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.569702  9216 ts_tablet_manager.cc:1916] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:54.576905  8137 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:54.577402  8137 raft_consensus.cc:606] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.580956  8130 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:54.581459  8130 raft_consensus.cc:606] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:54.583704  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:12:54.585140  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:54.606542  9216 ts_tablet_manager.cc:1929] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:54.609284  9216 log.cc:1199] T b8fabe9526434f818c106b17c57b3b85 P 8a968cada1e4446ebb2a5e167176e649: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/wals/b8fabe9526434f818c106b17c57b3b85
I20260709 13:12:54.609798  9209 ts_tablet_manager.cc:1929] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:12:54.611574  9209 log.cc:1199] T 5124ddc96f984ec2ae0ee97be6732a90 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/wals/5124ddc96f984ec2ae0ee97be6732a90
I20260709 13:12:54.611974  7945 catalog_manager.cc:5028] TS 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): tablet b8fabe9526434f818c106b17c57b3b85 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:54.612972  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet 5124ddc96f984ec2ae0ee97be6732a90 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:54.621522  9201 raft_consensus.cc:993] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:12:54.622045  9153 raft_consensus.cc:1081] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:12:54.623674  8060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b4358ace86545339036f21610735b47"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49222
I20260709 13:12:54.624318  8060 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:54.624668  8060 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.652588  9176 raft_consensus.cc:993] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:12:54.653093  9176 raft_consensus.cc:1081] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:12:54.657266  8060 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:54.658007  8208 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59868
I20260709 13:12:54.658578  8208 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:54.658922  8208 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.660168  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:54.660692  8130 raft_consensus.cc:3055] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:54.661108  8130 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:54.661840  8130 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:54.662886  8130 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.667443  8208 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:54.669355  8208 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:54.671787  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:54.672329  8137 raft_consensus.cc:3055] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:54.672606  8137 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:54.673342  8137 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:54.675390  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:54.675925  8059 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.699052  8137 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.711362  9131 raft_consensus.cc:993] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:12:54.711897  9131 raft_consensus.cc:1081] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:12:54.713383  8208 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59868
I20260709 13:12:54.713915  8208 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:54.714290  8208 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.721830  8208 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:54.723498  8208 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:54.727980  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:54.737394  8132 raft_consensus.cc:3055] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:54.737718  8132 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:54.738502  8132 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:54.739830  8132 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.730716  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:54.741714  8061 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.746451  8059 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:12:54.753019  8060 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:54.749574  8137 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:12:54.751966  8130 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:12:54.761673  8026 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:54.762800  9129 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:54.763188  9129 raft_consensus.cc:697] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:54.763958  9129 consensus_queue.cc:237] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:54.767275  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:54.767855  8206 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:54.771240  7945 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leader transfer in progress [suppressed 5 similar messages]
I20260709 13:12:54.779898  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:54.799355  9134 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:54.799889  9134 raft_consensus.cc:697] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:54.801120  9134 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:54.813086  8061 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:12:54.815083  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:54.815958  9134 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:54.816500  9134 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:54.817414  9134 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:54.820561  7945 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 49358.1us
I20260709 13:12:54.824767  7959 catalog_manager.cc:5697] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:54.827184  8132 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:12:54.827862  8206 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
W20260709 13:12:54.835716  7945 catalog_manager.cc:4755] TS 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): ChangeConfig:REMOVE_PEER RPC failed for tablet 5d0e2cd74dac4358aba60edf5382befd: Remote error: Service unavailable: leader transfer in progress
I20260709 13:12:54.853981  7959 catalog_manager.cc:5697] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:54.869525  7960 catalog_manager.cc:5697] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:54.909284  9182 raft_consensus.cc:993] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:54.909765  9182 raft_consensus.cc:1081] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:54.912273  8730 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55826
I20260709 13:12:54.912815  8730 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:54.913154  8730 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.921214  8730 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:54.924903  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:54.925402  8208 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.926045  8730 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:54.930948  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:54.931535  8137 raft_consensus.cc:3055] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:54.931849  8137 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:54.933063  8208 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:54.932662  8137 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:54.935425  8137 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.942508  8137 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:54.943125  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:54.943670  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:54.944339  8061 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.944767  9177 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:54.966931  9177 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:12:54.983768  7945 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 5d0e2cd74dac4358aba60edf5382befd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8a968cada1e4446ebb2a5e167176e649 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:54.988200  9131 raft_consensus.cc:993] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:12:54.988664  9131 raft_consensus.cc:1081] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:12:54.991348  8206 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59868
I20260709 13:12:54.991814  8206 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:54.992125  8206 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:54.998179  9177 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:55.003607  8061 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:55.013139  8206 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:55.016589  8206 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:55.019097  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:55.019657  8061 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:55.024914  7959 catalog_manager.cc:5697] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:55.031850  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:55.032436  8132 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:55.032795  8132 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:55.037962  8132 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:55.042809  8132 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:55.157351  8061 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:12:55.181258  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:55.182190  9180 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:55.182718  9180 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:55.183651  9180 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:55.190299  8132 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:12:55.209764  7960 catalog_manager.cc:5697] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:55.253343  8132 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:55.255425  8061 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:55.259011  9134 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:55.260622  9134 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:55.276190  8132 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:55.279125  9171 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:55.288551  8130 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:55.310060  9134 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:55.312937  8206 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:55.321615  9196 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:55.451256  8061 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:55.478888  8209 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:12:55.488551  9177 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:55.489420  9173 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:55.499111  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 63895.3us
I20260709 13:12:55.563058  8061 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:55.564406  9120 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:55.570372  8132 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:55.572281  9120 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:12:55.586164  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:55.598196  9240 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:55.598757  9240 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:55.614636  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:12:55.614979  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:12:55.615252  8057 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:12:55.615505  8132 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:12:55.613319  9240 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:55.619102  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:55.620573  9236 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:12:55.620919  9236 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:55.621239  9236 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:55.626447  8132 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:12:55.627890  9177 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:55.592119  8060 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:12:55.642107  9177 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:55.674939  8730 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:55.969956  8057 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:55.928869  9236 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:55.973704  9236 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:55.989161  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:55.989840  8058 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:55.994037  9249 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:12:55.976372  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:56.012310  9250 raft_consensus.cc:2955] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:56.014930  8059 raft_consensus.cc:2955] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:56.029620  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:56.030207  8135 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:12:56.033525  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 124696us
I20260709 13:12:56.037025  8132 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:56.040643  8135 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:12:56.043200  7946 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5d0e2cd74dac4358aba60edf5382befd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20260709 13:12:56.001917  8058 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:12:56.049940  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:56.050932  9236 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:56.051383  9236 raft_consensus.cc:697] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:56.052160  9236 consensus_queue.cc:237] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:56.054641  7958 catalog_manager.cc:5697] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 3 to 5, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:56.078073  9177 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:56.097260  8135 raft_consensus.cc:2955] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:12:56.103484  7959 catalog_manager.cc:5697] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 3, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 3 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:56.151136  8198 tablet_service.cc:1558] Processing DeleteTablet for tablet 5d0e2cd74dac4358aba60edf5382befd with delete_type TABLET_DATA_TOMBSTONED (TS d9cd2999aa314b39997b423ec1ae509e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59846
I20260709 13:12:56.195355  9269 tablet_replica.cc:333] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:12:56.196493  9269 raft_consensus.cc:2243] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.197276  9269 pending_rounds.cc:70] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e: Trying to abort 1 pending ops.
I20260709 13:12:56.197633  9269 pending_rounds.cc:77] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e: Aborting op as it isn't in flight: id { term: 2 index: 4 } timestamp: 7305636966440419328 op_type: NO_OP noop_request { }
I20260709 13:12:56.198148  9269 raft_consensus.cc:2889] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:12:56.198582  9269 raft_consensus.cc:2272] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.201907  9269 ts_tablet_manager.cc:1916] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:56.208323  8135 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:56.208894  8135 raft_consensus.cc:606] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:56.215828  8135 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:56.216280  8135 raft_consensus.cc:606] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:12:56.218777  8135 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:56.219275  8135 raft_consensus.cc:606] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:56.221663  8135 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:56.222189  8135 raft_consensus.cc:606] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:12:56.223172  9269 ts_tablet_manager.cc:1929] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:12:56.223652  9269 log.cc:1199] T 5d0e2cd74dac4358aba60edf5382befd P d9cd2999aa314b39997b423ec1ae509e: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/wals/5d0e2cd74dac4358aba60edf5382befd
I20260709 13:12:56.224853  8135 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:56.225364  8135 raft_consensus.cc:606] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:12:56.226558  7948 catalog_manager.cc:5028] TS d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): tablet 5d0e2cd74dac4358aba60edf5382befd (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:12:56.275800  8212 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:48498
I20260709 13:12:56.276301  8212 raft_consensus.cc:606] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:56.277295  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:12:56.278465  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:56.441035  9251 raft_consensus.cc:993] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:56.441505  9251 raft_consensus.cc:1081] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:56.443718  8727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55826
I20260709 13:12:56.444432  8727 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:56.444844  8727 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:56.479363  8727 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:56.482053  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:56.482594  8132 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:56.482851  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:56.483539  8212 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:56.482895  8132 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:56.484485  8132 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:56.485561  8132 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:56.482940  8727 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:56.490166  8212 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:56.491324  8696 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:56.492265  9177 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:56.492722  9177 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:12:56.493556  9177 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:56.504510  8132 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:56.644781  9278 raft_consensus.cc:993] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:56.645167  9251 raft_consensus.cc:993] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:56.645726  9251 raft_consensus.cc:1081] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:56.646838  9182 raft_consensus.cc:1081] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:56.647234  8726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55826
I20260709 13:12:56.647766  8726 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:56.648134  8726 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:56.660017  8058 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:12:56.671324  8137 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:12:56.671453  8730 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55826
I20260709 13:12:56.672539  8730 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:56.672883  8730 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:56.674002  7959 catalog_manager.cc:5697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:56.717944  9241 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:56.735981  9241 raft_consensus.cc:993] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:12:56.736600  9280 raft_consensus.cc:1081] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:12:56.744339  9239 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:56.747063  8726 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:56.749533  8059 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49210
I20260709 13:12:56.745741  8730 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:56.750026  8059 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:56.750180  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:56.750519  8059 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:56.752354  8137 raft_consensus.cc:3055] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:56.752655  8137 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:56.753350  8137 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:56.749557  8726 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:56.760278  8137 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:56.763383  8730 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:56.774721  8137 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:56.775548  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:56.776281  8212 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:56.793252  8059 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:56.801021  8212 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:56.823896  8059 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:56.824766  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:56.825753  8208 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:56.824887  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:56.827425  8212 raft_consensus.cc:3055] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:56.828032  8212 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:12:56.832087  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 73809.3us
I20260709 13:12:56.843046  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:12:56.829162  8212 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:56.844298  9177 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:56.844509  8212 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:56.844973  9177 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:12:56.845984  9177 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:56.852962  8208 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
W20260709 13:12:56.854584  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58711.8us
I20260709 13:12:56.856532  8212 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:12:56.861361  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:56.867393  8137 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:56.867784  8137 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:56.868656  8137 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:56.883780  8137 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:56.943708  8137 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:56.944813  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:12:56.945886  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 88887.3us
I20260709 13:12:56.945629  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:56.962844  8133 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:56.958761  9250 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:56.964731  9250 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:12:56.965751  9250 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:56.962881  8029 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:56.980633  9297 raft_consensus.cc:993] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:12:56.981094  9297 raft_consensus.cc:1081] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:12:56.974720  9274 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:56.987155  9274 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:56.987921  9274 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:56.989995  8133 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:12:56.994374  8208 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59868
I20260709 13:12:56.995010  8208 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:56.995343  8208 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.001557  8208 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:57.003566  8208 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:57.014086  7958 catalog_manager.cc:5697] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:57.038120  9288 raft_consensus.cc:993] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:12:57.044469  9288 raft_consensus.cc:1081] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:12:57.042798  8209 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:57.047137  8058 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49222
I20260709 13:12:57.047768  8058 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:57.048202  8058 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:57.062388  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 117618us
I20260709 13:12:57.066170  9177 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:57.091571  8136 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:57.093204  7961 catalog_manager.cc:5697] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 2 to 3, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 3 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:57.058266  8058 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:57.106432  9302 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:57.109185  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:57.109706  8206 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:57.097198  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 263852us
I20260709 13:12:57.112840  8058 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:57.120066  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:57.120577  8062 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.126818  8058 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:57.089910  7958 catalog_manager.cc:5697] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:57.129401  8206 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:12:57.132536  9251 consensus_queue.cc:1059] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:57.133512  9282 raft_consensus.cc:3055] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:57.133858  9282 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:57.134238  8208 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:57.135347  9282 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:57.136404  9282 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.137910  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:57.142704  8029 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:57.143671  9304 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:57.144349  9304 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:57.144577  8062 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:12:57.145227  9304 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:57.146909  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:57.147441  8137 raft_consensus.cc:3055] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:57.147744  8137 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:57.148478  8137 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:57.149606  8137 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.168946  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:57.169802  9303 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:57.170995  9303 raft_consensus.cc:697] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:57.171924  9303 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:57.181396  8135 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:12:57.208875  8137 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:12:57.210438  7958 catalog_manager.cc:5697] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:57.308851  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:57.312561  8136 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:57.313090  7959 catalog_manager.cc:5697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:57.315060  9250 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:57.373137  8209 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:57.403164  9250 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:12:57.413518  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45648.9us
I20260709 13:12:57.459435  8137 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:57.461001  9305 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:57.478690  8209 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:57.480217  9274 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:12:57.536811  7992 debug-util.cc:398] Leaking SignalData structure 0x7b0800038340 after lost signal to thread 6940
W20260709 13:12:57.537748  7992 debug-util.cc:398] Leaking SignalData structure 0x7b0800038a20 after lost signal to thread 7995
W20260709 13:12:57.538861  7992 debug-util.cc:398] Leaking SignalData structure 0x7b08002ecda0 after lost signal to thread 8087
W20260709 13:12:57.539841  7992 debug-util.cc:398] Leaking SignalData structure 0x7b0800319120 after lost signal to thread 8162
W20260709 13:12:57.540820  7992 debug-util.cc:398] Leaking SignalData structure 0x7b080033ea20 after lost signal to thread 8236
I20260709 13:12:57.541072  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:57.541846  7992 debug-util.cc:398] Leaking SignalData structure 0x7b080034dcc0 after lost signal to thread 8754
I20260709 13:12:57.546981  8137 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:57.549363  9302 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:57.546998  8206 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:57.556145  9322 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:57.578948  8136 consensus_queue.cc:237] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:57.619036  8137 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:57.619596  8137 raft_consensus.cc:606] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:57.626325  8133 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:57.626909  8133 raft_consensus.cc:606] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:12:57.632829  8102 consensus_peers.cc:597] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b186be823643475982e269b73d886a42. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:57.638960  8130 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:57.640338  9134 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:57.646165  8212 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:57.653589  9282 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:57.659154  8058 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:57.660825  9288 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:57.697695  8062 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:12:57.697857  8057 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:57.698253  8062 raft_consensus.cc:606] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:57.699034  9134 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:57.707988  8206 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:48498
I20260709 13:12:57.708539  8206 raft_consensus.cc:606] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:12:57.713153  9297 raft_consensus.cc:2955] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:57.720834  8057 raft_consensus.cc:2955] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:57.724965  8130 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:57.734632  9304 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:57.739012  8058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:12:57.738404  8209 raft_consensus.cc:2955] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:57.739617  8058 raft_consensus.cc:606] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:12:57.746970  8130 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:57.747509  8130 raft_consensus.cc:606] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:57.758859  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:12:57.760432  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:57.767637  8212 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:57.769438  9305 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:57.775735  9304 raft_consensus.cc:993] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:12:57.776255  9299 raft_consensus.cc:1081] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:12:57.777751  8209 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59856
I20260709 13:12:57.778254  8209 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:57.778620  8209 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.791986  8209 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:57.797407  8209 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:57.798508  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:57.799309  8137 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.802626  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:57.803264  8058 raft_consensus.cc:3055] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:57.805126  8058 raft_consensus.cc:740] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:57.805917  8058 consensus_queue.cc:260] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:57.807166  8058 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.817528  8137 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:12:57.818058  7956 catalog_manager.cc:5697] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:57.824199  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:57.825695  9134 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:57.826264  9134 raft_consensus.cc:697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:57.827272  9134 consensus_queue.cc:237] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:57.838423  8058 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:12:57.839236  8137 consensus_queue.cc:237] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:57.859583  9297 raft_consensus.cc:993] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:12:57.860168  9297 raft_consensus.cc:1081] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:12:57.874434  8058 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49222
I20260709 13:12:57.875016  8058 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:57.875350  8058 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.888295  8058 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:57.891389  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 48856.2us
I20260709 13:12:57.895664  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:57.896236  8209 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.900396  8058 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:57.902948  8209 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:12:57.905979  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:57.906976  9274 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:57.907431  9274 raft_consensus.cc:697] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:57.908226  9274 consensus_queue.cc:237] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:57.909453  8206 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:57.911103  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:57.911473  8130 raft_consensus.cc:3055] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:57.911643  8130 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:57.913450  8130 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:57.922699  8130 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.926968  9304 raft_consensus.cc:993] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:57.927507  9305 raft_consensus.cc:1081] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:57.937237  8130 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:12:57.914570  9288 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:12:57.947474  8730 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55848
I20260709 13:12:57.947983  8730 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:57.948352  8730 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:57.955215  8102 consensus_peers.cc:597] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b578d88c1c1f4ca58b7291fed4909f09. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:57.956514  8058 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:57.958369  9345 raft_consensus.cc:993] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:12:57.959131  9282 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:57.961396  9345 raft_consensus.cc:1081] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:12:57.961589  8730 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:57.965201  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:57.965871  8057 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:57.966218  8057 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:57.966410  8730 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:57.967120  8057 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:57.970526  8058 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49222
I20260709 13:12:57.971203  8058 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:57.971621  8058 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.975433  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:57.976154  8137 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.981289  8058 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:57.981952  8057 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:57.990689  8057 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:57.991365  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:57.991922  8130 raft_consensus.cc:3055] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:57.992209  8130 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:57.992901  8130 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:57.994796  8130 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:58.003432  8130 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:12:58.005817  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:12:58.006433  9299 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:58.006882  9299 raft_consensus.cc:697] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:58.007762  9299 consensus_queue.cc:237] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:58.009545  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:12:58.010309  9177 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:58.010854  9177 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:12:58.011703  9177 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:58.012533  8137 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:58.012650  9282 raft_consensus.cc:2955] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:58.016893  8058 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:58.018352  8206 raft_consensus.cc:2955] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:58.020395  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:58.020993  8212 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:58.026804  8212 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:12:58.034790  8058 raft_consensus.cc:2955] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:58.040144  7960 catalog_manager.cc:5697] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 2, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:58.068491  8212 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:58.088126  8137 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:12:58.089991  9134 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:12:58.096562  8176 consensus_peers.cc:597] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a478a4313e6149378cae4b6f833e3c2a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:58.099071  8061 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:12:58.098594  7959 catalog_manager.cc:5697] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:58.101055  9303 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:58.099495  9297 raft_consensus.cc:993] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:12:58.103466  9297 raft_consensus.cc:1081] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:12:58.113669  8057 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49222
I20260709 13:12:58.114158  8057 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:58.114097  7960 catalog_manager.cc:5697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 2, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:58.114995  8057 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:58.125197  8057 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:58.126929  8057 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:58.128239  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:58.129045  8212 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:58.137594  8212 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:12:58.138607  8029 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:58.139330  9299 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:58.139779  9299 raft_consensus.cc:697] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:58.140584  9299 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:58.143643  7956 catalog_manager.cc:5697] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:58.144704  9340 raft_consensus.cc:2955] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:58.147652  8059 raft_consensus.cc:2955] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:58.148981  7956 catalog_manager.cc:5697] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:58.178834  8130 raft_consensus.cc:2955] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:58.181725  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:58.182274  8137 raft_consensus.cc:3055] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:58.182554  8137 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:58.183241  8137 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:58.184324  8137 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:58.184588  7961 catalog_manager.cc:5697] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:58.204697  8137 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:12:58.221370  9241 raft_consensus.cc:993] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:58.224002  9241 raft_consensus.cc:1081] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:58.225513  8730 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55838
I20260709 13:12:58.226001  8730 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:58.226321  8730 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:58.225875  7961 catalog_manager.cc:5697] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 3, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 2 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
W20260709 13:12:58.372263  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 155250us
I20260709 13:12:58.410832  8730 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:58.413026  8730 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:12:58.417472  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 44312.4us
W20260709 13:12:58.418606  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 185426us
I20260709 13:12:58.452169  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:58.469583  8206 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:58.470978  8206 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:58.471995  8206 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:58.473161  8206 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:58.487592  8059 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:58.491191  8212 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:58.492857  9358 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:58.493643  8130 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:58.495714  9359 ts_tablet_manager.cc:933] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:58.498015  8212 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:58.512641  9359 tablet_copy_client.cc:323] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:12:58.514657  9305 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:58.517488  9358 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:58.519657  9305 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:58.524578  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:58.525159  8058 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:58.532151  8058 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:58.532521  8206 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:58.535238  9362 ts_tablet_manager.cc:933] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:58.553889  8696 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:58.556387  9362 tablet_copy_client.cc:323] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.67:35745
I20260709 13:12:58.559609  9364 ts_tablet_manager.cc:933] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:58.562574  8058 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:58.609103  9364 tablet_copy_client.cc:323] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:12:58.577981  8058 consensus_queue.cc:237] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
W20260709 13:12:58.616245  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 62474.5us
I20260709 13:12:58.617195  9302 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:58.617755  9302 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:12:58.618718  9302 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:58.639120  7961 catalog_manager.cc:5697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 2, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:12:58.640595  8027 consensus_peers.cc:597] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a003b4584d3747a381522d67299aaed2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:58.641458  8206 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:58.644214  9274 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:58.652161  8148 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet b186be823643475982e269b73d886a42 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:12:58.652393  8147 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet b578d88c1c1f4ca58b7291fed4909f09 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:12:58.653180  8148 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet b186be823643475982e269b73d886a42 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-b186be823643475982e269b73d886a42
I20260709 13:12:58.655092  8147 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet b578d88c1c1f4ca58b7291fed4909f09 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-b578d88c1c1f4ca58b7291fed4909f09
I20260709 13:12:58.661985  8148 tablet_copy_source_session.cc:215] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:58.666815  8147 tablet_copy_source_session.cc:215] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:12:58.674474  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 54628us
W20260709 13:12:58.675083  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:58.698797  8132 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:58.700299  8136 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:12:58.702302  8130 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:58.708607  8222 tablet_copy_service.cc:140] P d9cd2999aa314b39997b423ec1ae509e: Received BeginTabletCopySession request for tablet a478a4313e6149378cae4b6f833e3c2a from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:55942)
I20260709 13:12:58.709118  8222 tablet_copy_service.cc:161] P d9cd2999aa314b39997b423ec1ae509e: Beginning new tablet copy session on tablet a478a4313e6149378cae4b6f833e3c2a from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:55942: session id = 965e59f272f143449b18f65db4e61d98-a478a4313e6149378cae4b6f833e3c2a
I20260709 13:12:58.723742  8222 tablet_copy_source_session.cc:215] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:12:58.725235  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 106445us
I20260709 13:12:58.726934  9177 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:12:58.771785  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:58.789191  8205 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:58.790611  9305 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:58.791185  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 59373.7us
W20260709 13:12:58.799644  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 71078.3us
I20260709 13:12:58.801033  9362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a478a4313e6149378cae4b6f833e3c2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:58.850121  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:58.801409  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 84778.8us
I20260709 13:12:58.808190  8130 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:58.882386  9364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b578d88c1c1f4ca58b7291fed4909f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:58.885974  9359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b186be823643475982e269b73d886a42. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:12:58.886435  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 90411.3us
I20260709 13:12:58.899962  9304 raft_consensus.cc:2955] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:58.904410  8206 raft_consensus.cc:2955] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:58.920939  9359 tablet_copy_client.cc:806] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:58.921501  9359 tablet_copy_client.cc:670] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:58.926959  9364 tablet_copy_client.cc:806] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:58.927474  9364 tablet_copy_client.cc:670] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
W20260709 13:12:58.939842  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 57655.8us
I20260709 13:12:58.968000  8058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:12:58.978560  8058 raft_consensus.cc:606] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:12:58.987571  9305 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:58.988914  9299 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:58.989202  9305 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:58.992164  8058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:12:58.992898  8058 raft_consensus.cc:606] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:12:59.006502  8136 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:59.008349  9362 tablet_copy_client.cc:806] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:59.008948  9362 tablet_copy_client.cc:670] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:59.009640  8206 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:48498
I20260709 13:12:59.010206  8206 raft_consensus.cc:606] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:12:59.020310  9362 tablet_copy_client.cc:538] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:59.023312  9177 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:12:58.993535  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 123259us
I20260709 13:12:59.019891  7961 catalog_manager.cc:5697] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:59.027675  9364 tablet_copy_client.cc:538] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:59.028127  8058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8dd7949df539418cbc00001111885883"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:12:59.028620  8058 raft_consensus.cc:606] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:12:59.030411  9359 tablet_copy_client.cc:538] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:59.021016  9299 raft_consensus.cc:993] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:59.040117  9274 raft_consensus.cc:1081] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:59.041100  8130 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:59.041682  8130 raft_consensus.cc:606] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:59.043746  8729 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55848
I20260709 13:12:59.044122  9362 tablet_bootstrap.cc:492] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:59.044364  8729 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:59.044711  8729 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:59.046237  9364 tablet_bootstrap.cc:492] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:59.051292  8729 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:59.070775  8130 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:12:59.071290  8130 raft_consensus.cc:606] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:12:59.073469  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:59.080909  8061 raft_consensus.cc:3055] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:59.078146  9359 tablet_bootstrap.cc:492] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:12:59.079619  8729 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:59.084590  8061 raft_consensus.cc:740] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:59.086797  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:12:59.088331  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:12:59.088775  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 100635us
I20260709 13:12:59.092468  9343 raft_consensus.cc:993] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:12:59.093008  9343 raft_consensus.cc:1081] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:12:59.079895  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:59.102084  8205 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:59.112653  8058 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49222
I20260709 13:12:59.113169  8058 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:59.113507  8058 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:59.086192  8061 consensus_queue.cc:260] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:59.133157  8137 raft_consensus.cc:2955] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:12:59.193702  8058 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:59.195315  8061 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:59.200605  8058 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:12:59.209810  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 77194.1us
W20260709 13:12:59.214754  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 60462us
I20260709 13:12:59.232228  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:59.232962  8206 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:59.242219  8206 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:12:59.252377  8061 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:59.253690  8693 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:12:59.262370  9302 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:59.264242  9302 raft_consensus.cc:697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:12:59.266022  8206 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:59.248739  9299 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
W20260709 13:12:59.300683  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 88437.5us
I20260709 13:12:59.301414  9396 raft_consensus.cc:993] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:59.265077  9302 consensus_queue.cc:237] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:12:59.346484  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 89565.1us
I20260709 13:12:59.347434  8029 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:12:59.349637  9396 raft_consensus.cc:1081] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:59.351094  8061 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:12:59.352828  9396 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:59.353296  9396 raft_consensus.cc:697] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:59.354123  9396 consensus_queue.cc:237] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:12:59.367780  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58746.7us
I20260709 13:12:59.385838  8730 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8dd7949df539418cbc00001111885883"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55848
W20260709 13:12:59.386212  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 83464.6us
I20260709 13:12:59.387044  8730 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:59.387689  8730 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:59.435341  9387 raft_consensus.cc:993] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:12:59.436097  9387 raft_consensus.cc:1081] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:12:59.374667  9393 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:12:59.460095  9372 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
W20260709 13:12:59.467023  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 119226us
I20260709 13:12:59.470494  8206 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59856
I20260709 13:12:59.471065  8206 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:59.471510  8206 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:59.473239  9356 raft_consensus.cc:993] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:59.481462  9358 raft_consensus.cc:1081] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:12:59.477896  8205 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:59.502884  8727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55838
I20260709 13:12:59.507328  8727 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:59.508041  8727 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:59.522842  8730 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:59.534221  8730 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:59.534555  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:59.540048  8133 raft_consensus.cc:3055] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:59.540398  8133 raft_consensus.cc:740] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:12:59.537670  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:59.541429  8132 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:59.541143  8133 consensus_queue.cc:260] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:59.553047  8206 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:12:59.644294  8206 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:12:59.571811  8132 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:59.654042  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:12:59.614115  8133 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:59.687793  9362 tablet_bootstrap.cc:492] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:59.688999  9406 ts_tablet_manager.cc:933] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
I20260709 13:12:59.693318  9393 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:59.740065  9393 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:12:59.753516  9362 tablet_bootstrap.cc:492] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:12:59.776176  8133 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:12:59.782972  9362 ts_tablet_manager.cc:1403] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.739s	user 0.024s	sys 0.010s
I20260709 13:12:59.796115  9362 raft_consensus.cc:359] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:59.800307  9362 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:12:59.801134  9362 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:12:59.802330  9393 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:59.809500  8727 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:12:59.812978  9410 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:12:59.832782  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:59.833552  8135 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:59.849200  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 158592us
I20260709 13:12:59.861160  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:59.861814  8058 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:59.862121  8058 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:12:59.862823  8058 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:59.864009  7961 catalog_manager.cc:5697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 2, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:59.878913  8135 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:12:59.880038  8693 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:12:59.880800  9393 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:59.847854  8727 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:12:59.850773  9362 ts_tablet_manager.cc:1434] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.001s
I20260709 13:12:59.881256  9393 raft_consensus.cc:697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:12:59.897630  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:12:59.897998  8136 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:12:59.898372  8205 raft_consensus.cc:3055] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:59.903685  8205 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:12:59.916260  8205 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:59.935613  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:12:59.936271  8134 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:59.891211  9406 tablet_copy_client.cc:323] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.65:45919
I20260709 13:12:59.927748  9393 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:12:59.940243  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:12:59.943130  8060 raft_consensus.cc:3055] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:59.943442  8060 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:12:59.961849  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:59.940681  8205 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:59.993187  8060 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:00.130546  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 102769us
W20260709 13:13:00.131378  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 104052us
I20260709 13:13:00.137593  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:00.140455  8060 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:00.183377  8208 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:00.158849  8073 tablet_copy_service.cc:140] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Received BeginTabletCopySession request for tablet a003b4584d3747a381522d67299aaed2 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:37156)
W20260709 13:13:00.198625  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 55409.5us
I20260709 13:13:00.206832  8073 tablet_copy_service.cc:161] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Beginning new tablet copy session on tablet a003b4584d3747a381522d67299aaed2 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:37156: session id = 965e59f272f143449b18f65db4e61d98-a003b4584d3747a381522d67299aaed2
I20260709 13:13:00.228190  8060 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
W20260709 13:13:00.236549  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 66512.4us
I20260709 13:13:00.223443  8222 tablet_copy_service.cc:342] P d9cd2999aa314b39997b423ec1ae509e: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-a478a4313e6149378cae4b6f833e3c2a received from {username='slave'} at 127.0.0.1:55942
I20260709 13:13:00.260309  8209 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:13:00.263300  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:00.247834  8222 tablet_copy_service.cc:434] P d9cd2999aa314b39997b423ec1ae509e: ending tablet copy session 965e59f272f143449b18f65db4e61d98-a478a4313e6149378cae4b6f833e3c2a on tablet a478a4313e6149378cae4b6f833e3c2a with peer 965e59f272f143449b18f65db4e61d98
W20260709 13:13:00.270598  9424 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.340s	user 0.000s	sys 0.001s
I20260709 13:13:00.302912  9359 tablet_bootstrap.cc:492] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:00.331621  9359 tablet_bootstrap.cc:492] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:13:00.332542  9364 tablet_bootstrap.cc:492] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:00.333006  9359 ts_tablet_manager.cc:1403] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 1.255s	user 0.030s	sys 0.034s
I20260709 13:13:00.333472  9364 tablet_bootstrap.cc:492] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:13:00.334110  9364 ts_tablet_manager.cc:1403] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 1.288s	user 0.020s	sys 0.021s
I20260709 13:13:00.361585  8205 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
W20260709 13:13:00.363953  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 164480us
I20260709 13:13:00.320286  9177 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:13:00.368126  8730 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:00.373205  9433 raft_consensus.cc:993] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:13:00.373965  9433 raft_consensus.cc:1081] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
W20260709 13:13:00.390729  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 184549us
I20260709 13:13:00.398171  8073 tablet_copy_source_session.cc:215] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:13:00.408174  9364 raft_consensus.cc:359] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:00.409778  9364 raft_consensus.cc:740] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
W20260709 13:13:00.427696  9427 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.176s	user 0.004s	sys 0.000s
I20260709 13:13:00.401489  9359 raft_consensus.cc:359] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:00.437389  9359 raft_consensus.cc:740] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:13:00.447926  9359 consensus_queue.cc:260] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:00.465241  9364 consensus_queue.cc:260] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:00.478863  8729 raft_consensus.cc:1217] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:13:00.483289  9364 ts_tablet_manager.cc:1434] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.149s	user 0.009s	sys 0.003s
I20260709 13:13:00.491786  8058 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:00.504920  8728 raft_consensus.cc:1217] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:13:00.519949  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:00.522914  8726 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:00.527042  9406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a003b4584d3747a381522d67299aaed2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:00.547590  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 168161us
I20260709 13:13:00.553004  8056 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49222
I20260709 13:13:00.553855  8056 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:00.554394  8056 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:00.559649  8135 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:00.560792  8132 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:00.562487  8133 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:00.563102  8147 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-b578d88c1c1f4ca58b7291fed4909f09 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:13:00.563706  8147 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-b578d88c1c1f4ca58b7291fed4909f09 on tablet b578d88c1c1f4ca58b7291fed4909f09 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:13:00.567790  8058 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:00.570387  9302 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:00.572008  8207 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:00.572012  9302 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:13:00.574543  8134 consensus_meta.cc:287] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.638s	user 0.013s	sys 0.008s
I20260709 13:13:00.575140  8134 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:00.578131  8059 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:00.578308  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:00.579408  8056 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:00.581115  9303 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:00.581709  9303 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:00.590085  9303 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:00.590798  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:00.591451  8727 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:00.583338  9359 ts_tablet_manager.cc:1434] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.250s	user 0.007s	sys 0.006s
I20260709 13:13:00.621241  8729 raft_consensus.cc:1217] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:13:00.617491  8058 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:00.595150  8056 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:00.620766  9302 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:13:00.646237  8147 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-b186be823643475982e269b73d886a42 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:13:00.654608  8147 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-b186be823643475982e269b73d886a42 on tablet b186be823643475982e269b73d886a42 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:13:00.687665  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:00.688638  8134 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:00.689070  8134 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:00.691510  9177 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:00.699187  8727 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:00.700722  8027 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:00.704751  8206 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:00.690074  8134 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:00.713137  8134 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:00.700659  8056 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:00.717312  9394 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:00.717882  9394 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:00.719300  9394 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:00.878779  7961 catalog_manager.cc:5697] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:00.883841  9406 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:55848 (request call id 228) took 1602 ms. Trace:
I20260709 13:13:00.884449  9406 rpcz_store.cc:276] 0709 13:12:59.281087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:59.281233 (+   146us) service_pool.cc:224] Handling call
0709 13:12:59.724533 (+443300us) ts_tablet_manager.cc:934] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
0709 13:13:00.525126 (+800593us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:13:00.636758 (+111632us) tablet_metadata.cc:665] Metadata flushed
0709 13:13:00.639893 (+  3135us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:13:00.756779 (+116886us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:13:00.883033 (+126254us) tablet_replica.cc:206] Creating consensus instance
0709 13:13:00.883526 (+   493us) ts_tablet_manager.cc:989] Replica registered.
0709 13:13:00.883679 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":43715,"tablet-copy.queue_time_us":223635,"thread_start_us":261318,"threads_started":2}
I20260709 13:13:00.893146  8210 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:00.894300  9443 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:13:00.896157  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 109186us
I20260709 13:13:00.907243  9434 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:00.931906  9406 tablet_copy_client.cc:806] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:13:00.965127  9397 raft_consensus.cc:1064] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:13:00.966846  8134 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
W20260709 13:13:01.042217  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 72322.9us
I20260709 13:13:00.967973  9397 consensus_queue.cc:237] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:01.084791  9447 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.304s	user 0.007s	sys 0.002s
I20260709 13:13:01.103143  9406 tablet_copy_client.cc:670] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
W20260709 13:13:01.153414  9452 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.285s	user 0.019s	sys 0.008s
I20260709 13:13:01.160035  9419 raft_consensus.cc:1064] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
W20260709 13:13:01.203989  9454 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:13:01.227862  9446 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.345s	user 0.003s	sys 0.002s
I20260709 13:13:01.246899  9419 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:01.255270  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 318777us
I20260709 13:13:01.292047  8730 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:01.293483  9387 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:01.314246  9406 tablet_copy_client.cc:538] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:13:01.347119  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 211912us
W20260709 13:13:01.352218  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 307637us
I20260709 13:13:01.355505  8135 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:01.370376  8131 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:01.373371  8059 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:01.387696  8726 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:13:01.389237  9460 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:01.398780  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:01.414484  9459 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:01.430574  9385 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:01.438776  9459 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:01.445894  8060 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:13:01.468076  9461 raft_consensus.cc:1064] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:13:01.470772  9461 consensus_queue.cc:237] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:01.492944  9344 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:13:01.494647  8728 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 2 LEARNER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:13:01.496143  8131 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:13:01.496544  9459 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:01.500805  9415 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:13:01.502384  8209 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:13:01.509855  9406 tablet_bootstrap.cc:492] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:13:01.512470  9282 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
W20260709 13:13:02.259050  9475 log.cc:927] Time spent T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:13:02.670044  9357 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:02.670701  9357 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:02.702108  8178 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.047s	sys 0.099s
W20260709 13:13:02.702778  8178 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.048s	sys 0.099s
W20260709 13:13:02.702487  8177 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.021s	sys 0.067s
W20260709 13:13:02.703532  8177 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.021s	sys 0.067s
W20260709 13:13:02.704849  8028 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.015s	sys 0.044s
W20260709 13:13:02.705219  8028 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.015s	sys 0.044s
I20260709 13:13:02.707043  9481 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:02.707468  9357 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:02.707545  9481 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:02.709687  9481 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:02.711386  9481 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:02.719419  8101 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.006s	sys 0.024s
W20260709 13:13:02.719887  8101 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.006s	sys 0.024s
W20260709 13:13:02.727427  8027 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.011s	sys 0.022s
W20260709 13:13:02.727916  8027 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.011s	sys 0.022s
I20260709 13:13:02.729919  9486 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:02.731128  9486 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:02.738075  9484 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:02.738682  9488 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:02.739228  9486 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:02.738798  9484 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:02.739547  9488 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:02.742111  9488 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:02.742645  9484 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:02.743325  9490 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:02.744083  9490 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:02.711901  9481 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:02.756404  9490 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:02.758456  9481 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:02.750864  9487 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:02.752940  9488 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:02.760511  9487 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:02.760699  9488 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:02.763749  9488 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:02.764424  9487 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:02.754218  9492 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:02.769742  9492 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:02.773059  9492 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:02.776048  9481 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:02.776625  9481 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:02.783691  9481 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:02.895267  9418 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:02.895890  9418 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:02.896286  9490 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:02.896795  9490 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:02.898010  9418 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:02.898536  9483 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:02.898806  9490 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:02.900089  9484 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:02.773878  9494 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:02.908075  9484 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:02.907070  9483 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:02.907389  9487 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:02.911201  9484 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:02.916535  9483 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:02.913501  9490 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:02.917421  9490 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:02.918668  9372 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:02.928649  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:02.929203  7961 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34120 (request call id 244) took 2224 ms. Trace:
I20260709 13:13:02.929728  7961 rpcz_store.cc:276] 0709 13:13:00.704803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:00.704975 (+   172us) service_pool.cc:224] Handling call
0709 13:13:00.727000 (+ 22025us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b5400172d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf80090 00007f6dbb01ef3a 00007f6dc0dd51cc 00007f6dc0ed337e 00007f6dbf3942a4 00007f6dbf394232 00007f6dbf394161 00007f6dbf3940dc 00007f6dbf393392 00007f6db935148c 00007f6db93508c6
0709 13:13:00.749422 (+ 22422us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b5400002210. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef2640 00007f6dc0dc9b76 00007f6dc0ed3c3e 00007f6dbf3942a4 00007f6dbf394232 00007f6dbf394161 00007f6dbf3940dc 00007f6dbf393392 00007f6db935148c 00007f6db93508c6
0709 13:13:02.929099 (+2179677us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:01.151244 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:01.151817 (+   573us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:01.151884 (+    67us) write_op.cc:435] Acquired schema lock
0709 13:13:01.151921 (+    37us) tablet.cc:662] Decoding operations
0709 13:13:01.152361 (+   440us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:01.152464 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:01.152504 (+    40us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:13:01.152802 (+   298us) tablet.cc:701] Row locks acquired
0709 13:13:01.152863 (+    61us) write_op.cc:260] PREPARE: finished
0709 13:13:01.153034 (+   171us) write_op.cc:270] Start()
0709 13:13:01.153185 (+   151us) write_op.cc:276] Timestamp: P: 1783602781152963 usec, L: 0
0709 13:13:01.153223 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:13:01.153889 (+   666us) log.cc:844] Serialized 538 byte log entry
0709 13:13:01.427821 (+273932us) op_driver.cc:464] REPLICATION: finished
0709 13:13:01.516668 (+ 88847us) write_op.cc:301] APPLY: starting
0709 13:13:01.517066 (+   398us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:01.518803 (+  1737us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:01.597064 (+ 78261us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:02.041056 (+443992us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:02.103316 (+ 62260us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:13:02.103389 (+    73us) write_op.cc:312] APPLY: finished
0709 13:13:02.103761 (+   372us) log.cc:844] Serialized 23 byte log entry
0709 13:13:02.110481 (+  6720us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:02.110599 (+   118us) write_op.cc:454] Released schema lock
0709 13:13:02.110660 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:02.111088 (+   428us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:02.111131 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"spinlock_wait_cycles":79846656,"child_traces":[["op",{"apply.queue_time_us":85495,"apply.run_cpu_time_us":2863,"apply.run_wall_time_us":594371,"num_ops":1,"prepare.queue_time_us":82791,"prepare.run_cpu_time_us":3117,"prepare.run_wall_time_us":59461,"replication_time_us":274457,"thread_start_us":68793,"threads_started":2}]]}
I20260709 13:13:02.909657  9481 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:02.912822  9488 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:02.919288  9372 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:02.924551  9486 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:02.913772  9492 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:02.946535  9418 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:02.958843  9418 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:02.960842  9418 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:02.935326  9487 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:02.947561  9481 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:02.924809  9490 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:02.957286  9499 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:02.908526  9494 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:02.937251  9483 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:02.980042  9483 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:02.981103  9497 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:02.981581  9497 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:02.982066  9483 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:02.937366  9484 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:02.982753  9484 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:02.983742  9497 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:02.967060  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:02.967643  9496 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:02.963629  9492 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:02.968356  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:02.969625  9418 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:02.970474  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:02.971980  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:02.985407  9495 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:02.986845  9484 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:02.963295  9487 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:02.995337  9481 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:02.995497  9492 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:02.978860  9486 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:02.965170  9488 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:02.988351  9496 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:02.991592  8725 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:03.110249  9485 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.087110  9500 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:02.997344  9491 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.115674  9302 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.115931  9488 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.116500  9488 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.116247  9491 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.117035  9488 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.117640  9486 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.115226  9485 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.118230  9486 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.072646  9499 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.118784  9486 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.116309  9302 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.120151  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:03.120530  9499 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.119837  9485 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.120849  9488 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:03.089826  9497 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.121296  9491 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.089103  9484 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.121224  9485 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.121749  9497 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.122216  9484 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.122355  9485 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:03.123930  9484 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.123888  9497 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:02.991605  9483 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.124433  9485 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:03.062460  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69592.2us
I20260709 13:13:03.124944  9485 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:03.124723  9483 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.125399  9485 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:02.991876  9495 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.126919  9483 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:03.115281  9500 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.128561  9495 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:03.131155  8027 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:03.120760  9486 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.140619  9503 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.089959  9487 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.114317  9494 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:02.997648  9372 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:02.995858  9481 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.133167  9483 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.137395  9484 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.141158  9503 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:02.996317  9418 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:03.145437  9491 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.145917  9487 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.148702  9484 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.287657  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.293486  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:03.274242  9485 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:03.294821  8727 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:03.284066  9503 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:03.292593  8207 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
I20260709 13:13:03.281257  9489 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:03.147889  9481 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.299860  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.151881  9500 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:03.301983  9506 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.292037  9484 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.302584  9506 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.162228  9483 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.304850  9501 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:03.305307  9489 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.308370  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.307050  9503 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.294453  9495 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:03.281438  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:03.313078  8206 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
I20260709 13:13:03.313239  9508 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.290452  9491 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.293820  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:03.294174  8722 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:03.371245  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.311076  9496 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:03.381871  8207 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
I20260709 13:13:03.374478  9510 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.380491  9514 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:03.384717  9514 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.387748  9514 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:03.393456  8696 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:03.373322  9503 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.375886  9495 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.358147  9501 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.388962  9418 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:03.522686  9500 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.523105  9500 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.523161  9418 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.523612  9500 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.523833  9418 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.524189  9517 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.525022  9517 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.379066  8723 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:03.500862  9490 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:13:03.526461  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 133103us
I20260709 13:13:03.529611  8696 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:03.530652  8696 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:03.530738  9177 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.527176  9484 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.531322  9177 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.531426  9494 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.532163  9177 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.532202  9494 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.532876  9494 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.533725  9484 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:03.539076  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:03.541389  9490 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.543494  9490 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.424582  9487 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:03.396795  9514 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.520895  9502 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.547346  9513 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.500665  9483 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:03.555282  9515 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.555856  9515 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.557410  9481 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:03.460196  9506 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:03.558313  9491 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.554589  9418 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:03.560591  9483 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:03.558033  9502 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.566821  9481 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.570160  9489 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:03.578809  9496 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.579345  9496 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.562498  9514 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.427034  9510 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.583712  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.581323  9483 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.584968  9481 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:03.588562  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1003) took 2124 ms. Trace:
I20260709 13:13:03.590175  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.392207  9508 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.592823  9481 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:03.564160  9513 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.592445  9483 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:03.592420  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.593344  8130 rpcz_store.cc:276] 0709 13:13:01.463687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.463809 (+   122us) service_pool.cc:224] Handling call
0709 13:13:01.464852 (+  1043us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:01.480591 (+ 15739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:01.480620 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:01.482962 (+  2342us) log.cc:844] Serialized 26 byte log entry
0709 13:13:03.466315 (+1983353us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b180036c520. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:13:03.559744 (+ 93429us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:03.559794 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.559917 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:03.588182 (+ 28265us) raft_consensus.cc:1685] finished
0709 13:13:03.588202 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.588387 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13312,"raft.queue_time_us":15577,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":2686,"spinlock_wait_cycles":2383744,"thread_start_us":93170,"threads_started":1,"wal-append.queue_time_us":2100862}
I20260709 13:13:03.603739  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.604314  8206 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
I20260709 13:13:03.604457  9177 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.606647  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.607954  9501 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:03.610025  9503 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:03.611615  8131 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.106s	user 0.012s	sys 0.018s
W20260709 13:13:03.612048  8131 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.106s	user 0.014s	sys 0.018s
I20260709 13:13:03.612563  9514 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.612761  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 686) took 2109 ms. Trace:
I20260709 13:13:03.612967  9495 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:03.615403  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.616007  8058 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
I20260709 13:13:03.616146  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.617569  8054 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:13:03.617980  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.618638  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.619450  9522 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.620543  9513 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:03.620163  9522 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.623663  9372 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.623270  8696 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:03.600968  9496 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.596647  9518 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.628278  9508 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:03.629597  9510 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.624487  9372 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.637831  9411 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.638586  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:03.613159  8131 rpcz_store.cc:276] 0709 13:13:01.503224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.503352 (+   128us) service_pool.cc:224] Handling call
0709 13:13:01.503889 (+   537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:01.504262 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:01.504291 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:01.505340 (+  1049us) log.cc:844] Serialized 556 byte log entry
0709 13:13:03.612420 (+2107080us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:03.612464 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.612534 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:03.612578 (+    44us) raft_consensus.cc:1685] finished
0709 13:13:03.612588 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.612710 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":2106672,"threads_started":1,"wal-append.queue_time_us":609380,"wal-append.run_cpu_time_us":1816,"wal-append.run_wall_time_us":1002225}
I20260709 13:13:03.642521  8206 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.641557  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.624565  9502 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.637270  9177 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.646554  9499 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.639228  9302 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.646984  9494 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.647153  9499 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.649016  9494 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.638226  9515 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.639070  9517 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:03.651206  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 76680us
I20260709 13:13:03.651824  9522 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.652531  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:03.657125  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:03.644951  9510 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.659103  8209 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.659538  8206 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:03.657473  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.654438  9491 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:03.645807  9411 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.664252  9411 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.638481  9518 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.662585  8723 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:03.649145  9499 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:03.663339  9492 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.664041  9514 raft_consensus.cc:2955] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:03.666318  9492 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.670217  8696 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:03.683115  8059 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.206s	user 0.141s	sys 0.101s
I20260709 13:13:03.670652  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:03.683615  8059 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.206s	user 0.141s	sys 0.101s
I20260709 13:13:03.649128  9372 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.675051  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:03.664080  9491 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.693675  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 836) took 2219 ms. Trace:
I20260709 13:13:03.693768  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.682691  9498 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.695022  9491 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:03.694075  8059 rpcz_store.cc:276] 0709 13:13:01.472126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.475029 (+  2903us) service_pool.cc:224] Handling call
0709 13:13:01.475450 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:01.476126 (+   676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:01.476180 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:01.476828 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 13:13:03.690956 (+2214128us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:03.690998 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:03.691077 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:03.691134 (+    57us) raft_consensus.cc:1685] finished
0709 13:13:03.691148 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:03.691253 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2417,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":2647,"thread_start_us":2206665,"threads_started":1,"wal-append.queue_time_us":1191841,"wal-append.run_cpu_time_us":1669,"wal-append.run_wall_time_us":1003642}
I20260709 13:13:03.698011  8027 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:03.691668  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:03.700173  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.701406  8061 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 2: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:03.700265  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.700006  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.702929  8208 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:03.667438  9496 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.692307  9486 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.706395  9518 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:03.707521  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.712088  9486 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.712930  9492 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.709151  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:03.710985  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 56267.9us
I20260709 13:13:03.711282  8101 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:03.697528  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.708364  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.720391  9525 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.721364  8059 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:13:03.703087  9491 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.671196  9510 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.713055  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:03.737310  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.737916  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.739280  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.730957  9491 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.741344  8054 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:03.741499  9491 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.741622  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.699570  9500 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.695240  9498 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.753170  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.729255  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:03.767251  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 65107.2us
I20260709 13:13:03.769471  8693 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:03.743073  9522 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.743134  9525 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.733666  9496 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.769932  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.870882  9500 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.825932  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.829155  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.833585  8027 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:03.856215  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.825250  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:03.809182  8209 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
W20260709 13:13:03.825227  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 53704.2us
W20260709 13:13:03.926162  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 92669.5us
I20260709 13:13:03.930190  8693 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:03.769892  9505 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.927850  8057 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
I20260709 13:13:03.933921  9434 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.933734  7956 catalog_manager.cc:5697] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 1 to 2, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:03.934666  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.860534  9406 tablet_bootstrap.cc:492] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:03.934911  8696 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:03.833949  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:03.938012  8133 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:03.770287  9494 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:03.939482  9528 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.939630  9177 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:03.769220  9499 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.770332  9372 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.829905  8724 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:03.883726  9486 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.773187  9529 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:03.940559  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.876591  9498 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.877732  9510 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.854655  9461 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.854460  9515 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.966789  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:03.855010  9501 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.855705  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:03.941797  7956 catalog_manager.cc:5697] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 1 to 2, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:03.957640  9177 raft_consensus.cc:697] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:03.910248  9496 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.940207  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.919157  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:03.828928  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:03.882995  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.936596  9490 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:03.829428  9527 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:03.940029  9528 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.986790  8205 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.961007  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.960364  8209 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:03.965899  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:03.884104  9525 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.910543  9522 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.956768  9372 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:04.006494  9372 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:03.931339  9491 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.848680  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:03.966789  9529 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:03.972486  9514 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:03.972962  9357 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:03.976277  8102 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:03.978015  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:03.978240  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:03.980130  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:04.012645  8727 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:03.996402  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:04.014570  8209 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:03.986799  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.993398  8210 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:03.962803  9499 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:03.998421  9528 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.999055  9522 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:03.981634  9510 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:04.002697  8063 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:04.007997  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:03.986625  8693 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.567s (SENT) [suppressed 1 similar messages]
I20260709 13:13:04.010470  9406 tablet_bootstrap.cc:492] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:13:03.986864  9501 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.992580  9527 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.970291  9515 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:03.965268  9505 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:03.987380  9461 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:04.019621  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:04.086653  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:03.993229  9411 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:04.114290  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 138081us
I20260709 13:13:04.114368  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:04.118105  8209 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:04.020087  8058 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:04.047904  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:04.048143  8060 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:13:04.115281  8211 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:04.121332  8210 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:04.070809  9528 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:04.131980  8132 raft_consensus.cc:2410] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:13:04.135826  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:04.072491  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:04.138386  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:04.140444  8061 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:04.112790  9535 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:04.141203  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:04.067229  9510 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:04.016808  9514 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:04.146178  9406 ts_tablet_manager.cc:1403] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 2.637s	user 0.035s	sys 0.031s
I20260709 13:13:04.147277  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:04.149708  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:04.015913  9357 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:04.127729  9522 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:04.154693  8209 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:04.158430  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:04.133405  9499 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:04.157904  8133 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:04.142169  8723 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:04.172060  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:04.141813  8136 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:04.152092  9517 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:04.071569  9177 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:04.155009  8063 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:04.156507  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:04.233161  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:04.236609  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:04.257567  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 287706us
I20260709 13:13:04.263227  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:04.154151  9535 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:04.264201  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:04.233595  9532 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:04.249051  9494 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:04.269114  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:04.269559  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:04.283988  9434 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:04.256618  9491 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:04.253515  8203 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
I20260709 13:13:04.289745  8133 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:04.260334  9499 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:04.236063  9522 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:04.292212  9537 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:04.254189  9517 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:04.293996  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:04.156154  9529 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:04.284482  9372 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:04.303807  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:04.292768  9538 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:04.288987  9540 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:04.313818  9372 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:04.314693  9461 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:04.315826  9357 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:04.316509  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:04.289875  9532 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:04.292788  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:04.295789  9536 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:04.288407  9494 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:04.416796  9536 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:04.344710  9501 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:04.328504  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:04.337069  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:04.307986  9537 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:04.310261  9499 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:04.349200  9525 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:04.408864  9545 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:04.418525  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:04.332026  9515 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:04.328178  9538 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:04.420166  9543 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:04.419133  9536 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:04.357909  9540 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:04.662326  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:04.662554  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:04.663152  9522 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:04.610466  9545 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:04.663789  9494 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:04.663666  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:04.665475  9525 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:04.666848  9537 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:04.667635  9538 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:04.669898  9532 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:04.661373  9543 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:04.864197  8063 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:04.863390  8131 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:04.975374  8054 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:04.997730  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:04.999164  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1012) took 1150 ms. Trace:
I20260709 13:13:04.999274  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:04.999496  8137 rpcz_store.cc:276] 0709 13:13:03.849076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.997194 (+1148118us) service_pool.cc:224] Handling call
0709 13:13:04.999097 (+  1903us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:04.999854  9543 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:04.999998  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 855) took 1006 ms. Trace:
I20260709 13:13:05.000435  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:04.997782  9499 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:05.001031  8137 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
W20260709 13:13:04.996632  8103 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46944 dropped due to backpressure: service queue is full with 50 items
I20260709 13:13:05.003105  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.003608  8137 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:05.003958  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.004158  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1021) took 1132 ms. Trace:
I20260709 13:13:05.004467  8137 rpcz_store.cc:276] 0709 13:13:03.872031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.002617 (+1130586us) service_pool.cc:224] Handling call
0709 13:13:05.004111 (+  1494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.004701  9545 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.005218  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.005416  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.005913  8137 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:05.006371  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.006669  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.007109  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 852) took 1019 ms. Trace:
I20260709 13:13:05.000509  8054 rpcz_store.cc:276] 0709 13:13:03.994119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.999027 (+1004908us) service_pool.cc:224] Handling call
0709 13:13:04.999947 (+   920us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.007459  8063 rpcz_store.cc:276] 0709 13:13:03.988121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.006097 (+1017976us) service_pool.cc:224] Handling call
0709 13:13:05.007056 (+   959us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:04.996817  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1022) took 1124 ms. Trace:
I20260709 13:13:05.004701  9540 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.010509  8131 rpcz_store.cc:276] 0709 13:13:03.872265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.661969 (+789704us) service_pool.cc:224] Handling call
0709 13:13:04.996577 (+334608us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.011698  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.012399  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 854) took 1023 ms. Trace:
I20260709 13:13:05.012669  8054 rpcz_store.cc:276] 0709 13:13:03.989498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.011462 (+1021964us) service_pool.cc:224] Handling call
0709 13:13:05.012358 (+   896us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.013530  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.015223  8054 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:05.015697  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.016036  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.016757  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 853) took 1028 ms. Trace:
I20260709 13:13:05.010020  9510 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.014640  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.017328  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.017808  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 1847) took 1295 ms. Trace:
I20260709 13:13:05.009311  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.018004  8129 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:05.018667  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1014) took 1164 ms. Trace:
I20260709 13:13:05.018608  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1013) took 1165 ms. Trace:
I20260709 13:13:05.016768  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.019421  8138 rpcz_store.cc:276] 0709 13:13:03.853560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.992719 (+1139159us) service_pool.cc:224] Handling call
0709 13:13:05.018555 (+ 25836us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.019775  8059 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:05.018177  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.021031  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 848) took 1036 ms. Trace:
I20260709 13:13:05.021420  8054 rpcz_store.cc:276] 0709 13:13:03.988417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.015821 (+1027404us) service_pool.cc:224] Handling call
0709 13:13:05.016707 (+   886us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.021380  8059 rpcz_store.cc:276] 0709 13:13:03.984157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.020372 (+1036215us) service_pool.cc:224] Handling call
0709 13:13:05.020621 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.020768 (+   147us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:05.020793 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:05.020830 (+    37us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:05.020857 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.020908 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.020988 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.018172  8058 rpcz_store.cc:276] 0709 13:13:03.722675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.722815 (+   140us) service_pool.cc:224] Handling call
0709 13:13:03.737457 (+ 14642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:04.121714 (+384257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.017608 (+895894us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b540025f8d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:05.017742 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":649422080}
I20260709 13:13:05.024267  8723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 265) took 1248 ms. Trace:
I20260709 13:13:05.024900  8723 rpcz_store.cc:276] 0709 13:13:03.776215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.776392 (+   177us) service_pool.cc:224] Handling call
0709 13:13:03.776692 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:04.174811 (+398119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.023709 (+848898us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b540017a054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:05.024189 (+   480us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":818477824}
W20260709 13:13:05.027227  9177 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.043s	sys 0.053s
W20260709 13:13:05.027557  9177 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.043s	sys 0.053s
I20260709 13:13:05.019081  8129 rpcz_store.cc:276] 0709 13:13:03.853930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.017077 (+1163147us) service_pool.cc:224] Handling call
0709 13:13:05.018612 (+  1535us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.008630  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1020) took 1136 ms. Trace:
I20260709 13:13:05.031373  8137 rpcz_store.cc:276] 0709 13:13:03.871774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.006456 (+1134682us) service_pool.cc:224] Handling call
0709 13:13:05.008582 (+  2126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.003775  8103 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:46944 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1041)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 113)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 693)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 108)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1040)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 699)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1037)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 695)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1039)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 692)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 700)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 114)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 112)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 696)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 119)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 120)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 121)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 116)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 697)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 117)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1024)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1025)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1026)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1027)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1028)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1031)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1032)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1033)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1035)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1038)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1042)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 105)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 106)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 107)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 110)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 115)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 687)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 688)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 689)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 690)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 691)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 694)
I20260709 13:13:05.044286  8103 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:36048 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 693)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 108)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1040)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 699)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1037)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 695)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1039)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 692)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 700)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 114)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 112)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 706)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 696)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 705)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 119)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 703)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 120)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 121)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 704)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 116)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 697)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 701)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 709)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 702)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 707)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1024)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1025)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1026)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1027)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1028)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1031)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1032)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1033)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1035)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1038)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1042)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 105)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 106)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 107)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 110)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 115)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 687)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 688)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 689)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 690)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 691)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 694)
I20260709 13:13:05.278539  9517 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.278748  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.279325  9499 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:05.280598  8103 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:36048 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 108)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1040)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 699)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1037)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 695)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1039)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 692)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 700)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 114)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 112)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 706)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 696)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 705)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 119)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 703)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 120)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 121)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 704)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 116)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 697)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 701)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 709)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 711)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 702)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 707)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1024)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1025)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1026)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1027)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1028)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1031)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1032)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1033)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1035)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1038)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1042)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 105)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 106)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 107)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 110)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 115)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 687)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 688)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 689)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 690)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 691)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 694)
I20260709 13:13:05.283934  9499 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:05.286211  9499 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:05.287385  9499 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:05.287865  9499 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:05.290268  8138 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:05.290724  9499 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:05.310940  8061 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 1851) took 1545 ms (client timeout 2030 ms). Trace:
W20260709 13:13:05.311538  8061 rpcz_store.cc:269] 0709 13:13:03.765853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.766247 (+   394us) service_pool.cc:224] Handling call
0709 13:13:05.310688 (+1544441us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b540043dd54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:05.310835 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":601261312}
I20260709 13:13:04.329857  9411 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.326190  9499 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:05.326745  9499 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.328655  9499 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.329461  9499 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:05.329950  9499 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:05.284925  8103 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:36048 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 108)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1040)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 699)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1037)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 698)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 695)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1039)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 692)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 700)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 114)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 112)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 706)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 696)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 705)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 119)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 703)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 120)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 121)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 704)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 116)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 697)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 701)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 709)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 711)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 702)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 707)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1024)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1025)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1026)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1027)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1028)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1031)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1032)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1033)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1035)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1038)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1042)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 105)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 106)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 107)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 110)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 115)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 687)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 688)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 689)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 690)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 691)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 694)
I20260709 13:13:05.335490  9499 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:05.336246  8175 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.045s	user 0.023s	sys 0.044s
W20260709 13:13:05.336696  8175 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.046s	user 0.024s	sys 0.044s
I20260709 13:13:05.337610  9456 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.338178  9456 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:05.339692  8159 debug-util.cc:398] Leaking SignalData structure 0x7b08001fba20 after lost signal to thread 6940
I20260709 13:13:05.341578  9456 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:05.341251  9499 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:13:05.345783  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 55475.9us
I20260709 13:13:05.342581  9499 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:05.348078  8729 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:05.349470  9499 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:05.350440  8729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 431) took 1039 ms. Trace:
I20260709 13:13:05.342175  9456 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.350737  8729 rpcz_store.cc:276] 0709 13:13:04.310629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.312451 (+  1822us) service_pool.cc:224] Handling call
0709 13:13:04.312701 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.348456 (+1035755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.349868 (+  1412us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540041e6d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:05.350382 (+   514us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2380740864}
I20260709 13:13:05.352895  9553 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.352638  9456 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.354177  9554 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.354717  9554 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.355837  9456 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:05.353438  9553 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.360252  9554 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:05.361406  9553 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:05.361944  9553 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:05.367194  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.367847  8203 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:05.368759  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.369331  8203 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:13:05.374485  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.375048  8212 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:13:05.375826  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.376032  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.376783  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.377810  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.377974  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.378567  8208 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:05.379350  9553 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:05.379734  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.370107  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.380424  9553 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.380815  9553 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:05.379451  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.381146  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:05.381422  9553 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.382066  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.382419  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.382853  8208 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:13:05.381451  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 263) took 1662 ms. Trace:
I20260709 13:13:05.380774  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.385684  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.385264  8724 rpcz_store.cc:276] 0709 13:13:03.718781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.719048 (+   267us) service_pool.cc:224] Handling call
0709 13:13:03.737685 (+ 18637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:03.961335 (+223650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.381214 (+1419879us) spinlock_profiling.cc:243] Waited 88.1 ms on lock 0x7b54004d21d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:05.381397 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":202658560}
I20260709 13:13:05.383541  9557 raft_consensus.cc:3055] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:05.386780  9557 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:05.387068  8203 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:05.387583  8212 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:13:05.387759  9557 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:05.388571  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.388597  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:04.332096  9406 raft_consensus.cc:359] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:05.390166  9406 raft_consensus.cc:740] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:13:05.390453  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 60) took 2645 ms. Trace:
I20260709 13:13:05.390925  9406 consensus_queue.cc:260] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
W20260709 13:13:05.391630  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:05.390893  8056 rpcz_store.cc:276] 0709 13:13:02.745013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.669152 (+924139us) service_pool.cc:224] Handling call
0709 13:13:03.669456 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:03.697295 (+ 27839us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:03.701546 (+  4251us) log.cc:844] Serialized 14 byte log entry
0709 13:13:03.744868 (+ 43322us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:03.744939 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:03.744998 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.390206 (+1645208us) spinlock_profiling.cc:243] Waited 26.2 ms on lock 0x7b540043d354. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:05.390254 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.390396 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22867,"prepare.queue_time_us":256,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":23900,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":15692,"spinlock_wait_cycles":60211968,"thread_start_us":13180,"threads_started":1,"wal-append.queue_time_us":17739,"wal-append.run_cpu_time_us":989,"wal-append.run_wall_time_us":1015896}
I20260709 13:13:05.393500  9406 ts_tablet_manager.cc:1434] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 1.221s	user 0.017s	sys 0.003s
W20260709 13:13:05.393811  9406 ts_tablet_manager.cc:1467] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Tablet startup took 1235ms
I20260709 13:13:05.389274  8212 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:05.385627  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.389199  9557 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.394908  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:05.395061  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.395581  8208 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:05.396863  8212 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:05.394028  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.399168  8134 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
W20260709 13:13:05.399684  8134 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1009) took 1576 ms (client timeout 1687 ms). Trace:
W20260709 13:13:05.400090  8134 rpcz_store.cc:269] 0709 13:13:03.823251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.393757 (+1570506us) service_pool.cc:224] Handling call
0709 13:13:05.399602 (+  5845us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.400705  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.401304  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:05.401525  8055 raft_consensus.cc:2393] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 2: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:13:05.402045  8055 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 69) took 1786 ms (client timeout 1905 ms). Trace:
W20260709 13:13:05.402405  8055 rpcz_store.cc:269] 0709 13:13:03.615969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.400428 (+1784459us) service_pool.cc:224] Handling call
0709 13:13:05.401999 (+  1571us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.402123  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1040) took 1067 ms. Trace:
I20260709 13:13:05.406630  8134 rpcz_store.cc:276] 0709 13:13:04.334926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.401080 (+1066154us) service_pool.cc:224] Handling call
0709 13:13:05.402047 (+   967us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.407789  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:05.394140  9406 ts_tablet_manager.cc:1469] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Trace:
0709 13:12:59.281087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:59.281233 (+   146us) service_pool.cc:224] Handling call
0709 13:12:59.724533 (+443300us) ts_tablet_manager.cc:934] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
0709 13:13:00.525126 (+800593us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:13:00.636758 (+111632us) tablet_metadata.cc:665] Metadata flushed
0709 13:13:00.639893 (+  3135us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:13:00.756779 (+116886us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:13:00.883033 (+126254us) tablet_replica.cc:206] Creating consensus instance
0709 13:13:00.883526 (+   493us) ts_tablet_manager.cc:989] Replica registered.
0709 13:13:00.883679 (+   153us) inbound_call.cc:177] Queueing success response
0709 13:13:01.509316 (+625637us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:13:04.172469 (+2663153us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:13:04.317042 (+144573us) tablet_replica.cc:261] Starting instrumentation
0709 13:13:04.330287 (+ 13245us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":43715,"raft.queue_time_us":104,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"tablet-copy.queue_time_us":223635,"tablet-download-pool.queue_time_us":117054,"tablet-download-pool.run_cpu_time_us":1329,"tablet-download-pool.run_wall_time_us":44978,"thread_start_us":261318,"threads_started":2}
I20260709 13:13:05.409111  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.409757  8134 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:05.410372  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1037) took 1152 ms. Trace:
I20260709 13:13:05.397948  8212 raft_consensus.cc:2955] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:05.398543  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.411137  8134 rpcz_store.cc:276] 0709 13:13:04.263650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.408880 (+1145230us) service_pool.cc:224] Handling call
0709 13:13:05.410277 (+  1397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.416453  9303 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:05.416947  9303 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:05.419113  9303 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:05.419926  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.421125  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.422555  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.422868  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.424136  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.425304  9499 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.425748  9499 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:05.425580  8134 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:05.426249  9499 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.426084  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1005) took 1671 ms. Trace:
I20260709 13:13:05.427657  8130 rpcz_store.cc:276] 0709 13:13:03.754990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.755102 (+   112us) service_pool.cc:224] Handling call
0709 13:13:03.755367 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:04.000618 (+245251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:04.000647 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:04.000685 (+    38us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:04.000712 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.425869 (+1425157us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400381b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:05.425910 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.426013 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":563640064}
I20260709 13:13:05.428957  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.429268  9459 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.429435  8130 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:05.429697  9459 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:05.422211  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.430331  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.432116  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.433422  8130 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:05.432126  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.434480  9499 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.435655  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.436488  9499 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.431032  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1039) took 1119 ms. Trace:
W20260709 13:13:05.425787  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 48042.4us
I20260709 13:13:05.437033  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:05.412685  8205 consensus_meta.cc:287] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 1.420s	user 0.014s	sys 0.019s
I20260709 13:13:05.439023  8208 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:05.440205  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.440275  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.441404  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.441551  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.441941  8058 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:13:05.444046  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.445163  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.437018  8129 rpcz_store.cc:276] 0709 13:13:04.312067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.421990 (+1109923us) service_pool.cc:224] Handling call
0709 13:13:05.430977 (+  8987us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.446509  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.438980  8205 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:05.447530  8203 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:05.449725  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.424966  8176 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.438284  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.447474  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.450713  9511 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.450917  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.451131  9511 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:05.451365  9419 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.451566  9511 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.451874  9419 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:05.452272  9419 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.434499  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.453457  8130 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:05.454523  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:05.430109  9459 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.451336  8134 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
W20260709 13:13:05.455690  8205 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55942 (request call id 88) took 1603 ms (client timeout 1689 ms). Trace:
I20260709 13:13:05.455904  8026 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:05.446480  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.456327  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:05.457026  8133 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1036) took 1295 ms (client timeout 1580 ms). Trace:
I20260709 13:13:05.457631  9505 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.462193  9490 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:05.462308  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.462623  9505 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.462985  8054 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:05.464007  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.464349  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.464679  8204 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:05.464558  9553 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.465708  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.458930  8026 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:05.461159  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 1950) took 1306 ms. Trace:
I20260709 13:13:05.461424  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.463037  9505 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:05.466864  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.468070  8176 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:05.460094  8133 rpcz_store.cc:269] 0709 13:13:04.158271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.192575 (+ 34304us) service_pool.cc:224] Handling call
0709 13:13:05.453693 (+1261118us) spinlock_profiling.cc:243] Waited 39.1 ms on lock 0x7b5400409cd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:05.453813 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89913600}
I20260709 13:13:05.462710  9490 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.470914  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.471565  8063 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:05.472278  9505 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.467032  8211 rpcz_store.cc:276] 0709 13:13:04.153844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.153943 (+    99us) service_pool.cc:224] Handling call
0709 13:13:05.458939 (+1304996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.459090 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:05.459112 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:05.459148 (+    36us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:05.459176 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.459225 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.459664 (+   439us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54002c8b50. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:05.459755 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2966337792}
I20260709 13:13:05.460722  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.469321  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.473753  8061 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:13:05.473961  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.474539  8208 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:05.475054  8176 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.473081  8026 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:05.489116  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.489576  8026 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:05.466127  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.473592  9499 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.491714  9499 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:13:05.459276  8205 rpcz_store.cc:269] 0709 13:13:03.852331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.853049 (+   718us) service_pool.cc:224] Handling call
0709 13:13:05.455611 (+1602562us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.489670  8176 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:05.491043  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:05.492097  9499 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.490588  9557 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:05.459842  8203 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:05.476136  9303 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.498587  8176 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.513136  9419 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.513906  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.513825  9490 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:05.506481  9303 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:05.499086  9557 raft_consensus.cc:2764] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:13:05.514917  9303 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.515458  9557 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.516022  9490 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:05.516554  9490 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.506927  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:05.517086  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 92375.1us
I20260709 13:13:05.511595  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.519049  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:05.497680  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.523885  9559 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.524344  9559 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:05.524825  9559 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.530269  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:05.511693  9553 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:05.534302  9560 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.535413  9560 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.516878  9490 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:05.537977  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.517741  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:05.539120  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.535956  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.521771  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:05.558279  8057 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
W20260709 13:13:05.559798  9550 log.cc:927] Time spent T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Append to log took a long time: real 0.897s	user 0.002s	sys 0.004s
I20260709 13:13:05.557952  9560 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:05.560904  9560 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.542004  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1006) took 1786 ms. Trace:
I20260709 13:13:05.561285  9419 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:05.522778  8129 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
I20260709 13:13:05.575608  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.576164  8129 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:05.576988  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.577512  8129 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:05.578898  9517 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.579406  9517 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.578251  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.587437  9517 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.587973  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.589057  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.592314  8101 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:05.592594  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.593351  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 108) took 1037 ms. Trace:
I20260709 13:13:05.593680  8727 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:13:05.593654  8138 rpcz_store.cc:276] 0709 13:13:04.555777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.592357 (+1036580us) service_pool.cc:224] Handling call
0709 13:13:05.593285 (+   928us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.594401  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 429) took 1289 ms. Trace:
I20260709 13:13:05.590023  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.538291  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.596825  8101 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:05.563102  8135 rpcz_store.cc:276] 0709 13:13:03.755296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.755425 (+   129us) service_pool.cc:224] Handling call
0709 13:13:03.755663 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:03.768944 (+ 13281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:03.768987 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:03.769039 (+    52us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:03.769081 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.541727 (+1772646us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b54003c9bd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:05.541784 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.541939 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30071680}
I20260709 13:13:05.600884  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1024) took 1728 ms. Trace:
I20260709 13:13:05.601186  8135 rpcz_store.cc:276] 0709 13:13:03.872751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.600240 (+1727489us) service_pool.cc:224] Handling call
0709 13:13:05.600479 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.600618 (+   139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:05.600642 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:05.600677 (+    35us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:05.600703 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.600748 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.600837 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.594741  8727 rpcz_store.cc:276] 0709 13:13:04.305076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.305350 (+   274us) service_pool.cc:224] Handling call
0709 13:13:05.593357 (+1288007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.594153 (+   796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.594334 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.597782  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.607242  8138 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:05.565922  9490 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.609716  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.611610  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.611810  9411 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.603056  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1025) took 1730 ms. Trace:
I20260709 13:13:05.573184  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:05.612358  9532 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:13:05.613721  8055 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:13:05.612016  9537 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.614403  9537 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:05.614790  9537 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.614945  8205 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:05.613888  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:05.612609  8135 rpcz_store.cc:276] 0709 13:13:03.875883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.602003 (+1726120us) service_pool.cc:224] Handling call
0709 13:13:05.602227 (+   224us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.602789 (+   562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:05.602814 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:05.602853 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:05.602881 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.602930 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.603013 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.616909  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1028) took 1628 ms. Trace:
I20260709 13:13:05.578051  9459 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.617502  8135 rpcz_store.cc:276] 0709 13:13:03.988680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.616255 (+1627575us) service_pool.cc:224] Handling call
0709 13:13:05.616512 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.616649 (+   137us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:05.616672 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:05.616708 (+    36us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:05.616735 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.616781 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.616863 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.611853  8104 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:05.616818  8696 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.621222  9499 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.623222  9557 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:13:05.628011  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 109028us
I20260709 13:13:05.612268  9411 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:05.613190  8056 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.633339  9532 raft_consensus.cc:3055] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:05.612774  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.634661  9527 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.634729  9497 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:05.638005  9536 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.617240  9560 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:05.645689  9560 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:05.646317  9487 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:05.639365  8104 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:05.639945  8132 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1010) took 1816 ms (client timeout 1800 ms). Trace:
I20260709 13:13:05.640307  9459 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:05.615461  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:05.647541  8132 rpcz_store.cc:269] 0709 13:13:03.823560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.823667 (+   107us) service_pool.cc:224] Handling call
0709 13:13:05.639760 (+1816093us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b540036f454. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:05.639887 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":472571520}
I20260709 13:13:05.642321  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1031) took 1593 ms. Trace:
I20260709 13:13:05.643548  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 1855) took 1817 ms. Trace:
I20260709 13:13:05.652179  8135 rpcz_store.cc:276] 0709 13:13:04.048760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.641658 (+1592898us) service_pool.cc:224] Handling call
0709 13:13:05.641902 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.642044 (+   142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:05.642069 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:05.642105 (+    36us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:05.642131 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.642177 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.642262 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.654201  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1035) took 1521 ms. Trace:
I20260709 13:13:05.654629  8135 rpcz_store.cc:276] 0709 13:13:04.132856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.653314 (+1520458us) service_pool.cc:224] Handling call
0709 13:13:05.653632 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.653834 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:05.653871 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:05.653921 (+    50us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:05.653961 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.654027 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.654141 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.656004  9559 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.656509  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1032) took 1604 ms. Trace:
I20260709 13:13:05.656935  8134 rpcz_store.cc:276] 0709 13:13:04.052056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.650125 (+1598069us) service_pool.cc:224] Handling call
0709 13:13:05.656039 (+  5914us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.656190 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:05.656228 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:05.656271 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:05.656304 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.656361 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.656458 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.657958  9559 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.627341  9535 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.661964  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:05.662500  8727 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.617293  9502 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:13:05.641120  9411 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.631065  9543 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.666548  9543 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:05.666714  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:05.666920  9543 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.651558  8060 rpcz_store.cc:276] 0709 13:13:03.826512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.826633 (+   121us) service_pool.cc:224] Handling call
0709 13:13:03.829143 (+  2510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:04.121397 (+292254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.641649 (+1520252us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b540001bdd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:05.643496 (+  1847us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502651392}
I20260709 13:13:05.635934  8029 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.671536  8727 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:05.671598  8101 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:13:05.672343  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 134836us
I20260709 13:13:05.672508  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:05.672673  8101 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:05.673607  8059 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:05.673477  8029 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:05.674994  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1033) took 1622 ms. Trace:
I20260709 13:13:05.675298  8132 rpcz_store.cc:276] 0709 13:13:04.052347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.651320 (+1598973us) service_pool.cc:224] Handling call
0709 13:13:05.674717 (+ 23397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.674819 (+   102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:05.674832 (+    13us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:05.674856 (+    24us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:05.674873 (+    17us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.674901 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.674958 (+    57us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.643168  9532 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:05.627272  9303 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:05.682876  9510 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:05.680572  9499 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.688493  9510 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:05.684230  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.689845  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:05.690237  8729 raft_consensus.cc:1217] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:13:05.690783  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:05.691874  8104 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:05.693101  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:05.689013  9522 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.661434  9505 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:05.694909  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 105) took 1316 ms. Trace:
I20260709 13:13:05.676317  9517 raft_consensus.cc:3055] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:05.671075  9497 raft_consensus.cc:697] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:05.695187  8132 rpcz_store.cc:276] 0709 13:13:04.378544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.678984 (+1300440us) service_pool.cc:224] Handling call
0709 13:13:05.694457 (+ 15473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.694608 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:05.694632 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:05.694670 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:05.694713 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.694763 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.694860 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.697475  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:05.673714  9560 consensus_queue.cc:1059] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:13:05.678473  9527 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:05.693663  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1042) took 1357 ms. Trace:
I20260709 13:13:05.642949  9511 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.696017  9489 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:05.719100  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:05.720651  8029 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.699276  8104 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:05.688913  9510 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:05.696908  9534 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:05.695528  9517 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:05.708388  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.696118  9538 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.695732  9559 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.695899  9536 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.695372  9522 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:05.683045  9461 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:05.695693  9545 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.715487  9514 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:05.707883  8134 rpcz_store.cc:276] 0709 13:13:04.335809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.657894 (+1322085us) service_pool.cc:224] Handling call
0709 13:13:05.658151 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.658319 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:05.670324 (+ 12005us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:05.671224 (+   900us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:05.677371 (+  6147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:05.682584 (+  5213us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:05.693597 (+ 11013us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:05.718237  8210 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:13:05.761524  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 231319us
I20260709 13:13:05.688930  9499 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.722826  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.770689  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:05.736647  9495 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.771266  8730 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:05.771225  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 48430.2us
I20260709 13:13:05.736900  9538 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.715503  9505 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:05.726828  9497 consensus_queue.cc:237] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:05.728108  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:05.707367  8726 raft_consensus.cc:1240] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:05.721194  9489 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.779955  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.738276  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:05.746591  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 70025.5us
I20260709 13:13:05.780684  8727 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.747614  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:05.677413  9564 consensus_queue.cc:1059] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:13:05.760260  9545 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.772116  8104 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:05.761749  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:05.749245  9303 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:05.749011  9559 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:05.758232  9560 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.748299  9536 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:05.759496  9522 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.762236  8026 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:13:05.783018  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 62427.9us
W20260709 13:13:05.792304  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 59906.5us
I20260709 13:13:05.723424  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.779105  9514 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.737133  9411 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:05.782831  9489 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:05.719017  9532 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:05.795219  9545 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.774618  9495 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.778995  9538 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.809561  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.803864  9536 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.793792  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:05.817663  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:05.801035  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.819195  8104 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.821089  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:05.802158  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:05.823770  8029 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:05.824119  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.804630  8027 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:05.770864  9499 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:05.889997  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 70882.1us
I20260709 13:13:05.798369  9543 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:05.807446  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45282.4us
I20260709 13:13:05.813531  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:05.809607  9569 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.810242  9495 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.811148  8056 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:05.811811  9532 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.800243  9511 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:05.800469  9559 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.801740  9560 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:05.804837  8727 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:05.803412  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.806623  9534 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.806231  9514 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.893954  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:05.809435  9568 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:13:05.894654  8104 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:05.796852  9537 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.896268  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.807299  9490 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.889668  9564 raft_consensus.cc:3055] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:13:05.894189  8159 debug-util.cc:398] Leaking SignalData structure 0x7b08001aa120 after lost signal to thread 7995
W20260709 13:13:05.896214  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 75227.7us
I20260709 13:13:05.918087  8026 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:05.918711  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:05.919817  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:05.920859  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:05.921166  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.807699  9303 consensus_queue.cc:1059] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 13:13:05.893455  9535 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.919951  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:05.894765  9485 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.893563  9499 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.921371  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.899643  9506 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.923353  8132 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:05.930297  9506 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.930358  9416 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.930748  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:05.930928  9506 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.931092  9416 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.931668  9416 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.805629  9517 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:05.899397  9560 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.901356  9514 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.899236  9534 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:13:05.901017  8136 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1008) took 2107 ms (client timeout 1744 ms). Trace:
I20260709 13:13:05.916551  9569 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:05.721321  9459 raft_consensus.cc:2955] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:05.918779  9505 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:05.919442  9515 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.918303  9564 raft_consensus.cc:740] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:05.934569  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:05.895254  9461 consensus_queue.cc:1059] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:13:05.924302  9485 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:05.924153  9535 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.929806  9511 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:13:05.943879  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 149373us
W20260709 13:13:05.946897  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 142309us
I20260709 13:13:05.947256  9578 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:05.948415  9482 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:05.948921  9357 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:05.939452  9569 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.937920  9562 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.935750  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:05.938442  9517 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.963347  9490 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.956048  8727 raft_consensus.cc:2955] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:05.957522  9561 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.936962  9534 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.939178  9505 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:05.939013  9515 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:13:05.952033  8159 debug-util.cc:398] Leaking SignalData structure 0x7b0800045300 after lost signal to thread 8087
I20260709 13:13:05.952698  9511 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.944662  9303 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:05.951169  9485 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.940023  9575 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:05.952280  9535 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.960225  9563 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:05.937543  8136 rpcz_store.cc:269] 0709 13:13:03.792198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.809712 (+ 17514us) service_pool.cc:224] Handling call
0709 13:13:05.895237 (+2085525us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400043654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:05.899566 (+  4329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343899648}
I20260709 13:13:05.966451  9411 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:05.972007  9515 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:05.961632  9357 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:05.971436  9561 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:05.970760  9505 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.967003  9490 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:06.045837  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 107) took 1513 ms. Trace:
I20260709 13:13:05.974170  9511 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:05.958051  9482 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:06.034128  9563 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:13:06.039764  9550 log.cc:927] Time spent T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:13:05.980877  9578 raft_consensus.cc:697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:05.991708  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:05.967800  9573 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:06.047966  9561 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.033490  9575 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:06.038049  9574 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:06.067777  9575 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:06.068636  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 150623us
I20260709 13:13:06.069897  8026 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:06.070941  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1038) took 1780 ms. Trace:
I20260709 13:13:06.071355  8135 rpcz_store.cc:276] 0709 13:13:04.290973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.668030 (+1377057us) service_pool.cc:224] Handling call
0709 13:13:05.668266 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:05.668427 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:05.895113 (+226686us) log.cc:844] Serialized 14 byte log entry
0709 13:13:06.070625 (+175512us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:06.070672 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:06.070709 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.070770 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.070890 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":253792,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":3394,"raft.queue_time_us":144,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"thread_start_us":390318,"threads_started":2,"wal-append.queue_time_us":173941}
I20260709 13:13:06.073628  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 110) took 1472 ms. Trace:
I20260709 13:13:06.074026  8135 rpcz_store.cc:276] 0709 13:13:04.601492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.072951 (+1471459us) service_pool.cc:224] Handling call
0709 13:13:06.073211 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:06.073362 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:06.073385 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:06.073422 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:06.073450 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.073496 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.073581 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:06.075080  9515 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.079237  8730 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:06.080497  8176 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:06.080302  9545 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.081434  9480 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:06.081905  9480 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:06.083956  9560 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.085199  9536 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:06.086273  8135 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:06.086114  9560 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.087653  9560 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:06.088006  9560 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:06.088351  9560 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.088944  9559 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.089141  9489 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.090904  9559 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.055617  9490 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:06.058315  9563 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:06.073745  9510 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.112255  8026 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:06.094170  9490 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:06.113797  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 290091us
I20260709 13:13:06.124446  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:06.097599  9177 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.093144  9543 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.132406  8026 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:06.132689  9357 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:05.987560  9564 consensus_queue.cc:260] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:06.055452  8136 rpcz_store.cc:276] 0709 13:13:04.532752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.038731 (+1505979us) service_pool.cc:224] Handling call
0709 13:13:06.039079 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:06.039263 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:06.039291 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:06.039335 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:06.039370 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.040757 (+  1387us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.040885 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:06.094163  9574 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:06.131340  9559 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:06.067549  9578 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:06.074287  9573 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:06.130769  9177 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:06.151122  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:06.141837  9574 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.082844  9480 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:06.147219  9559 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:06.138856  9538 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.172338  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:06.153890  9564 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.151095  9489 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.203696  8205 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:48498
W20260709 13:13:06.206305  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 55226.5us
I20260709 13:13:06.208725  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:06.151805  8131 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:06.173449  9559 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:06.215651  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 262294us
I20260709 13:13:06.202538  9579 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:06.221026  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 687) took 1615 ms. Trace:
I20260709 13:13:06.224819  8136 rpcz_store.cc:276] 0709 13:13:04.604646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.145862 (+1541216us) service_pool.cc:224] Handling call
0709 13:13:06.168740 (+ 22878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:06.207445 (+ 38705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:06.208061 (+   616us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:06.210638 (+  2577us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:06.212102 (+  1464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.214283 (+  2181us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.220190 (+  5907us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:06.226558  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 688) took 1621 ms. Trace:
I20260709 13:13:06.226890  8136 rpcz_store.cc:276] 0709 13:13:04.604811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.225767 (+1620956us) service_pool.cc:224] Handling call
0709 13:13:06.226063 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:06.226228 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:06.226255 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:06.226294 (+    39us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:06.226329 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.226408 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.226510 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:06.228212  9303 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:06.228497  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 689) took 1623 ms. Trace:
I20260709 13:13:06.228353  9511 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.228741  9303 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:06.228937  8136 rpcz_store.cc:276] 0709 13:13:04.604961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.227739 (+1622778us) service_pool.cc:224] Handling call
0709 13:13:06.228017 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:06.228185 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:06.228211 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:06.228255 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:06.228285 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.228339 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.228449 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:06.254984  8027 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:06.221390  9510 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.258775  9495 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.260895  9495 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.230866  9303 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:06.222010  9580 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:06.264873  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:06.230297  9511 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:06.230420  9583 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:06.231190  9177 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.257747  8136 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:06.254217  9579 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:06.252203  9573 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.268662  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:06.304800  8137 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1041) took 1969 ms (client timeout 1787 ms). Trace:
W20260709 13:13:06.306857  8137 rpcz_store.cc:269] 0709 13:13:04.335396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.304554 (+1969158us) service_pool.cc:211] Skipping call since client already timed out
0709 13:13:06.304702 (+   148us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:13:06.308748  9583 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:13:06.308781  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 100151us
I20260709 13:13:06.314775  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 690) took 1705 ms. Trace:
I20260709 13:13:06.315769  8136 rpcz_store.cc:276] 0709 13:13:04.605098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.230006 (+1624908us) service_pool.cc:224] Handling call
0709 13:13:06.230260 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:06.280065 (+ 49805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.310588 (+ 30523us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:06.281874  9579 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.318142  8137 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:06.318372  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:06.318780  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.317390  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 694) took 1711 ms. Trace:
I20260709 13:13:06.319500  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 691) took 1714 ms. Trace:
I20260709 13:13:06.319841  8137 rpcz_store.cc:276] 0709 13:13:04.605248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.308271 (+1703023us) service_pool.cc:224] Handling call
0709 13:13:06.317769 (+  9498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:06.319275 (+  1506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.319451 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:06.319092  8136 rpcz_store.cc:276] 0709 13:13:04.605666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.316548 (+1710882us) service_pool.cc:224] Handling call
0709 13:13:06.316872 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:06.317057 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:06.317085 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:06.317127 (+    42us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:06.317162 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.317217 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.317322 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:06.281138  9505 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.327018  9586 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:06.314612  9482 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:06.314304  9585 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:06.320852  8029 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:06.347579  8029 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:06.334609  8159 debug-util.cc:398] Leaking SignalData structure 0x7b0800366440 after lost signal to thread 8162
I20260709 13:13:06.342624  9586 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:06.334913  9538 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.363445  9586 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.356287  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:06.362037  9416 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:06.358743  8159 debug-util.cc:398] Leaking SignalData structure 0x7b0800366360 after lost signal to thread 8163
I20260709 13:13:06.334228  9583 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:06.364317  9580 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:06.371377  9499 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.381379  9580 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:06.376462  9514 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.387001  9588 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:06.410040  8101 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:06.396951  9580 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.416657  9538 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.422848  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:06.423861  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:06.424338  8058 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:06.427484  8060 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:06.405633  9490 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.417548  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:06.429708  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:06.432518  8055 raft_consensus.cc:2393] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 2: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:06.433342  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.437441  9564 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:06.449218  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:06.450322  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:06.451380  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:06.452400  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:06.452316  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.456440  9505 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.451210  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:06.460330  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:06.463519  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.472136  9490 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.473075  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.475728  9574 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.478288  9554 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:06.478860  9554 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:06.479290  9554 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.479786  9574 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.443853  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.458740  8054 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.488232  9563 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.425408  9510 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:13:06.454679  9416 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:06.481038  8059 raft_consensus.cc:2955] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:06.477090  9456 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:06.458191  8134 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:06.431564  9560 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.503343  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:06.505991  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 365876us
I20260709 13:13:06.506459  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:06.506404  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.507762  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:06.508456  8136 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:06.509507  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:06.510604  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.510898  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:06.511853  8061 raft_consensus.cc:1240] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:06.512032  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:06.512974  8061 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:06.513085  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:06.513897  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:06.514161  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:06.514569  8060 raft_consensus.cc:2376] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2. Re-sending same reply.
I20260709 13:13:06.515282  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:06.515610  9532 consensus_queue.cc:1059] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:06.516343  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:06.516501  9540 raft_consensus.cc:3055] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:06.514593  9489 consensus_queue.cc:1059] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:13:06.517153  9540 raft_consensus.cc:740] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:06.517448  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:06.517731  9489 raft_consensus.cc:3055] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:06.518079  9489 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:06.518620  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:06.518276  9540 consensus_queue.cc:260] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.518929  9489 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.519937  8136 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:06.520301  9489 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.521090  8136 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:06.523237  8101 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:06.525426  9543 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:06.525094  9598 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:06.534509  9600 raft_consensus.cc:3055] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:06.534850  9600 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:06.535599  9600 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.522212  8136 raft_consensus.cc:2955] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:06.537933  8205 raft_consensus.cc:606] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:06.540328  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 48751.6us
I20260709 13:13:06.541502  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:06.542245  9597 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:06.542645  9597 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:06.543073  9597 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.542891  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:06.554178  8175 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:06.543826  9459 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:06.572624  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:06.448589  9569 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.437747  9579 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.493834  9562 consensus_queue.cc:1059] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.100s
I20260709 13:13:06.486711  9575 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.482436  9583 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:06.493501  9357 consensus_queue.cc:1059] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
W20260709 13:13:06.595027  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 238830us
I20260709 13:13:06.669534  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:06.663092  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 115) took 2059 ms. Trace:
I20260709 13:13:06.633692  8060 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:06.669188  8210 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:06.599562  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 92977.4us
I20260709 13:13:06.803171  8135 rpcz_store.cc:276] 0709 13:13:04.603420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.075134 (+1471714us) service_pool.cc:224] Handling call
0709 13:13:06.075379 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:06.086714 (+ 11335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.662814 (+576100us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b540008cad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:06.663026 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24691968}
W20260709 13:13:06.804256  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 109428us
I20260709 13:13:06.805703  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:06.582279  8058 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:06.671182  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:06.807036  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.807497  8058 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
W20260709 13:13:06.735311  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 54079.5us
I20260709 13:13:06.806249  8135 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:06.685676  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:06.670738  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:06.810097  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:06.603716  8055 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:06.814431  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:06.812107  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:06.815068  8026 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:06.815488  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:06.815601  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.813303  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:06.609596  9506 consensus_meta.cc:287] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.678s	user 0.003s	sys 0.017s
I20260709 13:13:06.574787  8063 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:06.819795  8057 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:06.818372  8026 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:06.610822  9522 consensus_meta.cc:287] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.803s	user 0.013s	sys 0.019s
I20260709 13:13:06.816250  8058 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.816148  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:06.574517  9416 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:06.608624  9534 consensus_meta.cc:287] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.618s	user 0.000s	sys 0.016s
I20260709 13:13:06.817458  8062 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 2: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:13:06.812800  8693 connection.cc:256] RPC call timeout handler was delayed by -1.2341573219998736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
I20260709 13:13:06.687776  8725 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:06.627331  9485 consensus_meta.cc:287] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.009s
I20260709 13:13:06.656486  9543 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 2 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:06.828194  9557 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:06.827979  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:06.598778  9303 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.226s
I20260709 13:13:06.753844  9525 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
W20260709 13:13:06.819145  8207 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 1965) took 1453 ms (client timeout 1774 ms). Trace:
I20260709 13:13:06.725411  9518 raft_consensus.cc:3055] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:06.606631  9499 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.821662  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.924220  8060 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:06.686657  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:06.688370  9545 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.606841  9514 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.695276  9536 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:06.688838  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:06.822482  8693 connection.cc:256] RPC call timeout handler was delayed by -1.112171282999725s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
W20260709 13:13:06.725253  9561 consensus_meta.cc:287] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.632s	user 0.005s	sys 0.005s
W20260709 13:13:06.930583  8693 connection.cc:256] RPC call timeout handler was delayed by -1.0995712829997046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21070000000000003s for a response
W20260709 13:13:06.930959  8693 connection.cc:256] RPC call timeout handler was delayed by -1.018571282999801s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2197s for a response
I20260709 13:13:06.930711  9561 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.931015  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:06.931701  8204 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:06.932581  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:06.932924  9561 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:06.738091  9560 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.933635  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:06.934262  9397 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:06.937703  8696 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:06.725767  8131 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 111) took 2123 ms (client timeout 1567 ms). Trace:
I20260709 13:13:06.821200  8073 tablet_copy_service.cc:342] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-a003b4584d3747a381522d67299aaed2 received from {username='slave'} at 127.0.0.1:37156
I20260709 13:13:06.836622  9537 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.838266  9569 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:06.839170  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.807232  8208 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 2: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:06.946991  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:06.947669  8208 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:06.947950  8101 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:06.949002  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:06.939369  8131 rpcz_store.cc:269] 0709 13:13:04.602637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.723119 (+1120482us) service_pool.cc:224] Handling call
0709 13:13:06.725557 (+1002438us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b54000d20d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:06.725679 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":786380032}
I20260709 13:13:06.818527  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:06.950479  9497 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:06.950531  9536 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:06.950984  9497 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:06.951093  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:06.824097  8026 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:06.596735  9559 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.847604  8058 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:06.922750  9518 raft_consensus.cc:740] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:06.808737  9580 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:06.959144  9518 consensus_queue.cc:260] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:06.850282  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:06.959651  9559 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.821825  9595 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:06.831581  9557 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.851989  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.853057  9556 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:06.859434  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.960805  8212 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:06.861677  9554 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.831303  9575 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.864631  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:06.864974  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:13:06.969928  8206 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 1968) took 1603 ms (client timeout 1798 ms). Trace:
W20260709 13:13:06.970158  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 105271us
I20260709 13:13:06.865511  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:06.826484  9585 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:06.866322  8057 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:06.866976  8159 debug-util.cc:398] Leaking SignalData structure 0x7b080030cdc0 after lost signal to thread 8236
I20260709 13:13:06.867856  9511 raft_consensus.cc:3055] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:06.868638  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1049) took 1178 ms. Trace:
W20260709 13:13:06.869478  8133 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 704) took 1827 ms (client timeout 1723 ms). Trace:
W20260709 13:13:06.978559  8133 rpcz_store.cc:269] 0709 13:13:05.042054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.869294 (+1827240us) service_pool.cc:211] Skipping call since client already timed out
0709 13:13:06.869402 (+   108us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:13:06.869985  9515 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:06.871429  9586 consensus_meta.cc:287] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.507s	user 0.004s	sys 0.005s
I20260709 13:13:06.840485  9538 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:06.827167  9522 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.827744  9534 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.824381  9563 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:06.876847  9600 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.879436  8210 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:06.880122  9571 raft_consensus.cc:3055] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:06.990890  9571 raft_consensus.cc:740] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:06.880215  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:06.991430  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:06.991806  8130 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 697) took 2385 ms (client timeout 2208 ms). Trace:
I20260709 13:13:06.992153  8210 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 2: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:06.993014  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:06.993682  9522 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:06.993912  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:06.882071  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:06.995245  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:06.950285  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:06.995958  8205 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 2: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:06.996702  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:06.996784  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:06.882146  9597 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.883435  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1027) took 2921 ms. Trace:
W20260709 13:13:06.889405  9535 consensus_meta.cc:287] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.856s	user 0.010s	sys 0.013s
I20260709 13:13:06.889590  8211 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.001852  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:06.890031  9501 raft_consensus.cc:3055] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:06.890218  9303 raft_consensus.cc:3055] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:06.894776  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.006867  9416 raft_consensus.cc:993] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: : Instructing follower 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:13:06.897244  9540 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.002477  8138 rpcz_store.cc:276] 0709 13:13:03.961619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.607885 (+1646266us) service_pool.cc:224] Handling call
0709 13:13:06.882734 (+1274849us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b58001c3e98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:13:06.882922 (+   188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:06.883088 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:06.883121 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:06.883163 (+    42us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:06.883219 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:06.883280 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:06.883383 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11775360}
I20260709 13:13:06.895448  8104 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:06.907804  7956 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34134 (request call id 262) took 5805 ms. Trace:
W20260709 13:13:06.917486  8207 rpcz_store.cc:269] 0709 13:13:05.366169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.685456 (+1319287us) service_pool.cc:224] Handling call
0709 13:13:06.819093 (+133637us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:07.008415  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 76921.4us
W20260709 13:13:07.008440  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 113074us
I20260709 13:13:07.009516  8104 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.009868  8693 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:06.601204  9459 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.229s
I20260709 13:13:06.821507  9506 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:06.941289  8073 tablet_copy_service.cc:434] P 84d775d1767e4cc4aa7b9c627d8f0fb8: ending tablet copy session 965e59f272f143449b18f65db4e61d98-a003b4584d3747a381522d67299aaed2 on tablet a003b4584d3747a381522d67299aaed2 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:13:06.941398  9561 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:06.950309  8101 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:06.951346  9497 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:06.955981  9536 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:06.956044  9537 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.984342  9586 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:06.956818  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:07.015563  8135 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:06.957149  8208 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:06.957511  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 133566us
I20260709 13:13:06.957177  9514 raft_consensus.cc:3055] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:06.957962  9499 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:13:07.018860  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 68595.6us
I20260709 13:13:06.849946  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.954068  7961 catalog_manager.cc:5697] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 2, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:06.975453  8060 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:13:06.976657  9580 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:07.026398  9554 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:06.976991  9511 raft_consensus.cc:740] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:07.014166  9522 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.026607  8693 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:06.970293  8206 rpcz_store.cc:269] 0709 13:13:05.366978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.864402 (+1497424us) service_pool.cc:224] Handling call
0709 13:13:06.969878 (+105476us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:06.977548  9595 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:13:06.977624  8159 debug-util.cc:398] Leaking SignalData structure 0x7b0800318f40 after lost signal to thread 8237
I20260709 13:13:06.830235  9485 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:06.981496  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:06.962023  9556 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:06.983333  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:07.037479  8209 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:06.986513  7960 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37774 (request call id 136) took 5770 ms. Trace:
I20260709 13:13:06.978024  8134 rpcz_store.cc:276] 0709 13:13:05.690039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.782099 (+ 92060us) service_pool.cc:224] Handling call
0709 13:13:06.868485 (+1086386us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b5400001754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:06.868584 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250393344}
W20260709 13:13:06.992172  8130 rpcz_store.cc:269] 0709 13:13:04.606119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.508144 (+902025us) service_pool.cc:224] Handling call
0709 13:13:06.880081 (+1371937us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b58001c3e98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:06.991750 (+111669us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5660928}
I20260709 13:13:06.997967  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.998270  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.998914  8056 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:06.999301  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:06.999531  8137 raft_consensus.cc:2393] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:07.018677  9499 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:06.999428  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:07.000375  9534 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:07.048026  8132 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:07.015202  9515 consensus_queue.cc:1059] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:13:07.003676  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:07.050961  8136 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:07.032209  9579 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:07.051769  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:06.953286  8054 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:07.004959  9501 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:07.005159  9303 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:06.960780  9543 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.007412  9416 raft_consensus.cc:1081] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Signalling peer 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:13:07.012868  8104 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.012904  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:06.956543  9397 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:07.016626  8026 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.004925  9518 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.048559  9499 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.005380  9563 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.008601  9538 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.024050  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:07.010809  9567 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.031097  9582 raft_consensus.cc:1064] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:13:07.034749  8061 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:07.004453  9535 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.036178  8101 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.008177  7956 rpcz_store.cc:276] 0709 13:13:01.096415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.100801 (+  4386us) service_pool.cc:224] Handling call
0709 13:13:06.902256 (+5801455us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:04.315431 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:04.315986 (+   555us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:04.316019 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:13:04.316032 (+    13us) tablet.cc:662] Decoding operations
0709 13:13:04.316420 (+   388us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:04.316471 (+    51us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:04.316497 (+    26us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:13:04.316797 (+   300us) tablet.cc:701] Row locks acquired
0709 13:13:04.316816 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:13:04.316945 (+   129us) write_op.cc:270] Start()
0709 13:13:04.317045 (+   100us) write_op.cc:276] Timestamp: P: 1783602784316903 usec, L: 0
0709 13:13:04.317061 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:13:04.546248 (+229187us) log.cc:844] Serialized 913 byte log entry
0709 13:13:05.556913 (+1010665us) spinlock_profiling.cc:243] Waited 6.65 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:05.688737 (+131824us) op_driver.cc:464] REPLICATION: finished
0709 13:13:05.813452 (+124715us) write_op.cc:301] APPLY: starting
0709 13:13:05.813526 (+    74us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:05.813634 (+   108us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:05.813652 (+    18us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:05.936314 (+122662us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:05.944952 (+  8638us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:13:05.954713 (+  9761us) write_op.cc:312] APPLY: finished
0709 13:13:05.961842 (+  7129us) log.cc:844] Serialized 29 byte log entry
0709 13:13:05.968514 (+  6672us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:05.971231 (+  2717us) write_op.cc:454] Released schema lock
0709 13:13:05.973389 (+  2158us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:06.039159 (+ 65770us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:06.039189 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1241,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":124487,"apply.run_cpu_time_us":2610,"apply.run_wall_time_us":225670,"num_ops":2,"prepare.queue_time_us":282934,"prepare.run_cpu_time_us":12061,"prepare.run_wall_time_us":309744,"raft.queue_time_us":76177,"raft.run_cpu_time_us":38681,"raft.run_wall_time_us":86953,"replication_time_us":1371524,"spinlock_wait_cycles":15289600,"thread_start_us":453281,"threads_started":4,"wal-append.queue_time_us":115734}]]}
I20260709 13:13:07.016573  9514 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:07.038465  7960 rpcz_store.cc:276] 0709 13:13:01.210385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:01.210598 (+   213us) service_pool.cc:224] Handling call
0709 13:13:06.980763 (+5770165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43836,"reported_tablets":12}
I20260709 13:13:07.040923  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:07.078886  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:07.079566  8208 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 2: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:07.042407  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:06.998697  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:07.030731  9545 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.024137  9536 raft_consensus.cc:697] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:07.080085  8130 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:07.033826  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:07.031507  9595 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:07.054149  8057 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:07.016608  9561 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.033879  9511 consensus_queue.cc:260] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.058203  9479 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.058820  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:07.058918  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:07.059638  8175 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:07.064292  8207 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 2: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:07.050251  8203 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:07.067437  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50878.4us
I20260709 13:13:07.067530  8209 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:07.025379  9487 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:07.117511  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 106) took 2673 ms. Trace:
I20260709 13:13:07.117903  8132 rpcz_store.cc:276] 0709 13:13:04.444362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.931614 (+1487252us) service_pool.cc:224] Handling call
0709 13:13:05.931866 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:07.048384 (+1116518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:07.117314 (+ 68930us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001ab154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:07.117463 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2566611328}
I20260709 13:13:07.067823  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:07.118988  8136 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:07.045533  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:07.067389  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 54579.1us
I20260709 13:13:07.126086  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:07.126667  8209 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.126842  8104 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.128283  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.079090  8058 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:07.041858  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:07.002558  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:07.081568  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:07.070966  8159 debug-util.cc:398] Leaking SignalData structure 0x7b080034a4c0 after lost signal to thread 8754
I20260709 13:13:07.084036  9514 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.084882  9303 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.086005  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:07.086867  9525 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.090641  8129 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:07.074366  9579 raft_consensus.cc:2764] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:13:07.083525  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 47417.8us
I20260709 13:13:07.117532  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:07.100148  9561 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:07.101859  8055 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.102958  9536 consensus_queue.cc:237] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.125958  9553 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:07.102643  9501 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.081314  9532 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.069877  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:07.116022  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 56470.8us
I20260709 13:13:07.158054  9538 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:13:07.123804  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.071615  9515 consensus_queue.cc:1243] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:13:07.133031  8059 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:07.155485  8211 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.162177  8209 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:07.164083  8062 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:07.164630  8058 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:07.229234  8058 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.096911  9595 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.171156  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.055892  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:07.249414  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 222862us
I20260709 13:13:07.174911  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:07.250244  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 116916us
I20260709 13:13:07.250698  8137 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:07.250952  8104 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.251421  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.251531  8693 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.251973  8137 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.194643  8061 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:07.194482  8133 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:07.203155  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:07.121362  8208 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.159464  9397 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.165817  9511 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:07.254671  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:07.206506  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.260618  8059 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.179268  9557 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.211732  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:07.213157  9499 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.213485  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:07.215698  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.252422  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:07.253311  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50263.8us
I20260709 13:13:07.104176  9556 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.181066  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:07.261849  8059 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:07.262778  8058 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.180622  9502 consensus_queue.cc:1059] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
I20260709 13:13:07.264060  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:07.261868  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 148299us
I20260709 13:13:07.195072  9514 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.200721  9553 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:07.253301  9506 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:07.249342  9501 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.286517  8058 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.261619  9554 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.201700  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.254395  9532 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.260270  9515 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:07.287215  9556 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:07.385114  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 132760us
I20260709 13:13:07.315092  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.315760  8056 raft_consensus.cc:2393] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 2: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:07.316679  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:07.327275  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.332031  9499 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:07.369902  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:07.284847  9416 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:07.387960  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.285713  9586 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:07.336436  9553 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.191535  9582 consensus_queue.cc:237] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.401137  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.402673  8057 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:07.403826  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.376688  9485 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:07.414163  9502 raft_consensus.cc:3055] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:07.414572  9502 raft_consensus.cc:740] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:07.415386  9502 consensus_queue.cc:260] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.416656  9502 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.418664  8136 raft_consensus.cc:2393] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 2: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:07.394265  9479 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:07.389907  9543 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.423063  8104 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.411219  8204 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 2: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:07.396098  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:07.436245  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.408313  8055 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.439055  8137 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:07.406522  9416 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.447501  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.428267  9479 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:07.455943  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:07.463629  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:07.468755  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 217300us
I20260709 13:13:07.469491  8693 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.469491  8101 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.461064  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:07.472358  9571 consensus_queue.cc:260] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.473016  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:07.473351  9571 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.473657  9557 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:07.473779  8134 raft_consensus.cc:2393] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:13:07.474514  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 196826us
I20260709 13:13:07.475775  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:07.451413  9562 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:07.467389  8133 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:07.420295  9538 consensus_queue.cc:1059] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:13:07.461580  9479 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.489187  8104 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.487346  8693 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.491866  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.501750  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.502951  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:07.491945  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.504129  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.493022  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:07.496280  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.488262  8175 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:07.504856  8131 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.505532  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:07.506186  9563 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:07.506595  9563 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:07.506958  9563 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:07.507898  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.483907  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.510504  9557 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.510950  8205 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:07.511096  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.455240  9525 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.512571  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.477066  9532 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:07.477345  9567 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.482999  9562 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:07.498565  9586 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.520078  8138 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:07.518785  9487 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:07.524531  9515 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.527170  8729 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.524551  9567 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:13:07.528283  9397 consensus_queue.cc:1059] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:13:07.531558  9487 raft_consensus.cc:697] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:07.532315  9532 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:13:07.535094  8212 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.027s	sys 0.029s
W20260709 13:13:07.535413  8212 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.028s	sys 0.029s
I20260709 13:13:07.535375  9487 consensus_queue.cc:237] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.535662  9532 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.533242  9416 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:07.545356  8054 raft_consensus.cc:1240] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:07.534628  9497 consensus_meta.cc:287] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.516s	user 0.003s	sys 0.007s
W20260709 13:13:07.555837  9522 consensus_meta.cc:287] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.529s	user 0.005s	sys 0.002s
I20260709 13:13:07.558521  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.559211  9562 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.568035  8134 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:07.549474  9595 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:07.572579  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 56410.2us
W20260709 13:13:07.576321  9537 consensus_meta.cc:287] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.003s
I20260709 13:13:07.589124  9537 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:07.589282  9540 consensus_meta.cc:287] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.582s	user 0.009s	sys 0.000s
I20260709 13:13:07.594700  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:07.594933  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:07.595937  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:07.575078  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.596637  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:07.596939  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:07.597230  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.597729  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:07.597991  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:07.599828  9562 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.600075  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:07.600345  8058 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:07.600900  8054 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:07.601177  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:07.602046  8054 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:07.602890  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.603880  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:07.606676  9511 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:13:07.608034  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:07.608505  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.609386  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:07.598367  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.598680  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:07.569932  9522 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.598794  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:07.599459  8136 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:07.542119  9543 consensus_queue.cc:1059] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 13:13:07.618270  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.622781  8130 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:07.662773  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:07.661919  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 43771.6us
I20260709 13:13:07.666010  8029 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.578605  9614 raft_consensus.cc:3055] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:07.596184  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:07.662281  9624 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:13:07.623423  9556 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.672497  9614 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:07.673249  8723 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 LEARNER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:13:07.673503  9614 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.668953  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.670578  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:07.665205  9580 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.670578  9506 consensus_queue.cc:1059] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:07.664253  9416 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:13:07.671530  9595 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:07.564226  9497 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.679425  8029 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:07.682634  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 155001us
I20260709 13:13:07.665673  9557 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.687687  8026 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.688762  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:07.679870  9579 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.661567  9511 consensus_queue.cc:1059] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.020s
W20260709 13:13:07.691125  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 172791us
I20260709 13:13:07.694453  8026 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.694670  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:07.696215  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.696966  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:07.697398  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:07.696568  8026 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.697664  8209 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:07.698511  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 180266us
W20260709 13:13:07.700765  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.708889  8209 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:07.709993  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:07.713754  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.678186  9571 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:13:07.712136  8029 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.715695  8104 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.695405  8134 raft_consensus.cc:2393] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 2: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:07.718888  8723 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:07.719332  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:07.719698  9545 consensus_meta.cc:287] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.000s
I20260709 13:13:07.719962  8134 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.690047  9515 consensus_queue.cc:1059] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:13:07.720165  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.721385  8725 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:07.722394  9553 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.724152  9497 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:07.724468  9553 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:07.725098  9553 raft_consensus.cc:3055] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:07.725422  9553 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:07.727397  8205 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:13:07.720165  9545 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.728566  9553 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.729724  9553 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.714916  9587 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.730262  8134 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:07.717927  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:07.696627  9582 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:07.731680  9582 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:07.732635  9582 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.712147  9595 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.733736  9595 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.714852  9559 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.691795  9580 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.719784  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.746569  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:07.703099  9564 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:07.748484  8104 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:07.749305  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:07.750092  9630 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.750574  9502 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.726163  9479 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.710403  8131 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:07.710577  9487 consensus_queue.cc:1059] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:13:07.755038  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:07.760610  8026 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:07.763262  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.747184  8209 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.770310  8133 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:07.761605  9534 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:07.771602  8134 raft_consensus.cc:2410] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:13:07.812568  9534 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:07.813338  8134 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b186be823643475982e269b73d886a42"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:13:07.813503  9534 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.814162  8134 raft_consensus.cc:606] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:07.814568  9534 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.814728  8134 raft_consensus.cc:614] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:07.816335  8134 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.817087  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 729) took 1307 ms. Trace:
I20260709 13:13:07.817517  8134 rpcz_store.cc:276] 0709 13:13:06.509452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.815705 (+1306253us) service_pool.cc:224] Handling call
0709 13:13:07.816089 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:07.816789 (+   700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:07.817024 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:07.771714  8730 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.695379  9556 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:13:07.774261  8207 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:07.819901  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.820928  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.822587  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.823345  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 129) took 1074 ms. Trace:
I20260709 13:13:07.823835  8134 rpcz_store.cc:276] 0709 13:13:06.748939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.822307 (+1073368us) service_pool.cc:224] Handling call
0709 13:13:07.822661 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:07.822885 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:07.822926 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:07.822983 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:07.823033 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:07.823126 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:07.823283 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:07.825448  8131 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:07.826026  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 131) took 1033 ms. Trace:
I20260709 13:13:07.826169  8134 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:07.827782  9479 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:07.828809  9618 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:13:07.777616  9518 consensus_meta.cc:287] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.708s	user 0.007s	sys 0.003s
I20260709 13:13:07.830148  9518 consensus_queue.cc:1059] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:07.830379  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:07.831195  9518 raft_consensus.cc:3055] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:07.831605  9518 raft_consensus.cc:740] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:07.831799  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:07.832468  9518 consensus_queue.cc:260] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.833737  9518 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.776664  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.774417  9564 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:07.834653  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58040us
I20260709 13:13:07.834720  9629 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:07.835278  9629 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:07.835139  9564 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.836833  9545 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.836344  9629 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.773489  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 1994) took 1105 ms. Trace:
I20260709 13:13:07.837888  9629 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:07.783649  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:07.774628  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:07.808645  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.735673  9563 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.752873  9562 consensus_queue.cc:1059] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
W20260709 13:13:07.811264  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50781.6us
I20260709 13:13:07.811489  9630 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.833913  9556 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.754493  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.843562  8209 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:07.844766  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:07.876804  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 45109.1us
I20260709 13:13:07.877972  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:07.845295  9627 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.838018  8203 rpcz_store.cc:276] 0709 13:13:06.668075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.686420 (+ 18345us) service_pool.cc:224] Handling call
0709 13:13:07.773270 (+1086850us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540019dcd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:07.773424 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":232482560}
I20260709 13:13:07.845849  8062 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.847136  8205 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.847612  9563 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:07.847159  9517 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:07.842517  9580 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:07.849722  9535 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:07.840878  8131 rpcz_store.cc:276] 0709 13:13:06.793047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.824847 (+1031800us) service_pool.cc:224] Handling call
0709 13:13:07.825151 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:07.825792 (+   641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:07.825977 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:07.851493  8063 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:13:07.849762  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.855614  8724 raft_consensus.cc:2955] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:07.856902  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:07.893965  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 733) took 1019 ms. Trace:
W20260709 13:13:07.858835  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.894330  8136 rpcz_store.cc:276] 0709 13:13:06.874668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.453801 (+579133us) service_pool.cc:224] Handling call
0709 13:13:07.892819 (+439018us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b540035cad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:07.893909 (+  1090us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282941696}
I20260709 13:13:07.859429  9622 consensus_queue.cc:1059] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:07.866137  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:07.870883  9515 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:07.902108  8133 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 2 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.902516  8132 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:07.875690  8026 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:07.879902  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.904214  8026 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.877642  8102 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:07.812368  9522 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:07.881932  8211 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:07.912106  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 104) took 1005 ms. Trace:
I20260709 13:13:07.899834  8131 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:07.914462  9621 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:07.914716  9501 raft_consensus.cc:3055] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:07.915122  9501 raft_consensus.cc:740] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:07.914219  9582 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:13:07.916154  8131 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:07.915930  9501 consensus_queue.cc:260] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.917109  9522 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:07.917343  8131 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.918013  9532 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.911702  9575 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:07.902572  9487 raft_consensus.cc:2764] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:13:07.917126  9621 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:07.904772  9515 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.919508  8721 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.912461  8212 rpcz_store.cc:276] 0709 13:13:06.906460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.968351 (+ 61891us) service_pool.cc:224] Handling call
0709 13:13:07.017605 (+ 49254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:07.017787 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:07.535879 (+518092us) log.cc:844] Serialized 14 byte log entry
0709 13:13:07.571617 (+ 35738us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:07.571699 (+    82us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:07.571755 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:07.571842 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:07.911963 (+340121us) spinlock_profiling.cc:243] Waited 49 ms on lock 0x7b5400189a50. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:07.912060 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":483977,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":128,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":132,"spinlock_wait_cycles":112751616,"thread_start_us":538275,"threads_started":2,"wal-append.queue_time_us":35279}
I20260709 13:13:07.882563  8175 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:07.884025  8203 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.922700  9515 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:07.923136  9562 raft_consensus.cc:2764] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:13:07.923815  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.923995  9485 raft_consensus.cc:3055] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:07.924381  9485 raft_consensus.cc:740] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:07.925132  8730 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.884893  8104 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.885347  9630 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.895375  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.896257  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.884828  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 132) took 1091 ms. Trace:
W20260709 13:13:07.946058  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.898391  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:07.899802  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.899916  8063 raft_consensus.cc:606] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:13:07.886134  9517 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:07.900386  9563 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:07.931559  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.893399  9580 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:07.877578  8696 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:07.918802  9575 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:07.897528  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:07.941025  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.918172  8131 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:36048
I20260709 13:13:07.947707  8029 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:07.949206  8211 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.963704  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:07.963932  8029 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.950716  9593 consensus_queue.cc:1059] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:07.965502  9560 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:07.966063  9593 raft_consensus.cc:3055] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:07.966468  9593 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:07.967404  9575 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.967890  9571 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.967439  9593 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.970992  8026 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.971557  9593 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.971659  9597 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:07.972296  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1026) took 4035 ms. Trace:
I20260709 13:13:07.972637  9554 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:07.973120  9554 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:07.952970  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:07.946812  8063 raft_consensus.cc:614] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:07.975235  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:07.949021  9532 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:13:07.976378  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:07.950866  9563 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.977234  9535 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:07.977617  9535 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:07.977686  8721 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:07.966527  9560 raft_consensus.cc:3055] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:07.979030  9621 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:07.979030  9560 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:07.979498  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:07.979990  9560 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.981122  9560 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.947218  9583 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:07.982749  9587 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:07.984936  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:07.978598  9501 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:07.930265  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:07.933053  9515 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:13:07.955803  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 78284.6us
I20260709 13:13:07.921728  9564 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:07.958695  9609 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:13:07.958926  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 61473us
I20260709 13:13:07.999809  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:07.959409  9372 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:07.959090  9580 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:07.925160  9485 consensus_queue.cc:260] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:07.964890  9582 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:07.964947  8211 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:07.978538  9563 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:07.988487  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 58304.3us
I20260709 13:13:08.011251  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:07.946362  8134 rpcz_store.cc:276] 0709 13:13:06.793798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.825418 (+1031620us) service_pool.cc:224] Handling call
0709 13:13:07.825753 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:07.826683 (+   930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:07.884763 (+ 58080us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:07.920635  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:07.947139  9303 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:08.014865  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:08.006973  9479 consensus_queue.cc:1059] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:13:08.015622  8135 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:08.015895  9479 raft_consensus.cc:3055] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:08.016244  9479 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:08.016444  8726 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:08.017212  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:08.016990  9479 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:08.017982  8721 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:07.996142  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:07.987257  9517 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.024022  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 752) took 1069 ms. Trace:
I20260709 13:13:08.024595  8133 rpcz_store.cc:276] 0709 13:13:06.954437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.903191 (+948754us) service_pool.cc:224] Handling call
0709 13:13:07.903445 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:07.921058 (+ 17613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:07.921085 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:07.921120 (+    35us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:07.921147 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.023803 (+102656us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b54001f09d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:08.023849 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:08.023961 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40123136}
I20260709 13:13:07.955715  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:08.003530  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:08.002388  9564 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:07.970993  9557 raft_consensus.cc:3055] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:08.006371  9500 raft_consensus.cc:3055] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:08.042326  9500 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:08.004048  9583 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:13:08.043390  9500 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:08.044415  9583 raft_consensus.cc:3055] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:08.044592  9500 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.044817  9583 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:08.045323  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 746) took 1125 ms. Trace:
I20260709 13:13:08.050101  9564 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.050792  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55942 (request call id 101) took 1145 ms. Trace:
I20260709 13:13:08.009482  9609 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:08.009862  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:08.023722  9586 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:08.064424  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:08.064976  8130 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:08.065843  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.010813  8063 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:13:07.993682  8131 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:07.997330  9614 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:07.946439  9627 consensus_queue.cc:1059] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:13:08.019615  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:08.080422  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 60883.7us
I20260709 13:13:08.003345  9629 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:13:07.972682  8129 rpcz_store.cc:276] 0709 13:13:03.937245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:05.606775 (+1669530us) service_pool.cc:224] Handling call
0709 13:13:06.408569 (+801794us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b58001c3e98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:13:06.416629 (+  8060us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:07.179390 (+762761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:07.972053 (+792663us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54001bb054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:07.972234 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1563187328}
I20260709 13:13:07.997433  8205 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.026170  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:08.027998  9563 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:07.973960  9554 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:08.024153  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.033897  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:07.948971  9534 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:13:08.040830  8102 leader_election.cc:336] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Timed out: RequestConsensusVote RPC to 127.6.198.68:44573 timed out after 1.720s (SENT)
I20260709 13:13:08.087008  8176 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:08.045738  8132 rpcz_store.cc:276] 0709 13:13:06.919527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.853596 (+934069us) service_pool.cc:224] Handling call
0709 13:13:07.902354 (+ 48758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:07.902953 (+   599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.045151 (+142198us) spinlock_profiling.cc:243] Waited 48.4 ms on lock 0x7b540035cfd0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:08.045271 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111315840}
W20260709 13:13:08.086961  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 53138.1us
I20260709 13:13:07.993983  9630 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:08.088156  8060 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.090008  8056 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.045704  9583 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.089780  9609 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:08.090490  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:08.090843  9583 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:08.091068  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:08.093966  9620 raft_consensus.cc:3055] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:08.051162  8204 rpcz_store.cc:276] 0709 13:13:06.905654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:06.933389 (+ 27735us) service_pool.cc:224] Handling call
0709 13:13:08.050590 (+1117201us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b540019e6d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:08.050729 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":922156288}
I20260709 13:13:08.018297  9479 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:08.061434  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.091161  8060 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:07.918040  9522 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.062335  8210 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:08.079696  9603 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:08.079887  9517 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:08.076660  8131 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.062556  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:08.018973  9582 raft_consensus.cc:697] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:08.068913  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:08.070082  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1037) took 1084 ms. Trace:
I20260709 13:13:08.072973  8130 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:08.074304  8211 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:13:07.978497  9535 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:08.080976  8058 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:08.079150  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:08.040453  9616 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.084213  8133 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.083076  8063 raft_consensus.cc:606] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:08.085637  9485 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:08.120147  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:08.124632  8058 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:08.085139  8062 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.084612  9585 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:13:08.127378  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.109642  8208 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 3. Current term is 4. Ops: []
W20260709 13:13:08.103962  9633 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:13:08.098781  8136 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.132375  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 750) took 1211 ms. Trace:
I20260709 13:13:08.132711  8136 rpcz_store.cc:276] 0709 13:13:06.920670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.075232 (+1154562us) service_pool.cc:224] Handling call
0709 13:13:08.098424 (+ 23192us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b58001c3e98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:13:08.098605 (+   181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.132163 (+ 33558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.132327 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6923008}
I20260709 13:13:08.107635  9534 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:08.134023  8136 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:08.103379  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1026) took 1119 ms. Trace:
I20260709 13:13:08.105445  8026 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:08.135251  9534 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:08.136466  9571 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:08.136935  9571 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:08.138800  8058 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:08.105816  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 43778.3us
I20260709 13:13:08.099349  9554 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:08.139698  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:08.139915  8060 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.140777  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:08.141768  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:08.142411  9416 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.117583  9563 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:08.077451  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:08.113466  8061 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.143126  9416 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:13:08.143762  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.143966  8134 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.144387  9563 raft_consensus.cc:697] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:08.110208  8211 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.145264  9563 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.127028  9587 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:13:08.147197  8211 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.104442  8054 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:08.109298  8130 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:08.095234  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 737) took 1219 ms. Trace:
I20260709 13:13:08.094379  9620 raft_consensus.cc:740] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:08.052377  9557 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:08.089891  9372 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:08.052124  8209 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:08.087530  8102 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:08.097880  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:08.170961  8175 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:08.172227  8137 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:08.173319  8137 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.129266  8133 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:08.174484  8133 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:08.088977  9567 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.138563  9569 consensus_queue.cc:1059] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:08.142479  9561 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:08.113847  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.094302  8056 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:08.108248  8207 rpcz_store.cc:276] 0709 13:13:06.986101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.122736 (+136635us) service_pool.cc:224] Handling call
0709 13:13:07.774096 (+651360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:07.819841 (+ 45745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.069918 (+250077us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b540019dcd0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:08.070034 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1497312640}
I20260709 13:13:08.106385  8101 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:08.125120  9621 consensus_queue.cc:1059] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:13:08.135062  8210 rpcz_store.cc:276] 0709 13:13:06.983562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.117314 (+133752us) service_pool.cc:224] Handling call
0709 13:13:08.103233 (+985919us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b540023e6d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:08.103328 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2011071744}
I20260709 13:13:08.095564  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:08.152941  8130 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:08.156589  8061 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
W20260709 13:13:08.158885  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.199815  8029 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:08.179680  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 142115us
I20260709 13:13:08.200845  8130 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:08.201004  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.201853  8102 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:08.202037  8130 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:08.203007  8136 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.113570  9497 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:13:08.181591  8137 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.137741  9571 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:08.194164  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 87856.5us
I20260709 13:13:08.102949  9597 raft_consensus.cc:3055] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:08.307772  9597 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:13:08.307971  9639 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:13:08.308705  9597 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:08.310051  9597 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:08.150629  8129 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.205940  8137 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:08.287846  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 178684us
I20260709 13:13:08.207531  8207 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:08.206624  8130 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:08.103570  9603 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:08.244968  9621 consensus_queue.cc:1243] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:13:08.179191  9518 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.177601  9616 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:13:08.180408  9562 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:08.203567  9525 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:08.207602  9559 consensus_queue.cc:1059] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:13:08.332356  9303 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.349814  9303 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:08.350400  8055 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:08.349794  9559 raft_consensus.cc:3055] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:08.351018  9559 raft_consensus.cc:740] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:08.351876  9559 consensus_queue.cc:260] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:08.352715  9643 log.cc:927] Time spent T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:13:08.353029  9559 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.127451  9586 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:13:08.203240  9416 consensus_queue.cc:1059] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
W20260709 13:13:08.292337  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.238524  9580 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:08.204160  9600 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.166919  8138 rpcz_store.cc:276] 0709 13:13:06.875647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.314811 (+439164us) service_pool.cc:224] Handling call
0709 13:13:08.095040 (+780229us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54003ca0d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:08.095174 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278427520}
I20260709 13:13:08.177277  9575 consensus_queue.cc:1059] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:13:08.244448  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:08.359227  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 114857us
I20260709 13:13:08.202643  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:08.202898  8102 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:08.361352  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 158512us
I20260709 13:13:08.362201  8102 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:08.329097  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.364897  8102 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:08.374693  8132 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.335796  8207 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:08.336768  9567 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:08.376576  8102 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:08.338562  8693 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:08.262868  9557 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.380235  8137 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:13:08.345029  9562 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:08.382583  8136 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:13:08.263799  9582 consensus_queue.cc:237] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:08.305704  9620 consensus_queue.cc:260] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:08.398756  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:08.363831  9556 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.367913  9501 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:08.366884  9525 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.364729  9580 raft_consensus.cc:697] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:08.381080  9616 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.385104  9534 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.390029  9562 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:08.397074  9543 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.399045  9600 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:13:08.437995  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 78709.5us
I20260709 13:13:08.384492  9532 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:13:08.452569  9511 raft_consensus.cc:3055] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:08.452956  9511 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:08.453984  9532 raft_consensus.cc:3055] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:08.454377  9532 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:08.455145  9532 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:08.436305  9645 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:08.456797  9645 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:08.458372  9645 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:08.462723  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 102843us
W20260709 13:13:08.461524  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 74511.5us
I20260709 13:13:08.464226  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:08.475035  8175 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:08.399794  9372 consensus_queue.cc:1059] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:13:08.462626  9522 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.435431  9567 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:13:08.448894  8138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d80e5571c873417aa73c79508290e836"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:13:08.489050  9653 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:08.539572  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:08.540385  8211 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 2: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:08.477938  9645 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.490382  9603 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:08.492991  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.493606  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:08.494692  9627 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:08.496428  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1060) took 1175 ms. Trace:
I20260709 13:13:08.498692  9557 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:08.500736  9580 consensus_queue.cc:237] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:08.504356  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:08.558995  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.480140  9416 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:08.544523  8203 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:08.474244  9543 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.547472  8132 rpcz_store.cc:276] 0709 13:13:07.321085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.087918 (+766833us) service_pool.cc:224] Handling call
0709 13:13:08.088158 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.375072 (+286914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.496215 (+121143us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5400381b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:08.496373 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":658611584}
W20260709 13:13:08.546658  9631 log.cc:927] Time spent T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:13:08.572192  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1066) took 1156 ms. Trace:
I20260709 13:13:08.546227  8204 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:13:08.579136  9637 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.476s	user 0.003s	sys 0.000s
W20260709 13:13:08.583318  9640 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.476s	user 0.001s	sys 0.000s
W20260709 13:13:08.618743  8102 leader_election.cc:336] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Timed out: RequestConsensusVote RPC to 127.6.198.68:44573 timed out after 2.188s (SENT)
I20260709 13:13:08.619207  8102 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:08.587575  9583 raft_consensus.cc:3055] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:08.593123  9497 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:13:08.596529  9646 log.cc:927] Time spent T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:13:08.600844  9593 raft_consensus.cc:3055] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:08.609664  8723 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:08.547930  9564 consensus_queue.cc:1059] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:13:08.615581  8129 rpcz_store.cc:276] 0709 13:13:07.415427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.101277 (+685850us) service_pool.cc:224] Handling call
0709 13:13:08.116779 (+ 15502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.339429 (+222650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.571940 (+232511us) spinlock_profiling.cc:243] Waited 8.42 ms on lock 0x7b5400408dd0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:08.572118 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19358464}
I20260709 13:13:08.614605  9416 raft_consensus.cc:697] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:08.610368  9620 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.581672  9511 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.611727  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:08.612023  9587 raft_consensus.cc:3055] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:08.612927  8212 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.546396  8138 raft_consensus.cc:606] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:08.556746  9627 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:08.620334  9621 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:08.625880  9497 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:08.652326  9497 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:08.653105  9648 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:13:08.653373  9303 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:08.653537  9497 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:08.621995  9487 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.000s	sys 0.002s
I20260709 13:13:08.626112  9583 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:08.560890  9479 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:13:08.649886  9556 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:08.648713  9587 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:08.648622  8138 raft_consensus.cc:614] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:08.629621  9593 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:08.648350  9586 raft_consensus.cc:3055] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:13:08.655964  9487 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.001s	sys 0.002s
I20260709 13:13:08.671057  9511 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.669487  9586 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:08.659055  9627 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:08.722751  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 335311us
I20260709 13:13:08.704497  9621 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:08.709483  8131 consensus_meta.cc:287] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.604s	user 0.009s	sys 0.000s
I20260709 13:13:08.728869  7961 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34146 (request call id 255) took 5287 ms. Trace:
W20260709 13:13:08.733595  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 283732us
I20260709 13:13:08.715073  9522 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:08.709177  9587 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.725252  9563 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.739758  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:08.742403  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 276636us
I20260709 13:13:08.729254  7961 rpcz_store.cc:276] 0709 13:13:03.441505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:03.441884 (+   379us) service_pool.cc:224] Handling call
0709 13:13:08.728808 (+5286924us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:07.213835 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:07.333054 (+119219us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:07.333103 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:13:07.333128 (+    25us) tablet.cc:662] Decoding operations
0709 13:13:07.333551 (+   423us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:07.333697 (+   146us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:07.333739 (+    42us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:13:07.333960 (+   221us) tablet.cc:701] Row locks acquired
0709 13:13:07.333980 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:13:07.334153 (+   173us) write_op.cc:270] Start()
0709 13:13:07.334288 (+   135us) write_op.cc:276] Timestamp: P: 1783602787334099 usec, L: 0
0709 13:13:07.334318 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:13:07.398989 (+ 64671us) log.cc:844] Serialized 538 byte log entry
0709 13:13:07.568316 (+169327us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6db7f0c0ac 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b
0709 13:13:07.883243 (+314927us) op_driver.cc:464] REPLICATION: finished
0709 13:13:07.960139 (+ 76896us) write_op.cc:301] APPLY: starting
0709 13:13:07.960217 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:07.960328 (+   111us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:07.960347 (+    19us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:07.960602 (+   255us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:07.960757 (+   155us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:13:07.960801 (+    44us) write_op.cc:312] APPLY: finished
0709 13:13:07.963658 (+  2857us) log.cc:844] Serialized 23 byte log entry
0709 13:13:07.963824 (+   166us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:07.963901 (+    77us) write_op.cc:454] Released schema lock
0709 13:13:07.963937 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:07.964256 (+   319us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:07.964282 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9084,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":76682,"apply.run_cpu_time_us":1635,"apply.run_wall_time_us":4083,"num_ops":1,"prepare.queue_time_us":131479,"prepare.run_cpu_time_us":49841,"prepare.run_wall_time_us":510873,"raft.queue_time_us":54214,"raft.run_cpu_time_us":9507,"raft.run_wall_time_us":38763,"replication_time_us":548722,"spinlock_wait_cycles":2998016,"thread_start_us":374981,"threads_started":4,"wal-append.queue_time_us":364212}]]}
W20260709 13:13:08.724520  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 325816us
I20260709 13:13:08.733510  9629 raft_consensus.cc:3055] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:08.738632  8723 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:08.708941  9583 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.723016  9593 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.753350  8209 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:13:08.748097  9485 consensus_meta.cc:287] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.615s	user 0.006s	sys 0.003s
I20260709 13:13:08.747344  7959 catalog_manager.cc:5697] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:08.756829  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 281856us
I20260709 13:13:08.759898  8175 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:08.747607  9543 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:08.762835  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 201098us
W20260709 13:13:08.767082  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:08.772948  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.751920  9522 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.763314  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:08.710776  9416 consensus_queue.cc:237] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.732870  9621 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:08.749522  9629 raft_consensus.cc:740] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:08.773699  9663 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.774238  9652 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:08.732647  8131 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:08.764258  9587 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.775753  8209 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:08.780328  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:08.753593  9536 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:13:08.746404  9563 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.788465  8057 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:08.774926  9583 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.741216  9303 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:13:08.800150  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:08.800791  8209 raft_consensus.cc:3055] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:08.801139  8209 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:13:08.787633  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.803265  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:08.803928  8208 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
W20260709 13:13:08.749792  8724 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.010s	sys 0.022s
I20260709 13:13:08.790261  9663 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.792115  8131 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:08.793900  9518 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.783576  9485 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:13:08.794523  8135 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.794904  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:08.795194  9502 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:08.795864  9621 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:08.846690  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:08.849152  8137 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:08.849463  8059 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.796419  8138 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:08.854681  8029 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:08.844862  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:13:08.804797  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.805830  8102 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:08.858094  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 52354us
I20260709 13:13:08.806519  8057 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:08.806999  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:08.744261  9586 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.778399  9593 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:08.785198  9582 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:13:08.810662  8212 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:08.801929  8209 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:08.915272  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 66646.4us
I20260709 13:13:08.916287  8209 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:08.841197  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.762017  9656 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:13:08.845850  8132 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:08.809811  8724 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.010s	sys 0.022s
I20260709 13:13:08.852885  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:36048 (request call id 757) took 1563 ms. Trace:
I20260709 13:13:08.853462  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:08.859479  8727 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:08.904706  9554 consensus_meta.cc:287] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.728s	user 0.006s	sys 0.007s
I20260709 13:13:08.778944  9522 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.866425  8059 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:08.810199  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:08.893779  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.907256  8136 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:08.810063  9560 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 13:13:08.910765  9660 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 13:13:08.911170  8134 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:08.911659  9582 consensus_queue.cc:1059] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:08.848856  9515 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:08.913033  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:08.949883  8060 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:08.866278  9543 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:08.788226  9630 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.842352  9518 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:08.935397  9661 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:13:08.953466  9624 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:08.955247  9649 raft_consensus.cc:3055] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:08.955619  9649 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:08.905395  9629 consensus_queue.cc:260] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.910789  8723 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:08.936335  9627 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:08.906284  9575 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:08.930964  8131 rpcz_store.cc:276] 0709 13:13:07.278906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.918063 (+639157us) service_pool.cc:224] Handling call
0709 13:13:08.842105 (+924042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:08.926172  8209 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:08.945037  8727 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:08.930694  9501 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.932785  9658 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.159s
W20260709 13:13:08.951045  9662 log.cc:927] Time spent T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 13:13:08.950896  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:08.913982  9603 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:13:08.904484  9562 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:08.956372  8063 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.932364  9657 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:13:08.947389  8054 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.962487  9563 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:08.940335  9586 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:08.959928  8206 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:08.968858  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 758) took 1687 ms. Trace:
I20260709 13:13:09.036904  8135 rpcz_store.cc:276] 0709 13:13:07.281420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.016200 (+734780us) service_pool.cc:224] Handling call
0709 13:13:08.016485 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:08.843354 (+826869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.966251 (+122897us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b540036f454. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:08.968789 (+  2538us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1788983168}
I20260709 13:13:08.939899  8061 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:08.943725  9621 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:13:08.974936  8062 raft_consensus.cc:3055] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:13:08.995487  8130 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.011s	sys 0.019s
W20260709 13:13:09.041456  8130 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.011s	sys 0.019s
I20260709 13:13:08.966835  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 445) took 1117 ms. Trace:
I20260709 13:13:08.976547  9501 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:09.037997  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:08.972967  9652 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:09.041062  8062 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:08.972040  8204 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:09.008338  9629 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.056176  9655 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.056685  9655 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.059041  9518 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:09.060314  9652 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:08.973968  9620 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
W20260709 13:13:09.073927  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:09.081163  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.086647  9677 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:08.956532  9649 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:09.088102  9557 consensus_meta.cc:287] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.000s
I20260709 13:13:09.099049  8062 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.087287  9677 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.108570  8062 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.110905  9673 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:13:09.111697  9673 consensus_queue.cc:1243] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:09.113647  9677 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:09.115068  9562 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:09.119465  9614 consensus_meta.cc:287] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 1.036s	user 0.007s	sys 0.004s
I20260709 13:13:09.120023  8728 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:09.114203  9663 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:09.122079  9645 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.122884  9655 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:09.142491  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 352504us
I20260709 13:13:09.143149  9501 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:09.143244  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:09.144188  9562 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:09.144615  9562 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:09.145037  9562 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:09.151933  9561 consensus_meta.cc:287] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.940s	user 0.006s	sys 0.004s
I20260709 13:13:09.178404  9561 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:09.142133  9575 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:09.186553  9567 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.015s	sys 0.037s
W20260709 13:13:09.186952  9567 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.015s	sys 0.037s
W20260709 13:13:09.176204  9597 consensus_meta.cc:287] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.824s	user 0.008s	sys 0.003s
I20260709 13:13:09.211279  8060 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:09.227519  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 78068.8us
I20260709 13:13:09.162804  9645 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:09.245982  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:09.246327  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 390701us
I20260709 13:13:09.228219  9561 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.251801  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:09.252617  8209 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:09.254128  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:09.255610  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.255905  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:09.256539  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:09.257301  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:09.257989  8209 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:09.256644  8212 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:09.234308  9677 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 13:13:09.246907  9517 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:09.286963  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.053637  8724 rpcz_store.cc:276] 0709 13:13:07.849726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.855019 (+  5293us) service_pool.cc:224] Handling call
0709 13:13:07.855290 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:07.855505 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:07.986108 (+130603us) log.cc:844] Serialized 14 byte log entry
0709 13:13:08.608661 (+622553us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6db7f0c0ac 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b
0709 13:13:08.944061 (+335400us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:08.945767 (+  1706us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:08.949851 (+  4084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:08.953149 (+  3298us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:08.959697 (+  6548us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10440,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":5913,"spinlock_wait_cycles":2710272,"thread_start_us":942447,"threads_started":1,"wal-append.queue_time_us":825288}
I20260709 13:13:09.290273  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:09.291054  8057 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:09.291499  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:09.292260  8059 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
W20260709 13:13:09.298233  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 438716us
I20260709 13:13:09.288024  9517 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.302805  8062 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:09.302632  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:09.303426  8056 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:09.304556  9517 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:09.284828  9665 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 13:13:09.305063  9517 consensus_queue.cc:1059] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:09.259050  9561 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:09.309209  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 460012us
I20260709 13:13:09.293826  9416 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.311554  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:09.312462  8210 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:13:09.312913  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:09.320735  9535 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:09.322540  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:09.323179  8063 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:09.323834  9518 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:09.324203  9518 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.324563  9518 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:09.324646  9587 consensus_meta.cc:287] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.524s	user 0.006s	sys 0.003s
I20260709 13:13:09.326783  8058 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:09.285702  9682 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:09.329674  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:09.329713  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:09.330472  8132 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:09.301714  9655 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:09.335271  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.334373  9580 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:13:09.336838  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:09.337335  9580 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:09.338071  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:09.305953  9372 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.339200  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:09.339151  9372 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.271932  9585 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:13:09.341291  9585 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:09.341632  9372 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:09.332969  9563 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:09.334192  9559 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:13:09.353721  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:09.342226  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:09.363116  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:09.342737  9660 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:09.346581  9567 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.380667  9645 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:09.381260  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69580.9us
I20260709 13:13:09.328809  9595 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:13:09.352221  9556 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:09.350942  9563 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.333365  9416 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.353626  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:09.350927  9685 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:09.366453  9622 raft_consensus.cc:3055] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:09.378015  9660 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.348541  9655 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.395212  9556 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.358285  9502 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:13:09.399745  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:09.422508  8135 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:09.378240  9614 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:09.392128  9563 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.393092  9681 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.394480  9622 raft_consensus.cc:740] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:09.400851  9660 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:09.422912  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:09.422183  9685 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.353474  9664 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.081s
I20260709 13:13:09.447026  8211 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:13:09.387671  9567 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.433380  9517 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:09.452330  8135 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:09.335059  9682 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.443784  9689 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.458128  9658 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
W20260709 13:13:09.464608  9671 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.687s	user 0.015s	sys 0.006s
I20260709 13:13:09.465034  9658 consensus_queue.cc:1243] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:13:09.465085  9653 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:09.449512  9685 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.430650  9614 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.436784  9681 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.472410  9653 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:13:09.489540  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.489403  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:09.458106  9517 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.491331  8135 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:09.489729  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:09.492868  9684 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:09.495849  9668 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:13:09.500633  9532 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:09.511164  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:09.527072  8132 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:09.494328  9561 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:09.472260  9689 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.533206  8209 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:09.472210  9622 consensus_queue.cc:260] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.534698  9501 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.534904  9622 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:09.537314  9690 log.cc:927] Time spent T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:13:09.535218  9501 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.539577  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:09.494406  9658 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:13:09.504504  9629 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:09.512784  9684 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.550165  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:09.543926  9479 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:13:09.547569  9501 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:09.530835  9497 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.547770  9682 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:09.549484  9559 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.549328  9518 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.553220  8723 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:09.554574  9564 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.500458  9697 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.567102  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:09.561992  9497 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.552867  9684 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:09.534734  9678 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:09.580096  9655 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:09.593626  9682 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.599581  9682 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.600163  8138 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:09.601503  9682 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:09.601894  9518 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:09.589188  8730 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:13:09.555962  9535 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:09.566169  9593 consensus_meta.cc:287] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.661s	user 0.006s	sys 0.005s
I20260709 13:13:09.603237  8176 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:09.596585  9564 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.605587  8102 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:09.605547  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 139) took 1226 ms. Trace:
I20260709 13:13:09.606248  8130 rpcz_store.cc:276] 0709 13:13:08.379345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.379480 (+   135us) service_pool.cc:224] Handling call
0709 13:13:08.379857 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:08.403741 (+ 23884us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:08.403785 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:08.432123 (+ 28338us) log.cc:844] Serialized 26 byte log entry
0709 13:13:09.041890 (+609767us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:09.260334 (+218444us) log.cc:844] Serialized 14 byte log entry
0709 13:13:09.459433 (+199099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:09.459516 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:09.605317 (+145801us) raft_consensus.cc:1685] finished
0709 13:13:09.605335 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:09.605477 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18149,"prepare.queue_time_us":277576,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":92566,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":32206,"thread_start_us":958796,"threads_started":3,"wal-append.queue_time_us":785092}
I20260709 13:13:09.612780  8102 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:09.602753  9582 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:09.604496  9501 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.606638  9518 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.608850  9525 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.589097  9559 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.606640  9655 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:09.609143  9682 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:09.616822  9629 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:09.630174  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.610697  9485 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:09.602563  9535 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.783689  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:09.787804  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:09.745114  9416 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:09.722420  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1083) took 1964 ms. Trace:
I20260709 13:13:09.900368  8133 rpcz_store.cc:276] 0709 13:13:07.758484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.175085 (+416601us) service_pool.cc:224] Handling call
0709 13:13:08.175296 (+   211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:09.513188 (+1337892us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:09.513233 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:09.513286 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:09.513342 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:09.722137 (+208795us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54003c87d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:09.722193 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:09.722354 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3727,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":13368,"spinlock_wait_cycles":3076081920}
I20260709 13:13:09.735260  9653 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.732062  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:09.757721  9593 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:09.739984  9696 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.313s	user 0.002s	sys 0.000s
I20260709 13:13:09.762723  9556 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:09.779685  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:09.753394  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:09.779440  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:09.911209  8026 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:09.722680  8723 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:09.686615  9649 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:09.712237  8728 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 443) took 1923 ms (client timeout 2188 ms). Trace:
I20260709 13:13:09.916479  8136 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:09.709640  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:09.710306  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:09.918047  8132 raft_consensus.cc:2393] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:09.729375  9691 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:09.713248  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:09.620900  9501 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:09.733865  8206 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1047) took 1785 ms (client timeout 1879 ms). Trace:
I20260709 13:13:09.678751  9525 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.679742  9518 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.788847  9684 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.936983  8729 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:09.912683  9416 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.757208  9575 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.944079  9575 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.946588  9575 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:09.762118  9663 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:09.729808  9560 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.926585  9569 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.748466  8727 raft_consensus.cc:3055] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:09.805538  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:09.952013  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 146578us
I20260709 13:13:09.952898  8175 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:09.935707  9556 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.730597  8058 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:09.762164  9678 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:09.773401  9534 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.766458  8721 raft_consensus.cc:2955] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:09.753768  9680 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.805341  9303 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.687682  9656 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.702733  9600 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:09.729596  9603 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.737965  9652 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:09.700217  9624 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.782511  9515 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.699448  9554 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:09.896241  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:09.676951  9627 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:09.677129  9621 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:09.997309  9621 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:09.997831  9621 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:09.895160  9668 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.276s	user 0.000s	sys 0.000s
I20260709 13:13:09.697134  9517 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:09.909556  9543 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:10.003549  9543 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.894873  9557 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:10.006691  9557 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.912000  9629 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.009164  9557 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:10.009881  9557 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:09.914592  9559 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:09.915015  9653 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:10.011869  9629 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:09.830945  9682 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:10.014576  9682 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.714047  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:09.915436  8056 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:09.914916  9685 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.019066  8133 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:10.020354  9685 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:09.831396  9567 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:09.608877  9697 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.619856  9582 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.897094  9511 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.958957  9569 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:09.924665  8206 rpcz_store.cc:269] 0709 13:13:07.949578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.949730 (+   152us) service_pool.cc:224] Handling call
0709 13:13:08.068784 (+119054us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b5800168398. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:09.733665 (+1664881us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54001545d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:09.733800 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920679552}
W20260709 13:13:09.916819  8728 rpcz_store.cc:269] 0709 13:13:07.796318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:07.820642 (+ 24324us) service_pool.cc:224] Handling call
0709 13:13:09.712049 (+1891407us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400653954. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:09.712156 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2985997056}
I20260709 13:13:09.894695  9630 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.928843  9659 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.732887  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:09.956285  9560 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.933737  8132 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:09.929334  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:09.935032  9525 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:09.753800  9585 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.895949  9655 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.923574  9583 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:13:09.924647  9664 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:09.957633  8727 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:09.963013  9501 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:10.023969  9563 consensus_meta.cc:287] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.588s	user 0.008s	sys 0.001s
I20260709 13:13:09.923863  9593 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.965703  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:09.967495  9684 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:09.943475  9614 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:09.978736  9699 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.291s	user 0.000s	sys 0.000s
I20260709 13:13:09.979595  9564 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:09.943516  9485 raft_consensus.cc:697] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:09.987780  9603 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.990207  9652 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.959649  9561 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:09.949820  9681 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:09.968284  9522 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:10.021961  8138 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:09.897078  9479 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:09.966560  9678 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.993894  9624 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.950500  9416 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.022610  9511 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.931324  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:09.954625  9663 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.986572  9497 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:10.031949  9697 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:10.032397  8063 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:10.039510  9563 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.063105  9563 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:10.028342  9630 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.996958  9656 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.006215  9543 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:09.984452  9534 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.037324  8727 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:10.072376  8727 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:09.986460  9303 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.078195  9603 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:09.996176  9515 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:09.987845  9600 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:10.028134  9557 raft_consensus.cc:697] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:10.025084  9627 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.965395  9556 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:10.045141  9586 consensus_meta.cc:287] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 1.005s	user 0.009s	sys 0.003s
I20260709 13:13:10.069197  9630 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:10.024956  9554 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:09.995267  9680 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.080276  9518 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:10.049552  9561 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:10.082813  8132 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:13:10.084640  9622 consensus_meta.cc:287] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.541s	user 0.005s	sys 0.005s
I20260709 13:13:10.087211  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:10.089392  8130 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
W20260709 13:13:10.078796  9562 consensus_meta.cc:287] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.927s	user 0.010s	sys 0.001s
I20260709 13:13:10.065529  9479 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:10.096269  8727 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:13:10.037418  9585 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.104578  9554 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:10.105393  8058 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:10.076599  9511 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:10.034525  9659 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.105418  9587 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.094978  9534 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:10.110245  9518 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.107654  9561 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:10.083739  9621 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:10.110036  8727 pending_rounds.cc:85] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Aborting all ops after (but not including) 4
I20260709 13:13:10.091432  9485 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:10.127731  8730 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:10.114387  9663 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:10.115419  9416 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:10.123383  9652 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:10.151839  9609 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:10.111752  9562 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:10.119391  9557 consensus_queue.cc:237] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.130071  8727 pending_rounds.cc:107] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:13:10.128602  9587 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.121168  9569 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:10.151958  9303 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:10.126601  9518 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:10.153182  9585 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:10.281149  8104 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.984s	user 0.028s	sys 0.043s
I20260709 13:13:10.224726  9689 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:10.113353  9680 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:10.296612  9554 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.116217  9655 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:10.142845  9582 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:10.571499  8029 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.67:35745 timed out after 1.879s (SENT) [suppressed 47 similar messages]
W20260709 13:13:10.233012  9677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.010s	sys 0.032s
I20260709 13:13:10.121059  9624 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:10.643832  9677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.027s	user 0.018s	sys 0.053s
I20260709 13:13:10.128114  8723 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 3. (term mismatch)
I20260709 13:13:10.223714  9479 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:10.158579  9560 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:10.223214  9682 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:10.645928  9554 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:10.257481  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 303322us
I20260709 13:13:10.654070  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:10.254209  9536 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.003s	sys 0.012s
W20260709 13:13:10.589383  8693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.011s	sys 0.018s
W20260709 13:13:10.661041  8693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.978s	user 0.012s	sys 0.020s
I20260709 13:13:10.663651  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:10.664362  8060 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:10.665496  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:10.503877  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:10.191787  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:13:10.641786  8104 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.345s	user 0.035s	sys 0.063s
I20260709 13:13:10.530133  9679 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.264650  9511 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.283291  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:10.430373  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:10.640547  8727 raft_consensus.cc:2889] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:10.142613  9627 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:10.667488  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:10.670709  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 1.374e+06us
I20260709 13:13:10.696261  8060 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:10.151865  8134 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:10.697175  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:10.356757  9563 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:10.699169  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:10.641075  9303 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.641096  9609 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:10.654786  9682 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:10.644332  8723 pending_rounds.cc:85] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Aborting all ops after (but not including) 2
I20260709 13:13:10.647722  9665 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:10.669274  9536 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.943s	user 0.005s	sys 0.034s
W20260709 13:13:10.703688  8724 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.009s	sys 0.034s
I20260709 13:13:10.707505  8104 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:10.711196  8062 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:10.484735  9534 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:10.656692  8721 consensus_meta.cc:287] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.661s	user 0.009s	sys 0.002s
I20260709 13:13:10.152726  9593 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:10.699733  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:10.152139  9656 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:10.703526  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:10.672546  8204 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.169s	user 0.075s	sys 0.116s
I20260709 13:13:10.896170  8723 pending_rounds.cc:107] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 13:13:10.296685  9515 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:10.698518  8210 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:10.691731  9679 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.173136  9600 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:10.722922  8693 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:10.999048  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:11.063184  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 340339us
I20260709 13:13:11.063416  8206 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:11.066259  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:11.066994  8210 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:11.067940  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:10.702920  8696 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:10.706660  8725 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.002s	sys 0.012s
W20260709 13:13:10.703161  8131 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.036s	sys 0.012s
I20260709 13:13:10.672633  9556 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:10.490334  9597 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.699685  9630 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:10.921084  8728 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
W20260709 13:13:10.919687  8724 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.010s	sys 0.034s
I20260709 13:13:11.112468  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:11.007302  9653 consensus_meta.cc:287] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.996s	user 0.010s	sys 0.000s
W20260709 13:13:10.712872  9500 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:13:10.639182  9621 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:11.094518  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 391650us
I20260709 13:13:10.705600  9624 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.721635  9562 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:10.954449  8726 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
W20260709 13:13:11.108464  8135 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.159s	user 0.022s	sys 0.091s
I20260709 13:13:10.904884  9655 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:10.896766  9616 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.110685  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:11.111907  9630 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:10.897974  9560 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:11.006049  9561 consensus_meta.cc:287] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.872s	user 0.009s	sys 0.000s
W20260709 13:13:11.004909  9660 consensus_meta.cc:287] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 1.549s	user 0.006s	sys 0.006s
W20260709 13:13:11.090553  8725 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.108s	user 0.003s	sys 0.012s
I20260709 13:13:10.909170  9627 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:10.917744  9659 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:10.703608  9511 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.719655  9563 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:10.956004  9677 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:10.956329  9416 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:11.013358  9703 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:11.022714  9593 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:11.113032  9515 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:11.001667  9652 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.143s
I20260709 13:13:10.642416  9518 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.113732  9656 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:10.918377  9665 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:11.139746  9645 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.015s	sys 0.041s
I20260709 13:13:11.132943  9556 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:11.106884  9534 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.150238  9655 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.165570  9630 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:11.181947  9707 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:13:11.153357  9653 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:13:11.144920  9624 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.193581  9656 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.168048  9560 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.167482  9660 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.167893  9561 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.195773  9665 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:11.192482  9645 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.017s	sys 0.043s
I20260709 13:13:11.196484  9563 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:11.196270  9556 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:11.196537  9678 consensus_meta.cc:287] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 1.129s	user 0.011s	sys 0.000s
I20260709 13:13:11.197535  9655 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:11.200244  9554 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.164076  9616 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.011732  9609 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.114367  9597 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.194566  9679 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.186177  9593 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.175231  9627 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:11.101593  8131 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.334s	user 0.036s	sys 0.012s
W20260709 13:13:11.144919  8135 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.196s	user 0.022s	sys 0.091s
W20260709 13:13:11.139873  8137 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.012s	sys 0.046s
I20260709 13:13:10.918558  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:11.069100  8204 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.569s	user 0.075s	sys 0.122s
I20260709 13:13:11.118394  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:11.153695  8208 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:11.096580  8723 raft_consensus.cc:2917] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 13:13:11.462306  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 305) took 1905 ms. Trace:
I20260709 13:13:11.147971  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:10.739410  9303 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.158732  9479 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:11.186403  9515 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.417045  9731 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.456059  8203 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:11.424721  8176 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:11.456941  9518 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.367825  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1104) took 1614 ms. Trace:
I20260709 13:13:11.351850  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 1940) took 1492 ms. Trace:
I20260709 13:13:11.368806  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:11.486493  8102 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.012s	sys 0.006s
W20260709 13:13:11.488376  8102 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.013s	sys 0.006s
I20260709 13:13:11.459174  9554 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.361532  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 1937) took 1569 ms. Trace:
I20260709 13:13:11.459357  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:11.365989  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 1936) took 1574 ms. Trace:
I20260709 13:13:11.460554  9557 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.488643  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:11.457427  8137 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.349s	user 0.015s	sys 0.051s
I20260709 13:13:11.467494  8729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 298) took 2770 ms. Trace:
I20260709 13:13:11.364869  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1099) took 1613 ms. Trace:
W20260709 13:13:11.360291  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:11.352061  8029 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.007s	sys 0.008s
W20260709 13:13:11.491039  8175 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.000s	sys 0.012s
W20260709 13:13:11.513844  8029 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.010s	sys 0.011s
W20260709 13:13:11.513936  8175 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.000s	sys 0.012s
I20260709 13:13:11.507496  9728 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:13:11.351831  8061 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.624s	user 0.017s	sys 0.042s
W20260709 13:13:11.514616  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 943208us
W20260709 13:13:11.514981  8061 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.787s	user 0.017s	sys 0.065s
I20260709 13:13:11.516284  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 97) took 1790 ms. Trace:
I20260709 13:13:11.517732  9738 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.516659  8061 rpcz_store.cc:276] 0709 13:13:09.725843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.726692 (+   849us) service_pool.cc:224] Handling call
0709 13:13:09.727018 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:09.727227 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:09.727514 (+   287us) log.cc:844] Serialized 14 byte log entry
0709 13:13:11.515763 (+1788249us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:11.515866 (+   103us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:11.515965 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.516038 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.516234 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":369,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":1437,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":307,"thread_start_us":1787684,"threads_started":1,"wal-append.queue_time_us":1477272}
I20260709 13:13:11.158347  8722 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:11.467024  9580 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.510283  9518 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.512208  8729 rpcz_store.cc:276] 0709 13:13:08.697018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.697184 (+   166us) service_pool.cc:224] Handling call
0709 13:13:09.712573 (+1015389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:09.938805 (+226232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:10.151072 (+212267us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b5400653954. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:11.466974 (+1315902us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400653950. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:11.467260 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2818753280}
I20260709 13:13:11.521117  9554 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.464422  8725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 297) took 1565 ms. Trace:
I20260709 13:13:11.522558  9372 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:11.523780  9554 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:11.523483  9372 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.526317  9660 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:11.530511  9515 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:11.530802  9728 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.519940  9580 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:11.369786  8130 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 777) took 1660 ms (client timeout 1755 ms). Trace:
I20260709 13:13:11.450945  9511 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:11.370914  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1106) took 1505 ms. Trace:
I20260709 13:13:11.482969  8056 rpcz_store.cc:276] 0709 13:13:09.859666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.018943 (+159277us) service_pool.cc:224] Handling call
0709 13:13:10.019287 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:10.019593 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:10.019627 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:10.020265 (+   638us) log.cc:844] Serialized 26 byte log entry
0709 13:13:10.259891 (+239626us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:10.259946 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:10.260051 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:11.350872 (+1090821us) raft_consensus.cc:1685] finished
0709 13:13:11.350974 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.351502 (+   528us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2063,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":211,"thread_start_us":239178,"threads_started":1,"wal-append.queue_time_us":1162040}
I20260709 13:13:11.538393  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 295) took 1940 ms. Trace:
I20260709 13:13:11.467630  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:11.539141  9557 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.540076  8206 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:11.363161  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:11.522251  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:11.479789  8132 rpcz_store.cc:276] 0709 13:13:09.753156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.029827 (+276671us) service_pool.cc:224] Handling call
0709 13:13:10.030137 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:10.106194 (+ 76057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.367182 (+1260988us) spinlock_profiling.cc:243] Waited 52.3 ms on lock 0x7b540035cfd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:11.367584 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120403200}
I20260709 13:13:11.497452  8058 rpcz_store.cc:276] 0709 13:13:09.793618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.965396 (+171778us) service_pool.cc:224] Handling call
0709 13:13:11.361067 (+1395671us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b5400334d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:11.361312 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149750656}
W20260709 13:13:11.352730  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 645091us
I20260709 13:13:11.498024  8059 rpcz_store.cc:276] 0709 13:13:09.791646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.791788 (+   142us) service_pool.cc:224] Handling call
0709 13:13:09.792100 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:10.022777 (+230677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:10.022809 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:10.022850 (+    41us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:10.022876 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.365482 (+1342606us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b540025f3d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:11.365548 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.365819 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530160896}
I20260709 13:13:11.513031  8133 rpcz_store.cc:276] 0709 13:13:09.751705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.907429 (+155724us) service_pool.cc:224] Handling call
0709 13:13:09.907754 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:10.019412 (+111658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.364092 (+1344680us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400174154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:11.364591 (+   499us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255480576}
I20260709 13:13:11.428346  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:11.507979  8721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 310) took 1742 ms. Trace:
I20260709 13:13:11.462911  8723 raft_consensus.cc:2889] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 2 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:11.532980  9518 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:11.534806  8130 rpcz_store.cc:269] 0709 13:13:09.709306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.709415 (+   109us) service_pool.cc:224] Handling call
0709 13:13:11.369321 (+1659906us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540008cd54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:11.369546 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":394682112}
I20260709 13:13:11.518568  9738 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.564479  9372 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.565984  9711 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:11.566479  8055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.776s	user 0.009s	sys 0.017s
I20260709 13:13:11.550798  8727 rpcz_store.cc:276] 0709 13:13:09.598364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.598510 (+   146us) service_pool.cc:224] Handling call
0709 13:13:09.598809 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:10.689912 (+1091103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.489651 (+799739us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b540041e6d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:11.538177 (+ 48526us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343857920}
I20260709 13:13:11.464511  9562 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:11.569841  8136 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:11.570384  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:11.465569  9416 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.574812  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:11.576459  9732 log.cc:927] Time spent T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.061s	user 0.002s	sys 0.000s
I20260709 13:13:11.576681  9682 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.586596  9557 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.577196  8205 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:11.577626  8211 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:11.578312  9562 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.578886  9303 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.579447  9627 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:11.579629  9678 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.580834  9535 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.583158  9723 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:11.583285  9679 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.583791  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:11.586135  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 161846us
I20260709 13:13:11.605685  8135 raft_consensus.cc:2955] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:11.607338  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:11.608570  8207 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:11.609077  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:11.187894  9677 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.612262  8210 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:11.617810  8129 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:11.610530  9727 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:11.621023  9597 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:11.622846  9616 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.586455  9652 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.624051  9652 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.625492  9678 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.587342  9682 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.570510  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:11.626878  8060 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:11.628096  9724 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.628393  9580 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.628716  9682 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.629583  8209 raft_consensus.cc:1240] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:11.629710  8208 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:11.630518  8205 raft_consensus.cc:2955] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:11.573645  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:11.567433  9561 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:11.572537  8055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.852s	user 0.009s	sys 0.018s
I20260709 13:13:11.591665  9660 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:11.592958  8726 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:11.595394  9593 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.595467  9562 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:11.595738  8723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 308) took 1873 ms. Trace:
I20260709 13:13:11.615449  9677 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.596290  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:11.597445  9556 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.598477  9656 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.598794  9609 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:11.600584  9416 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:11.600693  9624 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.641429  9535 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:11.588726  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:11.618635  8129 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1088) took 2774 ms (client timeout 1836 ms). Trace:
I20260709 13:13:11.228027  9706 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:11.542574  8134 rpcz_store.cc:276] 0709 13:13:09.865279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.124014 (+258735us) service_pool.cc:224] Handling call
0709 13:13:10.126386 (+  2372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:10.706406 (+580020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.370373 (+663967us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b5400042ed4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:11.370715 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36699520}
I20260709 13:13:11.635510  8207 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:11.615909  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:11.622098  8722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 303) took 2077 ms. Trace:
I20260709 13:13:11.632268  9560 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:11.640301  8054 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.738s	user 0.028s	sys 0.011s
W20260709 13:13:11.663357  8054 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.885s	user 0.029s	sys 0.012s
I20260709 13:13:11.663267  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:11.650641  8061 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:11.657172  8059 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
W20260709 13:13:11.654839  8129 rpcz_store.cc:269] 0709 13:13:08.844553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.848976 (+  4423us) service_pool.cc:224] Handling call
0709 13:13:09.336608 (+487632us) spinlock_profiling.cc:243] Waited 36.8 ms on lock 0x7b58001c3e98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:11.362648 (+2026040us) spinlock_profiling.cc:243] Waited 6.24 ms on lock 0x7b58001c3e98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1e9c 00007f6db7ef1a5b 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:11.618501 (+255853us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98890880}
I20260709 13:13:11.662038  9560 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:11.662575  9416 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:11.664557  9556 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:11.666754  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 100) took 1890 ms. Trace:
I20260709 13:13:11.667028  9689 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.667737  9659 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:11.667855  9587 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.606269  9723 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.592041  9731 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.628858  9724 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.619519  9727 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.686793  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:11.522818  8725 rpcz_store.cc:276] 0709 13:13:09.899142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.902120 (+  2978us) service_pool.cc:224] Handling call
0709 13:13:09.913424 (+ 11304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:09.927098 (+ 13674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:09.930768 (+  3670us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:09.982514 (+ 51746us) log.cc:844] Serialized 26 byte log entry
0709 13:13:11.463551 (+1481037us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:11.463676 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.463871 (+   195us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:11.463947 (+    76us) raft_consensus.cc:1685] finished
0709 13:13:11.463966 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.464239 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2182,"raft.queue_time_us":10838,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":2828,"thread_start_us":1184667,"threads_started":1,"wal-append.queue_time_us":1215178}
I20260709 13:13:11.678156  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:11.677970  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:11.713182  8727 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 2 LEARNER]: Advancing to term 3
I20260709 13:13:11.669150  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:11.642704  8723 rpcz_store.cc:276] 0709 13:13:09.721975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.722133 (+   158us) service_pool.cc:224] Handling call
0709 13:13:09.722543 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:11.595440 (+1872897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.595687 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:11.305290  9630 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:11.668391  9705 log.cc:927] Time spent T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.653s	user 0.000s	sys 0.003s
I20260709 13:13:11.658311  9593 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.674275  9652 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:11.687088  9556 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:11.187652  9703 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.657127  9535 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:11.721093  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 54339.2us
I20260709 13:13:11.722661  8136 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:11.723752  9723 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.716323  9624 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:11.726094  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:11.726191  9727 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.726958  9665 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.607776  9651 raft_consensus.cc:2955] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:11.730501  9560 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.732582  9616 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:11.733259  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 1928) took 2885 ms. Trace:
I20260709 13:13:11.734185  9738 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.462977  8724 rpcz_store.cc:276] 0709 13:13:09.556252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.566272 (+ 10020us) service_pool.cc:224] Handling call
0709 13:13:09.606545 (+ 40273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:09.620682 (+ 14137us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:09.675713 (+ 55031us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:09.941734 (+266021us) log.cc:844] Serialized 26 byte log entry
0709 13:13:11.138946 (+1197212us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:11.147504 (+  8558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.159427 (+ 11923us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:11.461708 (+302281us) raft_consensus.cc:1685] finished
0709 13:13:11.461750 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.462039 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58519,"raft.queue_time_us":73,"raft.run_cpu_time_us":23390,"raft.run_wall_time_us":58542,"thread_start_us":1154432,"threads_started":1,"wal-append.queue_time_us":1245865}
I20260709 13:13:11.735355  9706 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:11.736287  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:11.736186  9731 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.736642  9536 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:11.737563  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:11.560669  8721 rpcz_store.cc:276] 0709 13:13:09.765598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.765763 (+   165us) service_pool.cc:224] Handling call
0709 13:13:09.766086 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:09.766322 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:10.983718 (+1217396us) log.cc:844] Serialized 14 byte log entry
0709 13:13:11.507146 (+523428us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:11.507284 (+   138us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:11.507419 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.507542 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.507768 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":351869,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":323,"thread_start_us":493939,"threads_started":2,"wal-append.queue_time_us":168118}
I20260709 13:13:11.720353  8133 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:11.695838  9656 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.706864  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:11.752625  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1101) took 2000 ms. Trace:
I20260709 13:13:11.727118  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:11.659828  9678 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:13:11.754544  8132 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:11.796926  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 793) took 1081 ms. Trace:
I20260709 13:13:11.797272  8132 rpcz_store.cc:276] 0709 13:13:10.716549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.604644 (+888095us) service_pool.cc:224] Handling call
0709 13:13:11.796873 (+192229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:11.664778  8722 rpcz_store.cc:276] 0709 13:13:09.544296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.548112 (+  3816us) service_pool.cc:224] Handling call
0709 13:13:09.548405 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:11.519509 (+1971104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.621697 (+102188us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b5400505fd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:11.621948 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3666701696}
I20260709 13:13:11.760466  9739 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:11.760965  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1097) took 1968 ms. Trace:
I20260709 13:13:11.738613  9561 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:11.710356  9580 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:11.743481  9556 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:11.769554  8206 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:11.772611  9560 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.774547  9600 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:11.775343  9689 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.838761  9600 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.840094  9630 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:11.718303  9416 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.841305  9689 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.842092  9416 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:11.783110  9665 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.784096  9593 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.782454  8728 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:11.721628  8063 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:11.792789  8208 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:11.794050  8135 rpcz_store.cc:276] 0709 13:13:09.752335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.919698 (+167363us) service_pool.cc:224] Handling call
0709 13:13:09.920177 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:09.923723 (+  3546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:09.923765 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:09.925989 (+  2224us) log.cc:844] Serialized 556 byte log entry
0709 13:13:11.602090 (+1676101us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:11.751918 (+149828us) log.cc:844] Serialized 14 byte log entry
0709 13:13:11.752255 (+   337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.752368 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:11.752407 (+    39us) raft_consensus.cc:1685] finished
0709 13:13:11.752423 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.752563 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":923,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"thread_start_us":1563611,"threads_started":1,"wal-append.queue_time_us":1288857}
I20260709 13:13:11.794556  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 787) took 1081 ms. Trace:
I20260709 13:13:11.803689  9739 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.804311  9580 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.860546  9739 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.862059  9580 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.805665  8129 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:11.862406  9665 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:11.862828  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 791) took 1151 ms. Trace:
I20260709 13:13:11.863229  8129 rpcz_store.cc:276] 0709 13:13:10.711136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.711717 (+1000581us) service_pool.cc:224] Handling call
0709 13:13:11.862767 (+151050us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:11.864284  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:11.806421  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1075) took 1085 ms. Trace:
I20260709 13:13:11.865011  9593 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.808388  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:11.801827  9561 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:11.808897  9723 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:11.810706  9535 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.811022  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1080) took 1092 ms. Trace:
I20260709 13:13:11.799962  8131 rpcz_store.cc:276] 0709 13:13:09.751029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.765377 (+ 14348us) service_pool.cc:224] Handling call
0709 13:13:09.765786 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:09.766197 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:09.766240 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:09.766821 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 13:13:11.643741 (+1876920us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:11.656724 (+ 12983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.675107 (+ 18383us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:11.706225 (+ 31118us) raft_consensus.cc:1685] finished
0709 13:13:11.706246 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.719049 (+ 12803us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1568,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":148,"thread_start_us":1746186,"threads_started":1,"wal-append.queue_time_us":1431957}
I20260709 13:13:11.813937  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 301) took 1122 ms. Trace:
I20260709 13:13:11.813719  9556 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.814690  9728 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:11.816792  9716 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.797s	user 0.001s	sys 0.001s
I20260709 13:13:11.818549  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 138) took 1116 ms. Trace:
I20260709 13:13:11.819502  9534 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.820241  9645 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.820756  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 773) took 2490 ms. Trace:
I20260709 13:13:11.823827  9706 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:11.800045  9731 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:11.824270  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:11.888443  9731 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.888871  8059 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:11.784689  9616 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.890861  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1105) took 2118 ms. Trace:
I20260709 13:13:11.891595  9556 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.892146  9616 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:11.824827  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:11.891234  8138 rpcz_store.cc:276] 0709 13:13:09.772246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.046594 (+274348us) service_pool.cc:224] Handling call
0709 13:13:10.046977 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:10.047351 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:10.047391 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:10.047853 (+   462us) log.cc:844] Serialized 26 byte log entry
0709 13:13:10.259149 (+211296us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:10.259215 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:10.259317 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:11.815755 (+1556438us) spinlock_profiling.cc:243] Waited 73.7 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:11.890407 (+ 74652us) raft_consensus.cc:1685] finished
0709 13:13:11.890433 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.890695 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2845,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":14823,"spinlock_wait_cycles":169611136,"thread_start_us":210867,"threads_started":1,"wal-append.queue_time_us":955771}
I20260709 13:13:11.892843  9723 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.893214  8721 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:13:11.893538  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:11.825018  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:11.875886  9689 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.826126  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:11.826848  9703 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.828337  9724 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.759101  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:11.829044  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:11.829859  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 140) took 1125 ms. Trace:
I20260709 13:13:11.830062  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:11.830462  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:11.882148  9630 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:11.830677  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:11.820766  9500 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:11.687942  8054 rpcz_store.cc:276] 0709 13:13:09.776590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.776753 (+   163us) service_pool.cc:224] Handling call
0709 13:13:09.777151 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:09.777550 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:09.777589 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:09.778125 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:13:11.366580 (+1588455us) spinlock_profiling.cc:243] Waited 60.5 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6db7f0c0ac 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b
0709 13:13:11.664159 (+297579us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:11.664258 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.664364 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:11.664418 (+    54us) raft_consensus.cc:1685] finished
0709 13:13:11.664435 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.666674 (+  2239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2731,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"spinlock_wait_cycles":139237760,"thread_start_us":1885465,"threads_started":1,"wal-append.queue_time_us":1737170}
I20260709 13:13:11.767866  9659 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.802004  9624 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.832193  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:11.833597  8730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 311) took 1746 ms (client timeout 1693 ms). Trace:
I20260709 13:13:11.833503  9536 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.834371  9479 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:11.834503  9656 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.834829  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:11.835408  8203 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:11.835572  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:11.882100  9620 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.032s	user 0.035s	sys 0.024s
I20260709 13:13:11.762126  8055 rpcz_store.cc:276] 0709 13:13:08.847576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:08.849111 (+  1535us) service_pool.cc:224] Handling call
0709 13:13:09.703240 (+854129us) spinlock_profiling.cc:243] Waited 6.38 ms on lock 0x7b5800000c98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:13:09.703501 (+   261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:09.703934 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:09.703975 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:09.720344 (+ 16369us) log.cc:844] Serialized 26 byte log entry
0709 13:13:11.379454 (+1659110us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6db7f0c0ac 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b
0709 13:13:11.668647 (+289193us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:11.674060 (+  5413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.712055 (+ 37995us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:11.732985 (+ 20930us) raft_consensus.cc:1685] finished
0709 13:13:11.733008 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.733165 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7837,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":136,"spinlock_wait_cycles":414296704,"thread_start_us":1915719,"threads_started":1,"wal-append.queue_time_us":1771230}
I20260709 13:13:11.836215  9660 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:11.782927  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:11.844507  8133 rpcz_store.cc:276] 0709 13:13:10.709437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.557323 (+847886us) service_pool.cc:224] Handling call
0709 13:13:11.790510 (+233187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:11.796581  9727 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.847733  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:11.849368  8136 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:11.750056  9679 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.878517  9416 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:11.851671  8693 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:11.785482  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:11.852782  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:11.858592  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1073) took 1142 ms. Trace:
I20260709 13:13:11.867031  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 302) took 1160 ms. Trace:
W20260709 13:13:11.867460  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 106935us
I20260709 13:13:11.865150  8206 rpcz_store.cc:276] 0709 13:13:10.717055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.632394 (+915339us) service_pool.cc:224] Handling call
0709 13:13:11.802150 (+169756us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:11.864988  8129 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:11.842258  9600 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.896317  8061 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:11.817395  8696 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:11.871371  9587 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.911624  8728 rpcz_store.cc:276] 0709 13:13:10.705247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.147643 (+442396us) service_pool.cc:224] Handling call
0709 13:13:11.865627 (+717984us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:11.871979  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 102648us
I20260709 13:13:11.872853  9745 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.904201  8724 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:11.925952  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:11.908892  9747 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:13:11.926709  8129 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:11.927312  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:11.894311  8138 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:11.904662  9727 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.928102  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:11.896166  8134 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:11.928792  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 790) took 1218 ms. Trace:
I20260709 13:13:11.873414  9580 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.873888  9561 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:11.881906  8209 rpcz_store.cc:276] 0709 13:13:10.702274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.702401 (+   127us) service_pool.cc:224] Handling call
0709 13:13:10.702659 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:11.630093 (+927434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.818236 (+188143us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b540014a354. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:11.818482 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2130862336}
I20260709 13:13:11.886482  9534 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:11.886200  9645 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.895843  8130 raft_consensus.cc:2393] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:11.900671  8062 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:11.903342  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:11.905725  8136 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 786) took 1196 ms (client timeout 1567 ms). Trace:
I20260709 13:13:11.929286  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:11.909664  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 58142.1us
I20260709 13:13:11.895081  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:11.904855  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:11.906629  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1072) took 1190 ms. Trace:
I20260709 13:13:11.918962  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:11.911173  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 784) took 1202 ms. Trace:
I20260709 13:13:11.897655  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:11.900803  8730 rpcz_store.cc:269] 0709 13:13:10.086836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.086999 (+   163us) service_pool.cc:224] Handling call
0709 13:13:11.833239 (+1746240us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b5400198f54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:11.833372 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46104320}
I20260709 13:13:11.924553  8727 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 3 LEARNER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:11.895283  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:11.903940  9660 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:11.896441  8211 rpcz_store.cc:276] 0709 13:13:10.704057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:10.704499 (+   442us) service_pool.cc:224] Handling call
0709 13:13:10.704735 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:11.593234 (+888499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.829593 (+236359us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b540025cbd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:11.829804 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2006434688}
I20260709 13:13:11.902448  9656 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.901033  9479 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:11.926163  9745 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.872677  9560 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:11.898303  9500 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.927973  8057 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
I20260709 13:13:11.928259  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 795) took 1023 ms. Trace:
I20260709 13:13:11.838075  9515 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.873311  9739 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.876111  8210 rpcz_store.cc:276] 0709 13:13:10.736918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.067693 (+330775us) service_pool.cc:224] Handling call
0709 13:13:11.810845 (+743152us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b5400208ad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:11.810973 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224702336}
I20260709 13:13:11.911487  8207 rpcz_store.cc:276] 0709 13:13:10.716592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.712988 (+996396us) service_pool.cc:224] Handling call
0709 13:13:11.726687 (+ 13699us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:11.842248 (+115561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:11.842305 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:11.842396 (+    91us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:11.842434 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.858369 (+ 15935us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540019e1d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:11.858409 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.858519 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":265161984}
I20260709 13:13:11.895471  9689 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:11.913092  9704 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.050s	user 0.008s	sys 0.041s
I20260709 13:13:11.926702  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:11.902149  8132 raft_consensus.cc:2368] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3.
I20260709 13:13:11.971846  8729 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:11.974108  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 794) took 1249 ms. Trace:
I20260709 13:13:11.894937  9723 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.902858  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:11.894968  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:11.897398  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:11.883780  8137 rpcz_store.cc:276] 0709 13:13:09.329863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.334282 (+  4419us) service_pool.cc:224] Handling call
0709 13:13:09.727342 (+393060us) spinlock_profiling.cc:243] Waited 8.1 ms on lock 0x7b58001c3e98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:13:09.727505 (+   163us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:10.090548 (+363043us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:10.090590 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:10.091089 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:13:11.519644 (+1428555us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:11.519746 (+   102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.820400 (+300654us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b540036ea54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:11.820446 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:11.820492 (+    46us) raft_consensus.cc:1685] finished
0709 13:13:11.820504 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.820670 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":4649,"spinlock_wait_cycles":852555392,"thread_start_us":1403687,"threads_started":1,"wal-append.queue_time_us":1139296}
W20260709 13:13:11.903118  9620 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.219s	user 0.035s	sys 0.024s
I20260709 13:13:11.907028  9536 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.895066  9621 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:11.901750  9624 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:11.897715  9731 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.905400  9659 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:11.896701  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:11.910256  8725 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:11.912243  9655 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:11.897302  9630 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:11.908749  9416 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:12.005421  8136 rpcz_store.cc:269] 0709 13:13:10.709115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.604294 (+895179us) service_pool.cc:224] Handling call
0709 13:13:11.905632 (+301338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:12.005131  8135 rpcz_store.cc:276] 0709 13:13:10.708457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.852482 (+1144025us) service_pool.cc:224] Handling call
0709 13:13:11.911106 (+ 58624us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:12.098402  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:11.878307  8726 rpcz_store.cc:276] 0709 13:13:10.691210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.461266 (+770056us) service_pool.cc:224] Handling call
0709 13:13:11.813744 (+352478us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b54006527d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:11.813860 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38826752}
I20260709 13:13:12.146646  8058 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:12.005247  8133 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:12.110647  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:12.150815  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:11.912699  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 130) took 2480 ms. Trace:
W20260709 13:13:12.168088  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 254634us
I20260709 13:13:11.929131  8134 rpcz_store.cc:276] 0709 13:13:10.710334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.696349 (+986015us) service_pool.cc:224] Handling call
0709 13:13:11.928735 (+232386us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:12.057446  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:12.351958  8132 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 146) took 1411 ms (client timeout 1762 ms). Trace:
I20260709 13:13:12.143998  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:12.115360  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 66592.4us
W20260709 13:13:12.353312  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 467859us
I20260709 13:13:12.277546  8176 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:12.005740  8208 rpcz_store.cc:276] 0709 13:13:10.716337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.630548 (+914211us) service_pool.cc:224] Handling call
0709 13:13:11.630941 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:11.865717 (+234776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.906326 (+ 40609us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b5400110654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:11.906538 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":316353408}
I20260709 13:13:12.043748  8212 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
W20260709 13:13:12.247570  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 430328us
I20260709 13:13:12.350811  8130 rpcz_store.cc:276] 0709 13:13:10.725223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.825861 (+1100638us) service_pool.cc:224] Handling call
0709 13:13:11.974037 (+148176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:12.055848  8206 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:12.007465  9600 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.312680  8104 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:12.364038  8132 rpcz_store.cc:269] 0709 13:13:10.940319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.834516 (+894197us) service_pool.cc:224] Handling call
0709 13:13:12.351857 (+517341us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:12.367583  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 191550us
I20260709 13:13:12.147080  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:12.307042  9727 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:12.019424  9724 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.083477  8056 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:11.909183  9679 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:12.379082  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.380390  9500 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:12.400105  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.382552  9655 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:12.404033  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:12.404747  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.384188  9515 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:12.387096  9621 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:12.389493  9745 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:12.390666  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.413211  8207 raft_consensus.cc:2393] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:12.378523  8211 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
W20260709 13:13:12.354590  9704 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.652s	user 0.011s	sys 0.051s
I20260709 13:13:12.398449  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.390527  8722 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:12.380975  9689 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:12.218578  8723 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:12.098167  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1071) took 1382 ms. Trace:
I20260709 13:13:12.006227  9678 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:12.007925  9747 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.365s
I20260709 13:13:12.386090  9416 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:12.405575  8132 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
I20260709 13:13:12.463330  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.459105  9727 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:12.463423  9678 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:12.463946  8132 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:12.464911  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.465445  8132 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:12.465634  9678 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:12.465926  9621 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:12.389992  9739 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:12.466763  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58827.8us
I20260709 13:13:12.466295  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.251093  9734 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 3 LEARNER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:12.407619  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:12.144302  8129 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:12.363408  8724 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:13:12.407195  8730 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
W20260709 13:13:12.408919  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 143704us
I20260709 13:13:12.317718  8138 rpcz_store.cc:276] 0709 13:13:10.916552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.893216 (+976664us) service_pool.cc:224] Handling call
0709 13:13:11.928194 (+ 34978us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:12.377108  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.145422  9534 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:12.237260  9580 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:12.146958  9587 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:12.415122  9652 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:12.390785  9706 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:12.416065  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.412271  8137 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
W20260709 13:13:12.422464  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 152755us
I20260709 13:13:12.390419  8131 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:12.416008  9600 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:12.419171  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:12.436151  9724 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:12.491998  8211 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:12.492148  8726 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:12.492875  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.493340  9724 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:12.439733  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.494131  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.440683  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.494683  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:12.458043  9660 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:12.495507  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.496182  8212 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:12.496585  9534 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:12.496869  9660 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:12.498008  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.498577  8130 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:12.498899  9580 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:12.499528  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.494709  8137 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:12.500118  8130 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:12.500715  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.501348  8137 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:12.501816  8133 raft_consensus.cc:2410] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:13:12.503125  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 148) took 1508 ms. Trace:
I20260709 13:13:12.503439  8133 rpcz_store.cc:276] 0709 13:13:11.028089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.904533 (+876444us) service_pool.cc:224] Handling call
0709 13:13:12.503074 (+598541us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:12.503892  9587 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:12.504226  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:12.504616  8131 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 788) took 1794 ms (client timeout 2295 ms). Trace:
I20260709 13:13:12.505007  8721 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:12.506192  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 789) took 1796 ms. Trace:
I20260709 13:13:12.506565  8130 rpcz_store.cc:276] 0709 13:13:10.710035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.499258 (+1789223us) service_pool.cc:224] Handling call
0709 13:13:12.506124 (+  6866us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:12.467944  9739 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:12.445742  9645 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:12.421918  8693 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:13:12.448112  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 338864us
I20260709 13:13:12.448722  8208 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:12.509840  8104 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:12.470112  8206 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:12.468143  8060 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:12.471827  8132 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:12.475461  8210 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:12.473417  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.476920  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.486920  9655 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.514182  9416 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:12.514544  9655 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:12.514743  9416 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:12.514465  8176 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:12.513630  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.459270  8205 rpcz_store.cc:276] 0709 13:13:10.716087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.629417 (+913330us) service_pool.cc:224] Handling call
0709 13:13:11.629911 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:11.630325 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:11.809112 (+178787us) log.cc:844] Serialized 14 byte log entry
0709 13:13:11.946429 (+137317us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:11.963701 (+ 17272us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:11.971353 (+  7652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:12.014018 (+ 42665us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:12.078140 (+ 64122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34561,"raft.queue_time_us":39492,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":75927,"thread_start_us":72200,"threads_started":1,"wal-append.queue_time_us":336252}
I20260709 13:13:12.490579  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:12.493692  8211 raft_consensus.cc:2410] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:12.501807  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 782) took 1794 ms. Trace:
I20260709 13:13:12.518417  8137 rpcz_store.cc:276] 0709 13:13:10.707833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.500432 (+1792599us) service_pool.cc:224] Handling call
0709 13:13:12.501739 (+  1307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:12.496874  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.444005  9561 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:12.478612  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 66127.4us
W20260709 13:13:12.505195  8131 rpcz_store.cc:269] 0709 13:13:10.709740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.896406 (+1186666us) service_pool.cc:224] Handling call
0709 13:13:12.504535 (+608129us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:12.513833  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 107434us
I20260709 13:13:12.491739  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.524166  8207 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:12.485339  9556 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:13:12.522053  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 526062us
I20260709 13:13:12.543915  9556 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:12.521983  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.545545  8208 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:12.546638  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.547592  8208 raft_consensus.cc:2410] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:12.548518  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.549214  8208 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:12.520517  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:12.551097  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.536964  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.551492  9655 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:12.509635  9652 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:12.552022  8137 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:12.552304  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.552999  8134 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:12.552820  9652 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:12.553150  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.553867  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:12.538939  9751 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:13:12.540437  9416 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:12.540932  8210 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:12.541884  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.562899  8203 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:12.563102  8131 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:12.564090  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.542706  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:13:12.565335  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 53434.9us
I20260709 13:13:12.566005  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:12.566543  9734 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:12.566960  9734 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:12.567296  9734 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:12.568235  9706 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:12.568766  9706 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:12.553905  8137 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:12.573128  8104 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:12.573308  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 796) took 1020 ms. Trace:
I20260709 13:13:12.537122  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.550143  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.573772  8137 rpcz_store.cc:276] 0709 13:13:11.582439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.552850 (+970411us) service_pool.cc:224] Handling call
0709 13:13:12.573241 (+ 20391us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:12.574436  8208 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:12.566974  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:12.554570  8138 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:12.366112  8204 rpcz_store.cc:276] 0709 13:13:09.431842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:09.432225 (+   383us) service_pool.cc:224] Handling call
0709 13:13:09.474326 (+ 42101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:09.481879 (+  7553us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:11.605264 (+2123385us) log.cc:844] Serialized 14 byte log entry
0709 13:13:11.801045 (+195781us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:11.866842 (+ 65797us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:11.876476 (+  9634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:11.883062 (+  6586us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:11.912511 (+ 29449us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":412109,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":43,"raft.queue_time_us":24614,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":27365,"thread_start_us":2058152,"threads_started":2,"wal-append.queue_time_us":145003}
I20260709 13:13:12.515516  9678 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:12.555642  8176 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:12.557976  8209 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:12.546043  9556 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:12.556044  8027 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:12.543237  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:12.566135  8693 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:12.524673  9534 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:13:12.581336  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 64327us
I20260709 13:13:12.585043  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.584506  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.586798  8137 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:12.580163  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.558730  9645 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:12.580928  8212 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:12.589419  8206 raft_consensus.cc:2410] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:13:12.589735  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:12.590658  8135 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:12.591589  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:12.607834  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.608911  8137 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:12.592552  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:12.585042  9655 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:12.578081  8134 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:12.616492  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:12.608065  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 52161.4us
I20260709 13:13:12.617607  8130 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:13:12.611312  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 44488.1us
I20260709 13:13:12.619011  8176 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:12.619077  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:12.611565  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.582620  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.578289  9739 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:13:12.562319  9660 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:12.623219  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.607612  9645 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:12.608325  9624 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:12.636133  8206 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:12.584261  9534 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:12.583225  9710 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:12.636276  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.652040  8130 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
W20260709 13:13:12.636518  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 54406.4us
I20260709 13:13:12.609786  9556 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:13:12.593541  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 118953us
I20260709 13:13:12.617709  9655 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:12.627854  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.654436  8135 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:12.633828  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:12.648733  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.592998  9416 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:12.588269  8129 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:12.650261  8212 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:12.593952  8104 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:12.673902  8027 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:12.633693  9609 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:12.655187  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:12.655454  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:12.685004  8206 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:12.685283  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:12.685853  8029 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:12.611933  9744 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:12.656792  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:12.686547  8210 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:12.686935  9731 raft_consensus.cc:3055] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:12.686579  9744 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:12.687831  9744 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:12.688030  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 142) took 1262 ms. Trace:
I20260709 13:13:12.688390  8210 rpcz_store.cc:276] 0709 13:13:11.425438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.687239 (+1261801us) service_pool.cc:224] Handling call
0709 13:13:12.687522 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:12.687738 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:12.687762 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:12.687798 (+    36us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:12.687823 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:12.687871 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:12.687964 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:12.688812  8207 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:12.689198  9734 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:12.622089  9678 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:12.658980  9561 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:12.658191  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.661809  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:12.661981  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:12.662598  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:12.662428  9556 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:12.622124  9758 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:12.647459  9645 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:12.647225  9624 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:12.651607  9710 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:12.663487  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:12.655987  9655 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:12.638031  9660 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:12.666734  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:12.668166  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.671164  8102 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:12.672257  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.636307  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:12.681479  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:12.653295  9534 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:12.683786  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.656867  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:12.657387  9479 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:12.657900  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.689945  8210 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:13:13.000237  8728 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:13.000522  8136 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
W20260709 13:13:12.998642  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 179509us
W20260709 13:13:12.764106  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 175437us
I20260709 13:13:13.002784  8132 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:13.003347  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1108) took 1295 ms. Trace:
I20260709 13:13:13.003841  8135 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:12.685849  8212 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.001665  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2045) took 1266 ms. Trace:
I20260709 13:13:12.751806  8137 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:13.004660  8210 rpcz_store.cc:276] 0709 13:13:11.735392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.689345 (+953953us) service_pool.cc:224] Handling call
0709 13:13:12.689660 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:13.001395 (+311735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.001611 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:13.004724  8061 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
W20260709 13:13:12.999423  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 315817us
W20260709 13:13:12.719441  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 45627.2us
I20260709 13:13:13.001204  8130 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
I20260709 13:13:12.944643  8209 raft_consensus.cc:2410] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:13:12.790208  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 104471us
I20260709 13:13:12.641698  8205 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:12.728825  8206 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
W20260709 13:13:13.009743  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 392688us
I20260709 13:13:13.003268  8062 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:13.003914  8132 rpcz_store.cc:276] 0709 13:13:11.714135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.667920 (+953785us) service_pool.cc:224] Handling call
0709 13:13:13.003288 (+335368us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:13.015730  8026 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.012s	sys 0.006s
W20260709 13:13:13.016130  8026 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.012s	sys 0.006s
W20260709 13:13:13.017326  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 646610us
W20260709 13:13:13.017105  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 398093us
I20260709 13:13:12.675021  9416 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.025007  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.026836  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:12.764348  9724 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:12.730140  9587 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.040696  8696 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:13.044167  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.049352  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:12.722041  9593 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:13:13.011322  9620 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.000s	sys 0.012s
I20260709 13:13:13.011543  9660 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.042156  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.053231  8135 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:12.684592  9609 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:12.790489  9747 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:12.776113  8725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 304) took 1314 ms. Trace:
W20260709 13:13:13.017787  8726 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.005s	sys 0.013s
I20260709 13:13:13.009181  9710 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.042912  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.054870  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.039172  9416 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.039894  9600 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:13.055558  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:13.056104  8029 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.031996  8104 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.058429  8730 raft_consensus.cc:2368] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
I20260709 13:13:13.058661  9616 consensus_queue.cc:1059] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:13.059226  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2047) took 1323 ms. Trace:
I20260709 13:13:13.080119  8207 rpcz_store.cc:276] 0709 13:13:11.735998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.717768 (+981770us) service_pool.cc:224] Handling call
0709 13:13:13.024124 (+306356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.039691 (+ 15567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.039735 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:13.058963 (+ 19228us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:13.059001 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.059059 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.059178 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:13.059599  9734 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:13.045650  9724 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:13.016307  9624 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:12.730189  9744 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:12.789363  9706 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.061848  8212 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:13.026175  8204 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:13.062918  9500 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:12.687361  9731 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:13.047895  8696 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.068900  9630 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:13.008836  9758 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:13.070905  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.071810  9609 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:13.074695  8726 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.005s	sys 0.013s
I20260709 13:13:13.079022  9600 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:13:13.064816  9620 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.001s	sys 0.012s
I20260709 13:13:13.071738  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:13.080492  8130 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:13.049903  9561 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:13.046099  9587 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:13.090843  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 79618.1us
I20260709 13:13:13.096081  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.114749  9500 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:13.115144  9500 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.115258  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.115507  9500 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.115824  8135 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:13.116652  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.117692  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.099222  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.118646  8132 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:13.119501  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.120075  8132 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:13.121156  8132 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:13.121687  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1107) took 3233 ms. Trace:
I20260709 13:13:13.100450  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.122861  9731 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.123435  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 798) took 1530 ms. Trace:
I20260709 13:13:13.123746  8137 rpcz_store.cc:276] 0709 13:13:11.593340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.071867 (+1478527us) service_pool.cc:224] Handling call
0709 13:13:13.123384 (+ 51517us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:13.124112  9731 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.125087  8137 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:13.125617  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 781) took 2418 ms. Trace:
I20260709 13:13:13.069851  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:13.126614  8721 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
I20260709 13:13:13.127449  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:13.128726  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:13.104761  9758 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.129509  8721 raft_consensus.cc:2393] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:13.113243  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.132159  8130 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:13.090376  8724 raft_consensus.cc:2410] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:13:13.070318  9747 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:13.103777  8136 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:13.064636  9593 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:13.103221  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2061) took 1284 ms. Trace:
I20260709 13:13:13.088001  9724 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.082316  8129 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:13.100318  9756 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.113492  9561 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.131428  9587 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.121994  8132 rpcz_store.cc:276] 0709 13:13:09.888077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.120671 (+3232594us) service_pool.cc:224] Handling call
0709 13:13:13.120992 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.121463 (+   471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.121620 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:13.104982  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 57168.3us
W20260709 13:13:13.161926  8729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 462) took 2069 ms (client timeout 1839 ms). Trace:
W20260709 13:13:13.163666  8729 rpcz_store.cc:269] 0709 13:13:11.092996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.608725 (+515729us) service_pool.cc:224] Handling call
0709 13:13:13.161745 (+1553020us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54004d2e54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:13.161858 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832752000}
I20260709 13:13:13.138129  8104 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.103878  9770 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:13.147643  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:13.125905  8137 rpcz_store.cc:276] 0709 13:13:10.707578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.124428 (+2416850us) service_pool.cc:224] Handling call
0709 13:13:13.124741 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.125397 (+   656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.125552 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:13.108824  9600 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:13.145174  9747 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:13.109378  9766 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.160904  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:13.118217  8135 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:13.186702  8130 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:13.187713  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:13.158032  9756 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.145447  9593 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:13.160540  9561 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.156158  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:13.166541  9770 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.199796  8136 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:13:13.147011  8204 rpcz_store.cc:276] 0709 13:13:11.819202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.593111 (+773909us) service_pool.cc:224] Handling call
0709 13:13:13.102678 (+509567us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:13.168503  9600 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:13.196625  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 150) took 2082 ms. Trace:
I20260709 13:13:13.204101  8137 rpcz_store.cc:276] 0709 13:13:11.115638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.184935 (+2069297us) service_pool.cc:224] Handling call
0709 13:13:13.185371 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.196243 (+ 10872us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.196272 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:13.196345 (+    73us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:13.196379 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.196442 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.196556 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:13.188535  9766 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.206574  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 147) took 2232 ms. Trace:
I20260709 13:13:13.210029  8136 rpcz_store.cc:276] 0709 13:13:10.968370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.157979 (+2189609us) service_pool.cc:224] Handling call
0709 13:13:13.199518 (+ 41539us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:13.200247 (+   729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.200457 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:13.210451  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 49620.3us
I20260709 13:13:13.160460  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 802) took 1396 ms. Trace:
W20260709 13:13:13.214649  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 417547us
I20260709 13:13:13.194659  9756 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.236999  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 152) took 1505 ms. Trace:
I20260709 13:13:13.237479  8135 rpcz_store.cc:276] 0709 13:13:11.731693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.196910 (+1465217us) service_pool.cc:224] Handling call
0709 13:13:13.208907 (+ 11997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.221028 (+ 12121us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.221053 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:13.236714 (+ 15661us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:13.236784 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.236848 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.236943 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:13.219852  8129 rpcz_store.cc:276] 0709 13:13:11.768648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.516485 (+747837us) service_pool.cc:224] Handling call
0709 13:13:13.157893 (+641408us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:13.202890  9652 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:13.247655  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:13.251374  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:13.247646  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 192880us
W20260709 13:13:13.251875  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.206387  9770 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.073038  8725 rpcz_store.cc:276] 0709 13:13:11.483824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.410076 (+926252us) service_pool.cc:224] Handling call
0709 13:13:12.477566 (+ 67490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:12.505482 (+ 27916us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:12.505527 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:12.505593 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:12.505640 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:12.505733 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:12.775856 (+270123us) spinlock_profiling.cc:243] Waited 45.5 ms on lock 0x7b540060f750. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:12.776016 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104692480}
I20260709 13:13:13.258087  8061 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:13.259694  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:13.260377  8137 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.213582  9766 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.222167  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:13.233299  9620 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.220790  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 108) took 1331 ms. Trace:
I20260709 13:13:13.264843  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2064) took 1442 ms. Trace:
I20260709 13:13:13.221724  9556 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:13.267199  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.260674  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:13.253103  9678 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:13.254540  9659 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:13:13.290202  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 157038us
I20260709 13:13:13.283043  8210 rpcz_store.cc:276] 0709 13:13:11.822887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.020964 (+1198077us) service_pool.cc:224] Handling call
0709 13:13:13.048588 (+ 27624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.264067 (+215479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.264111 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:13.264500 (+   389us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:13.264545 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.264627 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.264780 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10529,"raft.queue_time_us":99161,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":22776,"thread_start_us":205078,"threads_started":1}
I20260709 13:13:13.238524  9710 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:13.316860  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 63256.4us
I20260709 13:13:13.269606  9652 raft_consensus.cc:697] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:13.331216  8104 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.284421  9479 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:13.339962  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 142528us
I20260709 13:13:13.303165  9659 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.288980  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:13.325572  9706 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:13.311522  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:13.357167  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:13.351928  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 308253us
I20260709 13:13:13.358213  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:13.352618  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.354542  8137 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
W20260709 13:13:13.360702  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 158924us
I20260709 13:13:13.343402  9659 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:13.373997  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 702861us
I20260709 13:13:13.375415  8102 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:13.379026  9616 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.379441  9616 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:13.379823  9616 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:13.402900  8138 consensus_meta.cc:287] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.673s	user 0.008s	sys 0.002s
I20260709 13:13:13.402978  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:13.403373  8138 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:13.403640  8207 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.404814  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 812) took 1554 ms. Trace:
I20260709 13:13:13.405166  8138 rpcz_store.cc:276] 0709 13:13:11.850596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.404029 (+1553433us) service_pool.cc:224] Handling call
0709 13:13:13.404295 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.404493 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.404522 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:13.404568 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:13.404600 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.404656 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.404765 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:13.406430  8723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 311) took 1602 ms (client timeout 1546 ms). Trace:
W20260709 13:13:13.406813  8723 rpcz_store.cc:269] 0709 13:13:11.803604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.808073 (+  4469us) service_pool.cc:224] Handling call
0709 13:13:13.367484 (+1559411us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54007611d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:13.406364 (+ 38880us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564028160}
I20260709 13:13:13.407846  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:13.408545  8723 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:13.409297  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:13.410521  9763 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.280s	user 0.001s	sys 0.001s
I20260709 13:13:13.348856  9744 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:13.411446  9744 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:13.412707  9744 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:13.412776  8723 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 3 LEARNER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 13:13:13.413247  9744 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:13.414579  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.302523  9620 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.418597  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2049) took 1682 ms. Trace:
I20260709 13:13:13.419004  8206 rpcz_store.cc:276] 0709 13:13:11.736545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.009935 (+1273390us) service_pool.cc:224] Handling call
0709 13:13:13.049156 (+ 39221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.062665 (+ 13509us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:13.257488 (+194823us) log.cc:844] Serialized 14 byte log entry
0709 13:13:13.418267 (+160779us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:13.418321 (+    54us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:13.418379 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.418441 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.418549 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":305697,"thread_start_us":286201,"threads_started":2}
I20260709 13:13:13.397814  9500 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:13.415769  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 125904us
I20260709 13:13:13.358633  9748 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:13.432701  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.434604  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:13.433507  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:13.437299  8210 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.437381  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:13.437639  8029 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.442852  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:13:13.446439  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 94210.4us
I20260709 13:13:13.434445  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:13.423254  9785 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.455878  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:13.456384  9500 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:13.457376  9774 log.cc:927] Time spent T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:13:13.458022  9620 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:13.436628  9748 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.442643  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:13.459415  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.463909  9784 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:13:13.458036  9779 log.cc:927] Time spent T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:13:13.465462  9795 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:13.466549  9534 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:13.467285  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:13.439651  9778 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:13.473371  8207 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
W20260709 13:13:13.448346  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 120373us
I20260709 13:13:13.475837  8210 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:13.458431  9652 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:13.477015  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:13.491950  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.493770  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:13.453763  9739 consensus_queue.cc:1059] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:13:13.454180  8138 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:13.527122  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 816) took 1110 ms. Trace:
I20260709 13:13:13.527514  8138 rpcz_store.cc:276] 0709 13:13:12.416947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.406091 (+989144us) service_pool.cc:224] Handling call
0709 13:13:13.417399 (+ 11308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.526824 (+109425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.527040 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:13.534889  9764 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.203s	user 0.000s	sys 0.000s
I20260709 13:13:13.536213  9758 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:13.508224  8209 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2056) took 1734 ms (client timeout 2198 ms). Trace:
I20260709 13:13:13.538787  8728 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:13:13.538674  8209 rpcz_store.cc:269] 0709 13:13:11.773412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.494434 (+721022us) service_pool.cc:224] Handling call
0709 13:13:13.500906 (+1006472us) spinlock_profiling.cc:243] Waited 62.7 ms on lock 0x7b5400189554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:13.508159 (+  7253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144210816}
I20260709 13:13:13.508671  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 783) took 2785 ms. Trace:
I20260709 13:13:13.540095  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 312) took 1693 ms. Trace:
I20260709 13:13:13.540215  8130 rpcz_store.cc:276] 0709 13:13:10.708138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.132884 (+2424746us) service_pool.cc:224] Handling call
0709 13:13:13.186411 (+ 53527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.187040 (+   629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.493236 (+306196us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b54000d20d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:13.493388 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42550144}
I20260709 13:13:13.506632  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.500216  9748 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.448989  9776 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:13.449638  9780 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:13.513751  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 54460.9us
I20260709 13:13:13.544160  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.471517  9785 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.514891  8730 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:13.470046  9775 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:13.548528  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 71598.8us
I20260709 13:13:13.520522  8133 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:13.520681  9630 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.525664  9710 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:13.521204  8203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2058) took 1694 ms (client timeout 1512 ms). Trace:
I20260709 13:13:13.540683  8727 rpcz_store.cc:276] 0709 13:13:11.847003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.488275 (+641272us) service_pool.cc:224] Handling call
0709 13:13:12.488655 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:12.489162 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:12.489202 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:12.489803 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 13:13:12.999935 (+510132us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:12.999973 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.000035 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:13.539873 (+539838us) raft_consensus.cc:1685] finished
0709 13:13:13.539898 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.540033 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1375,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"thread_start_us":255845,"threads_started":1,"wal-append.queue_time_us":764664}
I20260709 13:13:13.478381  8724 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
W20260709 13:13:13.546378  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.497258  9784 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:13.512153  9795 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:13.546988  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 77701.9us
I20260709 13:13:13.551273  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:13.551716  8104 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:13.552233  8133 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 797) took 1959 ms (client timeout 2242 ms). Trace:
I20260709 13:13:13.522276  9791 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:13.579851  8133 rpcz_store.cc:269] 0709 13:13:11.592996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.504026 (+911030us) service_pool.cc:224] Handling call
0709 13:13:13.477502 (+973476us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b5c001b040c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbe980161 00007f6dbeb21a79 00007f6dbffdb9b6 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:13.552166 (+ 74664us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49549056}
I20260709 13:13:13.549304  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:13.475682  9766 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:13.554116  9500 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.515221  9778 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.544873  9776 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:13.550480  9630 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.529963  9416 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:13.516954  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 314) took 1025 ms. Trace:
W20260709 13:13:13.582790  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 207438us
I20260709 13:13:13.583685  9795 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.534098  9747 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:13.630105  8104 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.626935  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:13.559363  8204 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 2: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:13.551896  9785 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:13.658532  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 164783us
I20260709 13:13:13.551796  9775 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.629011  8029 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.554546  9678 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.588636  9739 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.590590  8131 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:13.547379  9609 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.625255  8102 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:13.588466  9500 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:13:13.680001  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 91452.4us
I20260709 13:13:13.592806  9630 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.588864  9780 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.581868  9784 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:13.630698  9758 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:13.683044  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 54130.1us
I20260709 13:13:13.584192  9791 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.590039  9778 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.594801  9744 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:13.560940  9794 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:13:13.627102  9768 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
W20260709 13:13:13.627106  9655 consensus_meta.cc:287] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.578s	user 0.004s	sys 0.004s
I20260709 13:13:13.683683  9500 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:13.554405  8203 rpcz_store.cc:269] 0709 13:13:11.818010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.902604 (+ 84594us) service_pool.cc:224] Handling call
0709 13:13:13.490986 (+1588382us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b5400110154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:13.508026 (+ 17040us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":550213888}
I20260709 13:13:13.591441  9776 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.558733  9660 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:13.629043  9645 consensus_meta.cc:287] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.000s
W20260709 13:13:13.696647  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 107407us
I20260709 13:13:13.693986  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.710103  9645 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.710724  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.711740  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.712118  9645 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:13.712929  8721 raft_consensus.cc:2368] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4.
I20260709 13:13:13.714738  9792 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:13.706069  9655 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:13.715155  9792 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:13.715569  9792 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.716598  9655 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:13.717377  8134 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:13.718623  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:13.719816  9782 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.720238  9782 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:13.720600  9782 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.720537  9500 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.626240  9724 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.698740  8727 raft_consensus.cc:2368] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4.
I20260709 13:13:13.698153  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:13.729185  8130 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.729445  9800 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:13.729809  9800 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:13.702386  8696 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:13.677906  9739 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.734405  9556 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.734644  9739 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.734948  9556 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.735392  9556 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.679493  9678 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.730154  9800 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:13.738405  9652 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:13.738497  9678 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.695186  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.739084  9652 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.739648  8136 raft_consensus.cc:2368] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3.
I20260709 13:13:13.739583  9500 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:13.678318  9775 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.706146  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:13.623165  9600 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.591032  9624 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.708212  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:13.744205  8133 raft_consensus.cc:2410] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:13:13.624599  9593 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:13.749755  9789 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:13:13.750207  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:13.625159  9770 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.630029  9534 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:13.285821  8054 rpcz_store.cc:276] 0709 13:13:11.880868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.898738 (+ 17870us) service_pool.cc:224] Handling call
0709 13:13:11.899200 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:12.117810 (+218610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:12.166006 (+ 48196us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:12.371406 (+205400us) log.cc:844] Serialized 26 byte log entry
0709 13:13:12.648324 (+276918us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:13.149646 (+501322us) log.cc:844] Serialized 14 byte log entry
0709 13:13:13.188509 (+ 38863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.196837 (+  8328us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:13.198930 (+  2093us) raft_consensus.cc:1685] finished
0709 13:13:13.203602 (+  4672us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.212361 (+  8759us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23417,"prepare.queue_time_us":109324,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":278318,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":45558,"thread_start_us":814235,"threads_started":3,"wal-append.queue_time_us":370610}
I20260709 13:13:13.758188  9782 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.696569  9780 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:13.762555  9780 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.763063  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:13.726758  9804 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:13.727241  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.727447  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:13.727702  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 102518us
I20260709 13:13:13.727185  9734 raft_consensus.cc:3055] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:13.728229  8207 raft_consensus.cc:1240] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:13.703054  9745 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.766422  9782 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:13.700650  9791 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.752207  8130 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:13.707655  9788 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:13.752003  9758 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.767746  8132 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:13.723479  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.818964  8209 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:13.750567  9593 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:13.767854  8102 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:13.741199  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.739611  9652 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:13.803828  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 47784.8us
I20260709 13:13:13.823736  9745 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:13.832336  9745 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:13.832710  9561 consensus_meta.cc:287] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.620s	user 0.001s	sys 0.005s
I20260709 13:13:13.821185  9758 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.833202  9758 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.833076  9561 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:13.835201  9724 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:13.818274  9788 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:13.835675  9624 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:13.836014  9788 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:13.804570  8729 raft_consensus.cc:2368] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3.
I20260709 13:13:13.766538  9804 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:13.838021  9804 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:13.839388  9600 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:13.744223  9655 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:13.807587  9778 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:13.769845  9744 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:13:13.808555  8176 leader_election.cc:400] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
W20260709 13:13:13.809352  9587 consensus_meta.cc:287] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.621s	user 0.003s	sys 0.003s
W20260709 13:13:13.811515  9797 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.183s	user 0.003s	sys 0.001s
W20260709 13:13:13.817111  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 91449.5us
I20260709 13:13:13.750751  9500 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:13.767223  9645 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:13:13.825413  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 84272.6us
W20260709 13:13:13.840556  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 72766.1us
W20260709 13:13:13.841791  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 118410us
I20260709 13:13:13.804023  9734 raft_consensus.cc:740] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:13.772858  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:13.878669  8696 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.875798  8176 leader_election.cc:403] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:13:13.845863  8027 leader_election.cc:400] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3.
W20260709 13:13:13.876173  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 183023us
I20260709 13:13:13.883052  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:13.886845  8027 leader_election.cc:403] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:13.879119  8175 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:13.884744  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.888865  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.888993  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1130) took 1167 ms. Trace:
I20260709 13:13:13.889545  8135 rpcz_store.cc:276] 0709 13:13:12.721979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.493876 (+771897us) service_pool.cc:224] Handling call
0709 13:13:13.518995 (+ 25119us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:13.538567 (+ 19572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.538595 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:13.557779 (+ 19184us) log.cc:844] Serialized 26 byte log entry
0709 13:13:13.858019 (+300240us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:13.859639 (+  1620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.859709 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:13.888701 (+ 28992us) raft_consensus.cc:1685] finished
0709 13:13:13.888727 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.888920 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":214246,"threads_started":1,"wal-append.queue_time_us":298006}
I20260709 13:13:13.891819  9778 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:13.895736  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 87398.7us
I20260709 13:13:13.896190  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 116) took 1233 ms. Trace:
I20260709 13:13:13.885816  8102 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:13.855669  9710 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.901118  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:13.843284  9655 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.824271  9805 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:13.906898  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.907984  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:13.595239  8726 rpcz_store.cc:276] 0709 13:13:12.479740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.493312 (+ 13572us) service_pool.cc:224] Handling call
0709 13:13:12.493752 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:12.494089 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:13.130747 (+636658us) log.cc:844] Serialized 14 byte log entry
0709 13:13:13.444057 (+313310us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:13.449315 (+  5258us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:13.452333 (+  3018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.476380 (+ 24047us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.498290 (+ 21910us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":614638,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":92084,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":850324,"threads_started":3,"wal-append.queue_time_us":307140}
I20260709 13:13:13.900171  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:13.903545  9778 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
I20260709 13:13:13.895361  9645 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:13.941672  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:13.955827  8131 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1109) took 2237 ms (client timeout 2564 ms). Trace:
I20260709 13:13:13.902859  9766 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:13.930588  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:13.900367  8129 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:13.934077  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:13.940483  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 99047.1us
I20260709 13:13:13.973382  9645 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.974124  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:13.903295  9781 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:13.974716  8058 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:13.974876  9781 raft_consensus.cc:697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:13.975564  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:13.920661  9784 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.976182  8058 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.975839  9781 consensus_queue.cc:237] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.976950  8729 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 3 LEARNER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 13:13:13.977772  9784 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:13.918788  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:13.978824  8729 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:13.978909  8212 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.980499  7959 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34120 (request call id 245) took 9561 ms. Trace:
I20260709 13:13:13.972388  8207 raft_consensus.cc:2393] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:13.903332  9710 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:13.967546  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:13.876866  9798 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:13.919994  9790 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:13.926457  9620 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:13.932407  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:13.858963  9806 raft_consensus.cc:2764] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:13:14.143703  8055 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:13.936436  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:13.927016  9734 consensus_queue.cc:260] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.981794  8726 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:14.139370  8101 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:14.008922  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1112) took 2250 ms. Trace:
I20260709 13:13:14.087004  9710 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.913699  9660 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:13.936576  9744 raft_consensus.cc:3055] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:13.939886  8061 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:13:13.939160  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 187751us
I20260709 13:13:14.208190  8212 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
W20260709 13:13:14.159261  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 273509us
I20260709 13:13:14.087257  9706 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:13:13.919924  9756 consensus_meta.cc:287] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.665s	user 0.005s	sys 0.003s
I20260709 13:13:14.180537  8137 rpcz_store.cc:276] 0709 13:13:11.766580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.372293 (+1605713us) service_pool.cc:224] Handling call
0709 13:13:13.372584 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:13.696134 (+323550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.696164 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:13.706498 (+ 10334us) log.cc:844] Serialized 26 byte log entry
0709 13:13:13.876448 (+169950us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:13.876472 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.876518 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:14.008677 (+132159us) raft_consensus.cc:1685] finished
0709 13:13:14.008695 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:14.008853 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11795,"raft.queue_time_us":272252,"raft.run_cpu_time_us":6805,"raft.run_wall_time_us":38395,"spinlock_wait_cycles":1642752,"thread_start_us":376573,"threads_started":2,"wal-append.queue_time_us":228196}
I20260709 13:13:13.947149  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 149) took 2836 ms. Trace:
I20260709 13:13:13.960186  9739 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.211905  8132 rpcz_store.cc:276] 0709 13:13:11.110721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.186526 (+2075805us) service_pool.cc:224] Handling call
0709 13:13:13.223953 (+ 37427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:13.818537 (+594584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.946857 (+128320us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b5400062054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:13.947054 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250241792}
W20260709 13:13:13.957263  9616 consensus_meta.cc:287] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.577s	user 0.005s	sys 0.003s
I20260709 13:13:14.140393  9766 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:13.913143  9770 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:13.940701  9580 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:13.883862  9587 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:13.940234  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:14.144665  9782 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:14.037976  9655 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:13.945277  9600 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:14.206606  9744 raft_consensus.cc:740] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:14.204289  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 140499us
I20260709 13:13:14.205569  9806 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:14.058456  9561 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:14.201795  8026 leader_election.cc:336] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.836s (SENT)
I20260709 13:13:13.957283  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.275142  8203 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:13.961519  9534 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:14.276043  9534 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:14.276404  9534 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.280117  9580 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3.
W20260709 13:13:14.281368  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 79604.1us
I20260709 13:13:13.953876  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1118) took 1634 ms. Trace:
I20260709 13:13:14.147325  9790 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:13:14.282131  9776 consensus_meta.cc:287] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.555s	user 0.006s	sys 0.002s
I20260709 13:13:14.282675  9776 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.283304  9790 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.284700  9776 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:14.243104  9766 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:13.955268  9772 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.588s	user 0.001s	sys 0.001s
W20260709 13:13:14.171571  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 282779us
I20260709 13:13:14.281977  8134 rpcz_store.cc:276] 0709 13:13:12.319634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.646768 (+327134us) service_pool.cc:224] Handling call
0709 13:13:13.953647 (+1306879us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b5400042754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:13.953807 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1628390656}
I20260709 13:13:13.926198  7961 catalog_manager.cc:5697] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 3 to 4, term changed from 2 to 3, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to <none>, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:13.953295  9651 raft_consensus.cc:3055] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:14.216666  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2066) took 2197 ms. Trace:
I20260709 13:13:14.216795  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.220810  8058 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:14.099591  9803 consensus_queue.cc:1059] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.149s
I20260709 13:13:14.289178  9806 consensus_queue.cc:1059] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:14.287720  8205 rpcz_store.cc:276] 0709 13:13:12.019133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.064167 (+1045034us) service_pool.cc:224] Handling call
0709 13:13:13.108996 (+ 44829us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:13.206354 (+ 97358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:13.211290 (+  4936us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:13.311355 (+100065us) log.cc:844] Serialized 26 byte log entry
0709 13:13:13.510520 (+199165us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:13.510581 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.510762 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:13.809745 (+298983us) spinlock_profiling.cc:243] Waited 4.06 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:14.209771 (+400026us) raft_consensus.cc:1685] finished
0709 13:13:14.210753 (+   982us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:14.216595 (+  5842us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125799,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":23901,"spinlock_wait_cycles":9332608,"thread_start_us":137934,"threads_started":1,"wal-append.queue_time_us":284764}
I20260709 13:13:14.290813  9806 raft_consensus.cc:3055] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:14.291184  9806 raft_consensus.cc:740] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:14.291525  9744 consensus_queue.cc:260] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:14.291960  9806 consensus_queue.cc:260] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.292690  9744 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.293100  9806 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:13.919273  9609 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:13.921743  9659 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.295998  9659 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:14.222579  9756 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.296983  8062 raft_consensus.cc:2368] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
I20260709 13:13:14.297891  9756 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.298666  8060 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:14.299214  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.299757  8062 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:14.299942  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:14.300504  8060 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:14.300683  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.301226  8062 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:14.301960  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.224509  9804 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:13.917258  9800 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.303725  9804 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.303854  9800 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:14.227056  8696 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:14.228401  9791 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.305186  9781 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:14.305907  9791 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.305683  9781 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.971107  8054 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:14.307031  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.307768  8054 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:14.307721  9781 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.308651  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:14.309283  8054 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:14.309792  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 82794.7us
I20260709 13:13:14.309957  9781 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:14.228802  9416 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:14.178285  9778 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:13:14.310509  9781 raft_consensus.cc:697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:14.229779  9620 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:14.311568  9620 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:14.311371  9781 consensus_queue.cc:237] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:14.231284  9748 consensus_meta.cc:287] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.674s	user 0.007s	sys 0.000s
I20260709 13:13:14.314613  9748 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.317030  9748 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.232575  9788 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.233398  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:14.235073  9785 consensus_meta.cc:287] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.541s	user 0.005s	sys 0.002s
I20260709 13:13:14.318753  8133 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 2: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:14.318624  9785 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.320199  8055 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:14.320627  9785 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:14.236353  9792 consensus_meta.cc:287] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.520s	user 0.007s	sys 0.000s
I20260709 13:13:14.321333  9792 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.321715  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:13.896559  8063 rpcz_store.cc:276] 0709 13:13:12.663051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.663958 (+   907us) service_pool.cc:224] Handling call
0709 13:13:12.664269 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:12.664473 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:13.032314 (+367841us) log.cc:844] Serialized 14 byte log entry
0709 13:13:13.807108 (+774794us) spinlock_profiling.cc:243] Waited 6.71 ms on lock 0x7b18000858a0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbb00e060 00007f6dbb020716 00007f6dbb031287 00007f6dbb031208
0709 13:13:13.895851 (+ 88743us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:13.895910 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:13.895953 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:13.896028 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:13.896140 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13984,"prepare.queue_time_us":98650,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":29811,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":21918,"spinlock_wait_cycles":15434112,"thread_start_us":616912,"threads_started":3,"wal-append.queue_time_us":288127}
I20260709 13:13:14.238221  9706 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:13.919698  9556 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.104645  9816 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:13.918905  9747 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:13.936712  9805 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:14.214988  8056 raft_consensus.cc:2393] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:13.960484  9630 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.245800  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:13.918655  9758 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.248680  8057 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:14.322312  8055 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.240476  9731 consensus_queue.cc:1059] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:13:13.964709  9795 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.335072  9731 raft_consensus.cc:3055] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:14.335439  9731 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:14.336042  9795 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:13.957572  9794 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:14.203949  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 302834us
I20260709 13:13:14.336300  9731 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.337715  9731 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:14.337987  8176 leader_election.cc:400] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4.
I20260709 13:13:14.338302  8176 leader_election.cc:403] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:13:14.324208  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.339128  8176 leader_election.cc:400] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4.
I20260709 13:13:14.339414  8176 leader_election.cc:403] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:13:14.325712  9706 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.251077  8212 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:14.327368  9556 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:14.327796  9788 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.328622  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.341924  9561 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4.
I20260709 13:13:14.330152  9630 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:14.342911  8057 raft_consensus.cc:2410] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:14.328559  8062 raft_consensus.cc:2393] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:14.287659  8104 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:14.343816  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.250476  9782 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:14.255743  9775 consensus_meta.cc:287] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.504s	user 0.005s	sys 0.002s
I20260709 13:13:14.255946  9600 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:14.350962  9600 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:14.103300  9754 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.141s
I20260709 13:13:13.980846  7959 rpcz_store.cc:276] 0709 13:13:04.341964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:04.342118 (+   154us) service_pool.cc:224] Handling call
0709 13:13:06.214230 (+1872112us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b5400172d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf80090 00007f6dbb01ef3a 00007f6dc0dd51cc 00007f6dc0ed337e 00007f6dbf3942a4 00007f6dbf394232 00007f6dbf394161 00007f6dbf3940dc 00007f6dbf393392 00007f6db935148c 00007f6db93508c6
0709 13:13:13.896862 (+7682632us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:09.184869 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:09.185394 (+   525us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:09.185429 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:13:09.185446 (+    17us) tablet.cc:662] Decoding operations
0709 13:13:09.185738 (+   292us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:09.185794 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:09.185825 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:13:09.185981 (+   156us) tablet.cc:701] Row locks acquired
0709 13:13:09.185997 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:13:09.186116 (+   119us) write_op.cc:270] Start()
0709 13:13:09.186212 (+    96us) write_op.cc:276] Timestamp: P: 1783602789186075 usec, L: 0
0709 13:13:09.186231 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:13:09.272246 (+ 86015us) log.cc:844] Serialized 533 byte log entry
0709 13:13:10.503566 (+1231320us) op_driver.cc:464] REPLICATION: finished
0709 13:13:11.641794 (+1138228us) spinlock_profiling.cc:243] Waited 39.5 ms on lock 0x7b1800092740. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f29522 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:13:11.654958 (+ 13164us) spinlock_profiling.cc:243] Waited 8.53 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6db7f0c0ac 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbea4cef0
0709 13:13:11.837727 (+182769us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b1800092740. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f29557 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:13:11.868612 (+ 30885us) write_op.cc:301] APPLY: starting
0709 13:13:11.868719 (+   107us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:11.868857 (+   138us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:11.868893 (+    36us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:11.869188 (+   295us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:11.869522 (+   334us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:13:11.869598 (+    76us) write_op.cc:312] APPLY: finished
0709 13:13:11.875285 (+  5687us) log.cc:844] Serialized 23 byte log entry
0709 13:13:11.908208 (+ 32923us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6db7f0c0ac 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b
0709 13:13:12.142951 (+234743us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:12.143110 (+   159us) write_op.cc:454] Released schema lock
0709 13:13:12.143368 (+   258us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:12.425104 (+281736us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:12.468621 (+ 43517us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":655104,"reported_tablets":9,"spinlock_wait_cycles":619273344,"child_traces":[["op",{"apply.queue_time_us":1220653,"apply.run_cpu_time_us":26628,"apply.run_wall_time_us":275315,"mutex_wait_us":26209,"num_ops":1,"prepare.queue_time_us":408679,"prepare.run_cpu_time_us":3074,"prepare.run_wall_time_us":102688,"raft.queue_time_us":806941,"raft.run_cpu_time_us":65130,"raft.run_wall_time_us":2198166,"replication_time_us":1302832,"spinlock_wait_cycles":121420160,"thread_start_us":2668324,"threads_started":5,"wal-append.queue_time_us":762356,"wal-append.run_cpu_time_us":23320,"wal-append.run_wall_time_us":2578569}]]}
I20260709 13:13:14.213488  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:14.259323  8059 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:14.247110  9624 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.264688  9652 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.244583  9616 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:14.200830  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 292914us
I20260709 13:13:14.273957  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:13.941148  9780 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.256484  9587 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.281456  9739 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.264828  9655 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.287644  9651 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:13.972239  8131 rpcz_store.cc:269] 0709 13:13:11.718585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:12.563835 (+845250us) service_pool.cc:224] Handling call
0709 13:13:13.955636 (+1391801us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b540035d4d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:13.955763 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2262695808}
I20260709 13:13:14.288591  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.288012  7961 catalog_manager.cc:5697] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 2, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:13.916244  9678 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.221144  9771 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:14.286042  9766 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:14.339236  8063 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:14.345126  9814 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:14.323575  9792 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.351881  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:14.400092  8057 raft_consensus.cc:2368] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
I20260709 13:13:14.400231  9794 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:14.400700  9780 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:14.401023  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.341934  8056 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:14.402961  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:14.403637  9652 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:14.404114  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.404991  8723 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:14.404458  7961 catalog_manager.cc:5697] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:14.405280  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:14.405818  8056 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.406459  9771 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:14.406651  9775 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.407158  9814 raft_consensus.cc:697] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:14.331269  9805 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.408174  9814 consensus_queue.cc:237] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.408736  9775 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.409118  9616 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:14.406908  9771 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.411245  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.411844  9655 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.411983  8059 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:14.412945  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.413435  8059 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:14.406921  8723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 325) took 1332 ms. Trace:
I20260709 13:13:14.285537  9710 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.286489  9787 consensus_queue.cc:1059] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.088s
W20260709 13:13:14.344435  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 56842.8us
I20260709 13:13:14.338094  8693 leader_election.cc:400] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3.
I20260709 13:13:14.406931  9754 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:14.414145  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:14.446877  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:14.373247  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:14.457899  8727 raft_consensus.cc:2393] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:14.401827  8057 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:13:14.423853  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 210405us
I20260709 13:13:14.439109  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:14.403649  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.460536  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.414173  8723 rpcz_store.cc:276] 0709 13:13:13.074762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:13.413919 (+339157us) service_pool.cc:224] Handling call
0709 13:13:13.414179 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:14.405377 (+991198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.406699 (+  1322us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b54002e1154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:14.406865 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2278273792}
I20260709 13:13:14.402983  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.440440  9534 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.406150  7961 catalog_manager.cc:5697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:14.398171  8203 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:14.418406  9600 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.401842  9651 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.331151  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:14.352334  9798 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.434280  8101 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:14.462278  9782 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:14.493023  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:14.535244  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.493264  8060 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:14.457664  9754 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:14.522526  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.523329  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.521368  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:14.524052  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.540633  8210 raft_consensus.cc:2368] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3.
I20260709 13:13:14.527146  8061 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:14.446494  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:14.496343  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.544171  8206 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.494300  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.530331  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:14.531994  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 194068us
W20260709 13:13:14.533118  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:14.494282  9825 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:13:14.551239  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:14.551877  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 117659us
I20260709 13:13:14.472119  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:14.537742  9820 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.193s	user 0.002s	sys 0.002s
I20260709 13:13:14.404412  9624 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.440007  9616 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:14.541749  8058 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
W20260709 13:13:14.542800  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 49835.8us
I20260709 13:13:14.527973  7961 catalog_manager.cc:5697] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 3, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 3 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:14.547680  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:14.559060  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 87012.2us
I20260709 13:13:14.559286  8210 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.528455  8063 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:14.531140  8725 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
W20260709 13:13:14.537947  9824 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:13:14.332705  9758 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:14.527858  9587 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:14.556679  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.569463  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.438777  9645 consensus_queue.cc:1059] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 13:13:14.571122  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.571688  8057 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:14.557946  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.436663  9805 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.527632  9766 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.560086  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.582252  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:14.527305  9561 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4.
I20260709 13:13:14.560699  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.560818  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.553992  8205 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.598658  8209 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.599283  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.599892  8205 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:14.551152  9816 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:14.562924  7961 catalog_manager.cc:5697] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:14.600994  8209 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:14.601269  7961 catalog_manager.cc:5697] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:14.601963  9794 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.602573  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.603329  8208 raft_consensus.cc:2368] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3.
I20260709 13:13:14.602873  7961 catalog_manager.cc:5697] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 2, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:14.604224  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.604408  9794 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.605082  8057 raft_consensus.cc:2410] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:13:14.607532  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:14.612134  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.443887  9620 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.565101  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.555218  9785 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:14.528976  9780 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:13:14.574683  9534 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.575294  9587 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.555748  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.564242  9652 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:14.551404  9747 consensus_queue.cc:1059] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:13:14.581296  8693 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:14.582409  8101 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:14.563687  9609 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:14.583163  8060 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:14.588099  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:14.565271  9651 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.592000  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.592785  8206 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:14.593219  8058 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:14.620525  9747 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:14.628949  9771 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.639712  8210 raft_consensus.cc:2393] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:14.594524  9823 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.171s	user 0.004s	sys 0.002s
I20260709 13:13:14.595417  8727 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:14.641045  9758 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:14.641536  9758 raft_consensus.cc:697] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:14.596922  9766 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:14.638015  9706 raft_consensus.cc:3055] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:14.619031  9587 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:14.643119  9706 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:13:14.637395  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 49364.5us
I20260709 13:13:14.612857  8208 raft_consensus.cc:2368] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3.
I20260709 13:13:14.644110  9706 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.645850  9561 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:14.645797  9776 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:14.623338  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:14.625017  8104 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:14.647646  9706 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:14.649770  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:14.639611  9747 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:14.629254  9754 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.626530  8207 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:14.569159  8722 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:14.637056  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.623539  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:14.631510  8062 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:14.612725  8061 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:14.651057  9747 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.613530  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:14.658654  9747 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.625176  9775 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:14.616816  8726 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:14.662563  9792 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:14.663286  9534 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:14.617031  9785 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:14.625274  9816 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:14.665630  8211 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.662473  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.653712  8054 raft_consensus.cc:2410] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:13:14.705065  8059 raft_consensus.cc:2410] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:14.702862  8210 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:14.619557  8204 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.664718  9785 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.564240  9616 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.621765  9620 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:14.642452  9758 consensus_queue.cc:237] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:14.704349  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:14.707580  9798 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:14.708355  9624 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:14.708804  8061 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:14.704891  9775 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:14.711992  9775 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:14.712360  9775 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.627526  9600 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:14.670439  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 88108.7us
W20260709 13:13:14.671818  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:14.623298  9734 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.619895  9652 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:14.621881  9710 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:14.656806  9678 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:14.629485  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:14.670912  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 45957.4us
I20260709 13:13:14.665060  9645 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:14.729830  8130 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.731132  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:14.651345  9754 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.663259  9792 raft_consensus.cc:697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:14.725545  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.726562  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.653980  9805 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:14.728188  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.723395  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.732527  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:14.736740  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.737398  8209 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.733636  8061 raft_consensus.cc:2393] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 2: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:14.739569  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:14.740028  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:14.654858  9660 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 13:13:14.740688  8727 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 2: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:14.741106  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:14.746044  8693 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:14.741469  8101 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:14.742151  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.747152  8211 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:14.750736  8134 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:14.751945  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.752822  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:14.752949  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.753387  8136 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:14.753666  8729 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:14.754767  8056 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:14.755653  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:14.758430  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:14.758641  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.759336  8204 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.759559  9616 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:14.760118  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.760870  8204 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.761611  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.750183  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.763370  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:14.764122  9652 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.764721  8060 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:14.753530  8206 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.768194  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.769106  8210 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.763067  8212 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.763046  9782 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:14.772919  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.772763  9620 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:14.776599  8211 raft_consensus.cc:2368] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 2. Current term is 3.
I20260709 13:13:14.731760  9645 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:14.776625  8130 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:14.874946  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:14.850397  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:14.898908  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:14.804999  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.913235  9724 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:13:14.913614  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 172208us
I20260709 13:13:14.914019  9724 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:14.914377  9724 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:14.762279  8204 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.857818  8696 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:14.915263  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.828372  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.854637  8208 raft_consensus.cc:2393] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.876006  8137 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:14.821447  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:14.742095  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:14.745108  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:14.656388  9745 consensus_queue.cc:1175] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "965e59f272f143449b18f65db4e61d98" responder_term: 3 status { last_received { term: 2 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:13:14.742255  8063 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:14.964378  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:14.949560  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 50764.7us
I20260709 13:13:14.947156  9724 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.966570  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.966792  9724 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:14.967219  8061 raft_consensus.cc:2368] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
I20260709 13:13:14.967770  8104 leader_election.cc:400] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 2. Current term is 3.
I20260709 13:13:14.968196  8104 leader_election.cc:403] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:13:14.949841  9655 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:14.968817  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.969302  9655 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:14.969549  8061 raft_consensus.cc:2376] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3. Re-sending same reply.
I20260709 13:13:14.969869  9655 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:14.960738  9785 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.972318  9785 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:14.953398  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:14.856015  8055 consensus_meta.cc:287] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.000s
I20260709 13:13:14.950248  9775 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.973611  8055 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:14.974978  9775 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:14.975061  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.854981  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.976068  8212 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.976903  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.977469  8212 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:14.868039  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:14.854692  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:14.978662  8136 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:14.979000  8129 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:14.979825  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:14.980541  8136 raft_consensus.cc:2393] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:14.981248  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:14.981886  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.819903  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:14.983058  8211 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.983117  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:14.911717  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.984012  8137 raft_consensus.cc:2393] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:14.984048  8206 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.984293  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.990540  8211 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.989357  8132 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:14.991742  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:14.992431  8210 raft_consensus.cc:2393] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:14.924744  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:14.985215  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:14.990888  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 133151us
I20260709 13:13:14.984879  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:14.885305  9587 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 13:13:14.876541  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.009925  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.010102  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.010614  8130 raft_consensus.cc:2368] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
I20260709 13:13:14.815197  9792 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:14.916857  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 170863us
I20260709 13:13:15.013540  8207 raft_consensus.cc:2393] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.985050  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:15.014195  8206 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:14.947189  8101 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:14.992363  8061 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:14.778378  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.774035  9814 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.016654  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.016858  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.017341  8132 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:15.017432  8058 raft_consensus.cc:2368] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4.
I20260709 13:13:15.014668  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:15.016069  8209 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:15.018302  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:15.018922  8730 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:14.994660  8137 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:14.987869  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:14.993400  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:15.020584  8210 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:15.004936  8696 leader_election.cc:400] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3.
I20260709 13:13:14.982352  8136 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:15.035863  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.036196  8696 leader_election.cc:403] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:13:14.948405  9734 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:14.947570  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:14.817706  9747 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.022853  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:14.802157  8056 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:14.861608  9771 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:14.856698  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:15.025234  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.026247  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:15.026170  8693 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:14.874176  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:14.854182  9782 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:15.013226  9479 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:13:15.009614  9620 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:15.030493  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.009315  9561 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:13:15.031075  9624 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:15.009138  9805 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:15.083642  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:15.084546  8208 raft_consensus.cc:2393] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:15.053784  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.079722  9706 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:15.042127  9794 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:15.094939  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.034412  9775 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:15.034947  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:14.947188  9798 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:14.816429  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:15.035394  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.112841  8061 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:14.820690  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:15.025309  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:14.995585  8133 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:15.039685  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:15.021168  8212 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:15.042822  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.119493  8137 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:14.911621  9645 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:15.044514  8130 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:13:15.027508  8175 connection.cc:256] RPC call timeout handler was delayed by -2.2611328559999038s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
I20260709 13:13:15.043720  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:15.120379  8175 connection.cc:256] RPC call timeout handler was delayed by -2.255732855999895s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
I20260709 13:13:15.120550  8136 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:15.120733  8175 connection.cc:256] RPC call timeout handler was delayed by -2.2357284689999233s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3002s for a response
W20260709 13:13:15.121081  8175 connection.cc:256] RPC call timeout handler was delayed by -2.1549328559999594s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
I20260709 13:13:15.079949  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:15.121683  8206 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:15.048928  8058 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:15.080605  8060 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
W20260709 13:13:15.122731  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:15.033694  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 86568.6us
I20260709 13:13:15.126979  8101 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.127615  8730 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:15.053280  8138 raft_consensus.cc:2393] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:15.038093  9814 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:14.815594  9816 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:15.095239  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 69161.4us
I20260709 13:13:15.096449  8062 raft_consensus.cc:2368] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
I20260709 13:13:14.814749  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:15.049106  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.118826  8212 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:15.047744  8209 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:15.133924  8210 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:15.136129  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.149715  8058 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:15.137357  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.084028  9620 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:13:15.145195  8026 leader_election.cc:336] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.687s (SENT)
I20260709 13:13:15.155321  8206 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:15.148591  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:15.145984  8102 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:15.047307  9747 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.156904  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.157117  8102 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.155731  8026 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.157939  8210 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:15.160229  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.163040  8060 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:15.094944  9624 raft_consensus.cc:697] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:15.168390  8061 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:15.170156  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.134151  8203 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:15.170883  8132 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:15.180119  8209 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:15.180905  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.181795  9814 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:15.170090  9780 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:15.138427  9616 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.184008  9587 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:15.183748  9624 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.154392  8134 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:15.171993  9561 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:15.193663  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.218782  8135 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:15.183566  8026 leader_election.cc:336] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.693s (SENT)
W20260709 13:13:15.228775  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45235.5us
W20260709 13:13:15.229509  8026 leader_election.cc:336] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.696s (SENT)
I20260709 13:13:15.229938  8026 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.216434  8063 raft_consensus.cc:2368] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4.
I20260709 13:13:15.204826  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.203280  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:15.238509  8132 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:15.201634  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.154668  9609 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
W20260709 13:13:15.251049  9831 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:13:15.251408  8204 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:15.252146  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 939) took 1034 ms. Trace:
I20260709 13:13:15.172818  9785 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:15.252740  9785 raft_consensus.cc:697] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:15.253150  8204 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:15.253868  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:15.253721  9785 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.254521  8722 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:15.161765  9775 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:15.256851  9775 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:15.257265  9775 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:15.258059  9660 raft_consensus.cc:3055] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:15.258553  9660 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:15.259526  9660 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.260664  9660 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:15.114251  9798 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.226799  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.261408  8062 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:15.262269  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.262662  9798 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:15.263302  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.197943  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:15.235098  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:15.269496  8026 leader_election.cc:336] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.715s (SENT)
I20260709 13:13:15.239814  8137 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:15.181375  9620 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.212582  8205 raft_consensus.cc:2393] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:15.241564  9745 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:15.243674  8029 leader_election.cc:400] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3.
I20260709 13:13:15.282768  8029 leader_election.cc:403] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:13:15.284879  8029 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.200059  8206 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:15.287060  9587 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:15.247695  9747 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:15.247968  9754 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:15.268842  8062 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:15.192585  8136 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
W20260709 13:13:15.298650  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.272882  8061 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:15.264000  8027 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:15.252517  8057 rpcz_store.cc:276] 0709 13:13:14.217434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.614951 (+397517us) service_pool.cc:224] Handling call
0709 13:13:14.615213 (+   262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:14.615533 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:14.615564 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:14.615964 (+   400us) log.cc:844] Serialized 26 byte log entry
0709 13:13:14.728031 (+112067us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:14.731795 (+  3764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:14.731880 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:15.251943 (+520063us) raft_consensus.cc:1685] finished
0709 13:13:15.251963 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.252073 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4812,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":111293,"threads_started":1,"wal-append.queue_time_us":479261}
I20260709 13:13:15.281852  8722 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:15.299882  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.214993  9616 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.263547  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.280697  8056 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:15.286059  8029 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:15.316146  8029 leader_election.cc:400] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 pre-election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3.
I20260709 13:13:15.264304  9561 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:15.300958  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.304347  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.306177  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.358954  8130 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:15.359061  8137 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:15.317186  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.270975  8058 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:15.360527  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.361392  8058 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 2: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:15.362015  8212 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:15.278383  8131 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:15.345504  8132 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
W20260709 13:13:15.365468  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 48380us
I20260709 13:13:15.374676  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:15.366654  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.376744  8057 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:15.371934  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.300611  9754 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:15.305884  9745 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:15.305536  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:15.383167  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.379565  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.384145  8058 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:15.316236  9479 raft_consensus.cc:670] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:15.386806  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.388063  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.389813  9616 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:15.306200  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.381784  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.398222  8061 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:15.398958  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.399644  8061 raft_consensus.cc:2410] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:15.400290  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.400741  8061 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:15.345664  9747 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:15.379372  9731 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:15.401278  9747 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.401751  9731 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:15.401787  9747 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:15.402232  9731 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.402647  9731 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:15.394555  8055 raft_consensus.cc:2410] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:15.403909  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.403820  9655 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:15.404534  8055 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:15.348323  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.407366  8138 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:15.407428  8136 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:15.407976  9754 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.409473  9754 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:15.294602  9798 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.068s
I20260709 13:13:15.405375  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.420535  8135 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:15.349325  8054 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:15.427830  8211 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:15.362859  9651 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:15.429333  8135 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:15.441515  8062 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:15.430307  8058 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:15.431452  9745 consensus_queue.cc:1059] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:15.373091  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.476508  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.477174  8062 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:15.477974  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.478629  8062 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:15.491859  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:15.492194  8135 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:15.492588  8212 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:13:15.506836  8061 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:15.508019  8061 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:15.422395  9655 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:15.442152  9587 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3.
I20260709 13:13:15.455233  8055 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:15.501363  8129 consensus_meta.cc:287] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.001s
I20260709 13:13:15.442796  9651 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:15.453540  9706 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:15.529260  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.532104  8136 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:15.494792  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.525681  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.495375  8134 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:15.534433  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.539429  8136 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:15.500856  9794 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3.
I20260709 13:13:15.540558  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.549691  8062 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:15.531117  8129 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:15.536208  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.554215  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.532347  9651 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:15.514391  9745 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:13:15.558233  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.560220  8136 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:15.540522  9587 raft_consensus.cc:3055] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:15.566547  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:15.553968  9600 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.006s	sys 0.017s
I20260709 13:13:15.567565  9787 raft_consensus.cc:2764] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:13:15.581175  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.581476  8059 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
W20260709 13:13:15.582072  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:15.586921  9706 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:15.590066  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:15.590956  9616 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:15.590833  9734 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:15.592875  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:15.593614  9734 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:15.593986  9734 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:15.594393  9734 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:15.567782  9794 consensus_queue.cc:1059] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:13:15.595448  9794 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:15.595803  9794 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:15.596858  9794 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.597720  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.598045  9794 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:15.598898  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.599912  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.600576  8132 raft_consensus.cc:2393] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:15.601251  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.589995  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.602007  8130 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:15.602751  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.603881  8131 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:13:15.605276  8131 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:15.613399  9734 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.613569  9747 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.602208  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.631644  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.653597  8136 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:15.622901  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.655901  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.652551  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.652484  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.657650  8059 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:15.657841  8058 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:15.659853  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.565800  9587 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:15.660547  8136 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:15.661341  8212 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:15.672731  8102 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:15.667969  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 513710us
I20260709 13:13:15.675521  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:15.569145  8054 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:15.573969  9814 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.606108  8138 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:15.615614  8131 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:15.660099  8062 raft_consensus.cc:2955] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:15.699818  8057 raft_consensus.cc:2368] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5.
I20260709 13:13:15.700672  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.545580  9561 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.710918  8059 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:15.671675  9792 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:15.688179  9787 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:13:15.720165  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:15.715725  8175 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.541s (SENT) [suppressed 34 similar messages]
W20260709 13:13:15.720788  8210 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.013s	sys 0.013s
W20260709 13:13:15.727783  8210 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.013s	sys 0.013s
W20260709 13:13:15.728942  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 479100us
I20260709 13:13:15.703223  9655 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:15.703353  9600 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:15.738497  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:15.746601  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 372009us
I20260709 13:13:15.737608  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:15.750195  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.750403  8206 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:15.750852  8059 raft_consensus.cc:2368] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 2. Current term is 3.
I20260709 13:13:15.751881  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.752944  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.752605  8696 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:15.756734  8206 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:15.721907  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.726971  9787 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:15.730844  9792 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:13:15.768951  8175 connection.cc:256] RPC call timeout handler was delayed by -2.092832855999859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2073s for a response
W20260709 13:13:15.771477  8175 connection.cc:256] RPC call timeout handler was delayed by -2.0298328559999845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
W20260709 13:13:15.771799  8175 connection.cc:256] RPC call timeout handler was delayed by -1.9740328559998943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22050000000000003s for a response
W20260709 13:13:15.755697  9805 consensus_meta.cc:287] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.664s	user 0.007s	sys 0.003s
I20260709 13:13:15.754524  9706 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.754423  9655 raft_consensus.cc:697] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:15.757824  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:15.705142  9814 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:15.774610  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 363048us
I20260709 13:13:15.776010  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.745802  9731 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:15.812973  9731 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:15.813207  8726 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:15.813923  8061 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:15.814080  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.763484  9616 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:13:15.807221  9706 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.815053  8131 raft_consensus.cc:2393] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:15.816009  9849 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:15.816074  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.817037  9734 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:15.817586  9706 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:15.809748  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:15.818104  9792 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.820168  9706 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:15.820744  9706 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:15.821293  8134 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:15.822556  9780 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.822786  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2091) took 1084 ms. Trace:
I20260709 13:13:15.822991  9780 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:15.823153  8208 rpcz_store.cc:276] 0709 13:13:14.739217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.109432 (+370215us) service_pool.cc:224] Handling call
0709 13:13:15.171016 (+ 61584us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:15.171239 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:15.171265 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:15.171700 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:13:15.474920 (+303220us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:15.474970 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.475058 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:15.822569 (+347511us) raft_consensus.cc:1685] finished
0709 13:13:15.822593 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.822719 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":271117,"threads_started":1,"wal-append.queue_time_us":551470}
I20260709 13:13:15.773401  9479 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:13:15.821774  9706 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:15.769963  9645 consensus_meta.cc:287] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.598s	user 0.008s	sys 0.001s
I20260709 13:13:15.824707  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:15.831195  9747 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:15.831709  9645 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.832257  9734 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:15.833168  8730 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
W20260709 13:13:15.833240  9775 consensus_meta.cc:287] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.000s
I20260709 13:13:15.834132  9775 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:15.836573  9775 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:15.837340  8137 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:15.838469  9849 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:15.839913  9651 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:15.840888  8059 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 2: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:15.841989  9651 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:15.842365  8129 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:15.795233  9805 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 2. Current term is 3.
I20260709 13:13:15.766057  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.809338  9655 consensus_queue.cc:237] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:15.846132  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:15.854169  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 178742us
I20260709 13:13:15.768987  9787 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:15.862296  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1105) took 1087 ms. Trace:
I20260709 13:13:15.866685  8207 rpcz_store.cc:276] 0709 13:13:14.774731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.115041 (+340310us) service_pool.cc:224] Handling call
0709 13:13:15.115410 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.136399 (+ 20989us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:15.544882 (+408483us) log.cc:844] Serialized 14 byte log entry
0709 13:13:15.861149 (+316267us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:15.861367 (+   218us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:15.861500 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.861738 (+   238us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.862132 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":397411,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":9355,"thread_start_us":560826,"threads_started":2}
I20260709 13:13:15.869148  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.848383  9645 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:15.796675  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.849526  8101 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:15.860843  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.876703  8101 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.854451  9624 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:15.863240  8026 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:15.822223  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.865130  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2094) took 1013 ms. Trace:
I20260709 13:13:15.768767  9587 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:15.881385  8210 rpcz_store.cc:276] 0709 13:13:14.851603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.180778 (+329175us) service_pool.cc:224] Handling call
0709 13:13:15.181054 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.181228 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:15.728568 (+547340us) log.cc:844] Serialized 14 byte log entry
0709 13:13:15.864542 (+135974us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:15.864623 (+    81us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:15.864695 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.864820 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:15.865045 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3543,"prepare.queue_time_us":352756,"prepare.run_cpu_time_us":21,"prepare.run_wall_time_us":22,"raft.queue_time_us":3001,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":6343,"thread_start_us":678687,"threads_started":2}
I20260709 13:13:15.855175  8056 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:15.884745  8133 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:15.887631  8058 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:15.879161  8062 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:15.852937  8131 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
W20260709 13:13:15.891208  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 138347us
I20260709 13:13:15.891225  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:15.776575  8175 connection.cc:256] RPC call timeout handler was delayed by -1.8930328559999907s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2295s for a response
W20260709 13:13:15.904395  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7105437389998315s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15780000000000002s for a response
W20260709 13:13:15.904681  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6556437389997427s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
W20260709 13:13:15.904943  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6295437389999279s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1668s for a response
W20260709 13:13:15.905197  8175 connection.cc:256] RPC call timeout handler was delayed by -1.5334364350001124s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
W20260709 13:13:15.905459  8175 connection.cc:256] RPC call timeout handler was delayed by -1.5073364350000702s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 13:13:15.905709  8175 connection.cc:256] RPC call timeout handler was delayed by -1.4641364350000003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
W20260709 13:13:15.905959  8175 connection.cc:256] RPC call timeout handler was delayed by -1.4002364350001244s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1665s for a response
W20260709 13:13:15.906206  8175 connection.cc:256] RPC call timeout handler was delayed by -1.3705364350000764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1698s for a response
W20260709 13:13:15.906486  8175 connection.cc:256] RPC call timeout handler was delayed by -1.2504328559998612s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3009s for a response
W20260709 13:13:15.906734  8175 connection.cc:256] RPC call timeout handler was delayed by -1.1129437389997747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2242s for a response
I20260709 13:13:15.912817  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.913574  8059 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:15.885373  8138 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:15.875701  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:15.903434  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.920178  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.880525  9624 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.931870  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.932580  8059 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:15.933413  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.885561  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:15.935560  9858 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:15.936470  9858 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:15.937228  9858 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:15.931550  8134 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:15.918671  9731 consensus_queue.cc:1059] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:13:15.938994  9731 raft_consensus.cc:3055] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:15.939215  8063 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:15.939343  9731 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:15.940423  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.940284  9731 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:15.941366  9731 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:15.941576  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.941382  8054 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:15.943507  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.909976  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:15.937794  9858 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:15.945909  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.951534  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.952512  8029 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:15.934234  8059 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:15.934950  9624 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:15.947831  8054 raft_consensus.cc:2393] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:15.963505  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 86858.5us
I20260709 13:13:15.965858  8101 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.967306  8101 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.945118  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.963495  8055 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:15.950875  8725 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:15.960498  9651 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:15.960827  9747 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:15.976377  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:15.976923  9771 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:15.977205  9858 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:15.978405  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.978852  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:15.978981  9849 raft_consensus.cc:2764] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:13:15.975606  9850 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:13:15.984393  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:15.992545  8729 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:15.984465  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:15.993536  8060 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:15.984488  9868 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:15.988700  8101 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.972859  9805 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:15.997771  8101 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:15.999341  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:15.998974  9857 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:16.001950  8055 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:16.001766  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.003417  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:15.989866  9862 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:16.004415  8129 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:16.004459  9862 raft_consensus.cc:697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:16.000502  9858 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:16.005725  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:15.992429  9776 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:16.015008  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.006250  9858 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:16.033721  9731 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:16.033542  8026 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.011433  9652 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:15.996825  9651 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:13:16.014171  8054 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:16.013339  9776 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:16.046597  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:16.000432  9852 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:13:16.035185  9858 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:16.037699  8129 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:16.052546  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 841) took 1023 ms. Trace:
I20260709 13:13:16.054507  8135 rpcz_store.cc:276] 0709 13:13:15.028872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.492964 (+464092us) service_pool.cc:224] Handling call
0709 13:13:15.493304 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:15.762952 (+269648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:15.762979 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:15.770096 (+  7117us) log.cc:844] Serialized 26 byte log entry
0709 13:13:15.954769 (+184673us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:15.954967 (+   198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.955187 (+   220us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:16.052387 (+ 97200us) raft_consensus.cc:1685] finished
0709 13:13:16.052401 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:16.052501 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":142,"raft.queue_time_us":32211,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":6865,"thread_start_us":392236,"threads_started":2,"wal-append.queue_time_us":226722}
I20260709 13:13:16.005721  9862 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:16.048019  9776 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:16.057032  8175 leader_election.cc:400] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
I20260709 13:13:16.062794  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.072314  8175 leader_election.cc:403] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:13:16.073104  8135 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:16.079371  9652 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
W20260709 13:13:16.079778  9856 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:13:16.073107  9731 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
W20260709 13:13:16.063913  9860 log.cc:927] Time spent T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:13:16.054200  9771 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:16.084780  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:16.086381  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.088364  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.088639  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:16.089214  8135 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:16.058531  9747 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:13:16.091262  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.071231  9870 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:16.096302  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 98601.1us
I20260709 13:13:16.091701  9652 raft_consensus.cc:2764] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:13:16.100368  9848 log.cc:927] Time spent T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:13:16.100464  8101 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.101200  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 327) took 1181 ms. Trace:
I20260709 13:13:16.101655  8101 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.101588  8726 rpcz_store.cc:276] 0709 13:13:14.919882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.947295 (+ 27413us) service_pool.cc:224] Handling call
0709 13:13:16.101050 (+1153755us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b540052c454. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:16.101149 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1162034560}
I20260709 13:13:16.101440  9868 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:16.103813  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:16.097947  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.089738  9838 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:16.103600  9805 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:16.130513  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.131109  8136 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:16.131666  8725 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:16.098955  9747 consensus_queue.cc:1243] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:13:16.112218  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 150457us
I20260709 13:13:16.104040  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.103266  8101 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.127357  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 836) took 1099 ms. Trace:
I20260709 13:13:16.127466  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.135986  8133 rpcz_store.cc:276] 0709 13:13:15.027485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.137121 (+109636us) service_pool.cc:224] Handling call
0709 13:13:16.107607 (+970486us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b5400409cd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:16.127314 (+ 19707us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1633602944}
I20260709 13:13:16.137079  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:16.137085  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.108942  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.137827  8133 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:16.136000  8029 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:16.138437  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.143466  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.143993  8133 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:16.144661  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.145622  8101 leader_election.cc:400] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 2. Current term is 3.
I20260709 13:13:16.146531  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.147006  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:16.147871  8133 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:16.148751  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.151822  8129 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:16.152298  9865 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:16.154304  9852 raft_consensus.cc:3055] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:16.154691  9852 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:16.143080  9868 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:16.155476  9852 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:16.156424  9852 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:16.157263  9787 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.158491  8722 raft_consensus.cc:2393] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:16.142871  9805 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:16.159283  8027 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:16.159543  9787 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:16.160470  8133 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:16.160806  8135 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:16.151310  8101 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:16.161870  8101 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.162559  8136 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:16.145900  9858 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:16.167059  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:16.163767  9862 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.164338  8060 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:16.175185  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:16.181057  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.166481  9785 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:16.169714  9858 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:16.169965  9731 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:16.166240  9805 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:16.171569  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.172947  9754 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:16.149150  9853 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:13:16.204977  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 43664.4us
I20260709 13:13:16.252909  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:16.253098  8696 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:16.215021  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 48028.8us
I20260709 13:13:16.164875  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:16.256034  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 91203.4us
I20260709 13:13:16.194128  8136 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:16.254491  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 102969us
I20260709 13:13:16.261567  9678 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:16.263052  8134 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:16.215292  9792 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:16.264384  9792 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:16.265328  9792 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:16.271334  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:16.270803  8136 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:16.207212  8721 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:16.217695  9587 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:16.194511  9652 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:16.257964  9731 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:16.164220  9620 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:13:16.240710  9780 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:16.287387  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:16.288029  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:16.260691  9785 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:16.217836  9624 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.292636  8138 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:16.267450  9754 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:16.280048  9805 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:16.255224  9771 consensus_queue.cc:1059] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:13:16.295667  8133 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:16.254509  9787 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:16.194366  9862 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.310178  9645 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:16.311772  9862 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:16.311259  9805 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.312603  8061 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:13:16.312798  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:16.313328  9787 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.316284  9787 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:16.322522  8026 leader_election.cc:336] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.555s (SENT)
I20260709 13:13:16.325279  8026 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.327219  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.341420  8133 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:16.286787  9845 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:16.331862  9561 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.003s	sys 0.013s
W20260709 13:13:16.338666  9875 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:13:16.300585  9858 consensus_queue.cc:1059] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:13:16.367133  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:16.363155  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:16.383400  8059 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:16.384285  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:16.325662  9794 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:13:16.376315  8212 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:48498
W20260709 13:13:16.361805  9561 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.004s	sys 0.013s
I20260709 13:13:16.399398  8212 raft_consensus.cc:606] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:16.331954  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.400230  8212 raft_consensus.cc:614] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:16.401896  8730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 329) took 1263 ms. Trace:
I20260709 13:13:16.402405  8175 leader_election.cc:400] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
I20260709 13:13:16.349773  9849 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:16.404786  8175 leader_election.cc:403] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:13:16.404742  8730 rpcz_store.cc:276] 0709 13:13:15.138067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.147011 (+  8944us) service_pool.cc:224] Handling call
0709 13:13:15.193684 (+ 46673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.833717 (+640033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.401674 (+567957us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b540060ead4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:16.401844 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1470228224}
I20260709 13:13:16.374065  9814 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:16.407095  9731 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:16.407833  9855 log.cc:927] Time spent T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.594s	user 0.004s	sys 0.000s
I20260709 13:13:16.409016  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:16.349752  9655 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:13:16.299584  9776 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.416271  9776 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:16.417340  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 843) took 1388 ms. Trace:
I20260709 13:13:16.418938  9754 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.368758  8134 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:16.417685  8130 rpcz_store.cc:276] 0709 13:13:15.029144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.603528 (+574384us) service_pool.cc:224] Handling call
0709 13:13:15.603765 (+   237us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:15.604072 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:15.604101 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:15.604477 (+   376us) log.cc:844] Serialized 26 byte log entry
0709 13:13:15.754019 (+149542us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:15.754071 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:15.754161 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:16.369406 (+615245us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:16.417083 (+ 47677us) raft_consensus.cc:1685] finished
0709 13:13:16.417103 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:16.417290 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":130,"spinlock_wait_cycles":41076608,"thread_start_us":149199,"threads_started":1,"wal-append.queue_time_us":209311}
I20260709 13:13:16.424247  8059 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:16.425434  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:16.425971  8206 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:16.426990  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:16.432063  8059 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:16.433117  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:16.433786  8059 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
W20260709 13:13:16.434712  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 295679us
I20260709 13:13:16.435464  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.406438  9889 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:16.437137  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.437336  8029 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:16.438125  8130 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:16.441648  8059 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:16.423583  8129 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
W20260709 13:13:16.459595  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 120012us
I20260709 13:13:16.459839  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:16.460294  8026 leader_election.cc:336] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.640s (SENT)
I20260709 13:13:16.460414  8206 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:16.460649  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:16.461625  8212 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:16.445181  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.441668  9754 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:13:16.446820  9885 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.115s	user 0.002s	sys 0.000s
I20260709 13:13:16.463392  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.447808  9624 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:16.437420  9889 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:16.464818  9889 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:16.465700  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:16.466219  8212 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:16.472033  8057 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:16.431519  9898 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:13:16.475100  9891 log.cc:927] Time spent T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:13:16.475462  9880 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.451733  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 823) took 1500 ms. Trace:
I20260709 13:13:16.475948  9880 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:16.476298  8137 rpcz_store.cc:276] 0709 13:13:14.953419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.427538 (+474119us) service_pool.cc:224] Handling call
0709 13:13:15.474450 (+ 46912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:15.837775 (+363325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.451543 (+613768us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b540036ea54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:16.451682 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":834079104}
I20260709 13:13:16.477587  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.478001  9880 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:16.479241  9893 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.479725  9893 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.481761  9893 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:16.438108  9645 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:16.483024  9645 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.484776  8721 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:16.484897  8725 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:16.485934  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:16.486577  8055 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:16.487200  9887 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:16.488547  9887 raft_consensus.cc:3055] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:16.488938  9887 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:16.456171  8130 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:16.456689  9745 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:16.429423  9771 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
I20260709 13:13:16.460677  8026 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.443585  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.440905  9706 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:13:16.492417  9849 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:16.493201  8208 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:16.494095  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:16.494248  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:16.494593  8211 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:16.494680  8207 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:16.495599  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:16.464444  8134 raft_consensus.cc:2393] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:16.492034  8026 leader_election.cc:336] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.671s (SENT)
I20260709 13:13:16.490607  9754 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:16.500516  8206 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:16.503011  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.504351  8026 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.505586  8696 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:16.506809  8137 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:16.508399  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:16.511761  8026 leader_election.cc:336] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.677s (SENT)
I20260709 13:13:16.443022  8059 raft_consensus.cc:3055] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:16.512274  8026 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.512856  8059 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:16.514112  8026 leader_election.cc:336] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.691s (SENT)
W20260709 13:13:16.515991  8026 leader_election.cc:336] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.897s (SENT)
I20260709 13:13:16.516479  8026 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.516079  8212 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:16.517974  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:16.471541  9889 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:16.513980  8059 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:16.515558  9587 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:16.525288  8057 raft_consensus.cc:2393] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:16.530144  9880 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:16.530210  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:16.506734  9771 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.531065  8057 raft_consensus.cc:2376] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4. Re-sending same reply.
W20260709 13:13:16.531307  9883 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.200s	user 0.002s	sys 0.000s
I20260709 13:13:16.531167  9771 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.531888  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:16.532505  8057 raft_consensus.cc:2376] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3. Re-sending same reply.
I20260709 13:13:16.525416  9724 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:16.533434  9771 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:16.535566  9902 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:13:16.525331  8205 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:16.539731  9889 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:16.540113  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:16.524962  8029 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:16.534124  9853 raft_consensus.cc:3055] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:13:16.528544  8026 leader_election.cc:336] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 2.224s (SENT)
I20260709 13:13:16.540982  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.543660  8133 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 2: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:16.489815  9887 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.533452  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:16.540623  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 138360us
I20260709 13:13:16.507241  9814 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.545418  8057 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:16.545318  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.545663  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:16.547535  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:16.547922  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:16.548655  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.511166  9838 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:16.550961  9898 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:16.551074  9900 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:16.551043  9814 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:16.551822  8134 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:16.553215  9814 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:16.553236  9706 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:16.553743  8055 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:16.554030  8059 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:16.551157  9838 raft_consensus.cc:697] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:16.557749  9754 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:16.558924  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:16.559590  8058 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:16.558136  9838 consensus_queue.cc:237] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.555668  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:16.555083  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:16.564638  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.527298  9849 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:16.575006  8059 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:13:16.576202  8059 pending_rounds.cc:85] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 1
I20260709 13:13:16.567893  8026 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.572824  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:16.579717  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:16.580421  8026 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.515616  9655 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:13:16.546656  9724 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:16.542920  9853 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:16.544348  9587 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:16.594238  8057 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:16.581521  8059 pending_rounds.cc:107] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:13:16.587347  8208 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:16.589107  9880 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:16.589066  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:16.589699  9905 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:16.588714  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:16.595739  8203 raft_consensus.cc:2410] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:16.596110  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:16.596511  9561 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:16.566881  9900 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:16.596798  9905 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:16.576507  9898 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:16.605783  9900 consensus_queue.cc:1059] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:16.607728  9678 raft_consensus.cc:3055] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:16.608147  9678 raft_consensus.cc:740] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:16.609020  9678 consensus_queue.cc:260] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.610054  9678 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:16.577023  9731 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:16.610663  9587 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:16.612790  9587 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:16.614460  8726 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:16.614578  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:16.591123  9724 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.588733  9889 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:16.602952  8059 raft_consensus.cc:2889] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:16.583554  9852 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:16.544902  9857 consensus_queue.cc:1059] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:13:16.593619  9887 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:16.588032  9771 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:16.618520  8026 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.620397  8055 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:16.629683  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:16.582597  8063 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:16.630316  8059 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:13:16.638679  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.625751  9724 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:16.617491  9645 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.640588  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.649187  8133 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:16.635226  9731 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:16.644949  8135 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:16.645396  8212 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:16.593021  9888 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
W20260709 13:13:16.635651  9870 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:13:16.655105  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.634220  9889 raft_consensus.cc:697] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:16.660634  9771 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:16.661759  9771 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:16.642217  9805 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:16.673913  8212 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:16.679749  8206 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:16.693567  8135 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:16.687103  9889 consensus_queue.cc:237] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:16.669888  8054 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:13:16.705621  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 108780us
I20260709 13:13:16.710542  8133 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:13:16.714668  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:16.759048  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:16.654994  9645 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:16.762643  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50060.4us
I20260709 13:13:16.689520  9561 raft_consensus.cc:697] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:16.733017  9905 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:13:16.767858  9906 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:13:16.768146  9905 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:16.767894  9561 consensus_queue.cc:237] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.647385  9853 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:16.774273  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 59674.4us
I20260709 13:13:16.770593  9792 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:13:16.805732  9745 raft_consensus.cc:3055] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:16.806648  9912 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:16.826876  8206 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:16.808429  9724 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:16.809325  9916 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:16.828034  8130 raft_consensus.cc:2368] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 3. Current term is 4.
I20260709 13:13:16.830629  9724 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:16.785636  9706 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:13:16.830832  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:16.833914  9912 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:16.835000  9587 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:16.835716  9912 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:16.824272  9745 raft_consensus.cc:740] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:16.842826  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 65530.6us
I20260709 13:13:16.838320  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.826262  9858 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.832059  9655 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.832930  9917 consensus_queue.cc:1059] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:13:16.850792  9916 consensus_queue.cc:1059] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:13:16.855106  8131 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:16.855034  8056 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 980) took 1612 ms (client timeout 2111 ms). Trace:
I20260709 13:13:16.857609  8207 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:16.858667  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:16.859427  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 1986) took 1052 ms. Trace:
I20260709 13:13:16.859858  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.831866  9888 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
W20260709 13:13:16.862246  8175 leader_election.cc:336] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.755s (SENT)
I20260709 13:13:16.864123  8135 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:13:16.862388  8059 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:16.859769  8060 rpcz_store.cc:276] 0709 13:13:15.806612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.993970 (+187358us) service_pool.cc:224] Handling call
0709 13:13:16.010143 (+ 16173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.174678 (+164535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:16.859205 (+684527us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b540027ae54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:16.859358 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354187648}
I20260709 13:13:16.866758  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.867933  8101 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.869014  8101 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.870931  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:16.871330  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:16.933277  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 64330.1us
I20260709 13:13:16.934033  8101 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:16.936894  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:16.937479  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 75250.9us
I20260709 13:13:16.867743  9857 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:16.852164  9655 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.934147  9798 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:16.958786  9798 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:13:16.958993  9927 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:13:16.959491  9890 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:13:16.932596  9561 raft_consensus.cc:3055] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:16.961419  9561 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:16.943112  9609 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:16.951229  8060 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:13:16.940026  9913 raft_consensus.cc:3055] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:16.852270  9858 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.938184  8062 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:16.967016  8060 raft_consensus.cc:606] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:16.970914  8060 raft_consensus.cc:614] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:16.934450  9814 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:13:16.856527  8056 rpcz_store.cc:269] 0709 13:13:15.242206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.387812 (+145606us) service_pool.cc:224] Handling call
0709 13:13:16.854878 (+1467066us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b5400335754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:16.854981 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338112896}
I20260709 13:13:16.947213  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:16.973174  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:16.952386  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:16.962594  9561 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.977041  8175 leader_election.cc:400] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4.
I20260709 13:13:16.982897  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:16.985107  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:16.969220  9913 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:16.941795  9745 consensus_queue.cc:260] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:16.994477  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:16.995862  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:16.999994  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.001468  8175 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:16.999377  9917 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:16.978426  9678 raft_consensus.cc:2764] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:13:17.007261  8175 leader_election.cc:400] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
I20260709 13:13:17.009913  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.011080  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.012146  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.000631  9919 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:17.005897  9561 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:17.014371  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.005120  9888 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:17.005260  9899 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:17.018632  9914 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:17.019940  9926 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:17.027315  9914 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:17.027768  9914 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:17.014963  9916 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:13:17.024597  9915 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:13:17.019639  9912 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:17.021340  9899 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.038713  9899 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:17.039507  9914 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:17.040663  9926 raft_consensus.cc:697] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:17.045205  9926 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:17.047776  8057 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:13:17.041886  9917 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:17.048130  8057 pending_rounds.cc:85] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 1
W20260709 13:13:17.043045  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.048448  8057 pending_rounds.cc:107] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:13:17.048787  8057 raft_consensus.cc:2889] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:17.043774  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.068104  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.009912  9838 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:13:17.077147  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:17.077359  8175 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:17.026293  8204 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:17.080107  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.078902  9917 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:17.068351  8726 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:17.085950  9917 raft_consensus.cc:697] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:17.075273  9678 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:17.089206  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.098647  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:17.099773  8207 raft_consensus.cc:2410] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:13:17.086839  9917 consensus_queue.cc:237] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:17.103812  8063 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b4358ace86545339036f21610735b47"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:13:17.104174  9912 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:17.088620  9928 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:17.104604  8063 raft_consensus.cc:606] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:17.104979  9928 raft_consensus.cc:697] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:17.079557  9919 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:13:17.106019  9928 consensus_queue.cc:237] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:17.081059  9905 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:17.108472  9905 raft_consensus.cc:697] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:17.089527  9858 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:17.103843  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:17.109978  8726 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 3 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:17.109347  9905 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:17.111110  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 186) took 1020 ms. Trace:
I20260709 13:13:17.103998  9678 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:17.111666  9655 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:17.103740  9899 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:17.112419  9678 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:17.080358  8204 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:17.113870  8204 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:17.114426  9899 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:17.109305  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:17.114981  8204 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:17.115156  8054 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:17.105057  8063 raft_consensus.cc:614] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:17.118367  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:17.118629  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:17.121479  8059 raft_consensus.cc:2376] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3. Re-sending same reply.
I20260709 13:13:17.125142  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:17.149646  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:17.150692  8059 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:17.151778  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:17.085944  9898 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:17.157266  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:17.131706  9914 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:17.149645  8721 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:17.164505  8060 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:17.165244  8059 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:17.159279  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:17.111534  8138 rpcz_store.cc:276] 0709 13:13:16.091089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.151811 (+ 60722us) service_pool.cc:224] Handling call
0709 13:13:16.152166 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.300186 (+148020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.110898 (+810712us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54003c8cd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:17.111057 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296564096}
I20260709 13:13:17.170475  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.154912  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:17.180478  9609 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:17.184342  9870 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.184417  9935 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.165635  9652 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:17.165974  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:17.200882  9899 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:17.188344  8135 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:17.217011  8204 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:13:17.226200  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.227012  9899 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:17.198592  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:17.213172  9706 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.218744  9935 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:17.224663  8730 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:17.218935  9870 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:17.224568  9771 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:13:17.227560  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.231455  8131 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:17.233556  8138 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:13:17.217068  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 283105us
I20260709 13:13:17.235494  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:17.228727  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:17.272459  8101 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:17.231336  9645 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:17.236343  9865 raft_consensus.cc:3055] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:17.276438  8101 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:17.291090  9853 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:17.295922  8210 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:17.308308  9780 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:17.309933  8054 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:17.322786  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:17.325516  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 49155.6us
I20260709 13:13:17.329121  8211 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:17.329351  8101 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:17.244268  9938 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:17.326025  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 45925.4us
I20260709 13:13:17.276476  9865 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:17.331394  9865 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:17.332506  9865 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:17.275007  9858 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.334313  8101 leader_election.cc:400] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5.
I20260709 13:13:17.354301  8101 leader_election.cc:403] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:13:17.354079  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 118656us
I20260709 13:13:17.242360  9893 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 13:13:17.242633  8726 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:17.278478  9868 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:17.355721  8101 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:17.361557  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:17.278046  8056 raft_consensus.cc:2376] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3. Re-sending same reply.
I20260709 13:13:17.266937  9706 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:17.325780  9780 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:17.360174  9868 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:17.333519  9814 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:17.327075  9928 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:17.353003  9785 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:13:17.447846  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 92842.2us
I20260709 13:13:17.480358  8210 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:17.481166  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:17.463928  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 102430us
I20260709 13:13:17.361282  9724 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5.
I20260709 13:13:17.477144  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:17.482851  8210 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:17.484026  8206 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:17.485996  8206 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:17.485914  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:17.487830  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.491904  9798 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:17.466933  9933 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:13:17.460501  8204 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:13:17.360834  9858 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:17.474563  9868 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:17.535476  9945 log.cc:927] Time spent T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:13:17.461458  9919 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:13:17.531252  9794 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.538702  9794 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:17.531229  8722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 482) took 1814 ms (client timeout 2034 ms). Trace:
W20260709 13:13:17.540858  8722 rpcz_store.cc:269] 0709 13:13:15.716620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.725719 (+  9099us) service_pool.cc:224] Handling call
0709 13:13:17.527284 (+1801565us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b540060de54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:17.531159 (+  3875us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":814192000}
I20260709 13:13:17.492448  9771 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:13:17.543145  8725 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:17.491389  9678 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:17.492336  9905 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:17.493384  9912 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:13:17.490124  9776 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:17.505334  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.501099  9798 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:17.356221  9938 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:17.469444  9838 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:13:17.497113  9561 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
W20260709 13:13:17.554399  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 49142.7us
I20260709 13:13:17.615926  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.459419  9857 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:17.498055  8723 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:13:17.556546  9897 log.cc:927] Time spent T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:13:17.562785  9947 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 13:13:17.562655  9798 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:17.562525  9771 raft_consensus.cc:2764] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:13:17.451396  9899 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:17.611195  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 142) took 1480 ms. Trace:
I20260709 13:13:17.553697  9905 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.622012  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 180) took 1947 ms. Trace:
I20260709 13:13:17.629727  8175 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:17.614940  9731 raft_consensus.cc:3055] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:17.639058  8132 rpcz_store.cc:276] 0709 13:13:15.674806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:15.700536 (+ 25730us) service_pool.cc:224] Handling call
0709 13:13:15.722417 (+ 21881us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:15.857001 (+134584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:15.857029 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:16.495980 (+638951us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b2c00011f88. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db990fa11 00007f6db991f372 00007f6dbb051a01 00007f6dbb01643a 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2
0709 13:13:16.496339 (+   359us) log.cc:844] Serialized 26 byte log entry
0709 13:13:16.641869 (+145530us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:16.641901 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.621710 (+979809us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b540036e554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:17.621760 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:17.621804 (+    44us) raft_consensus.cc:1685] finished
0709 13:13:17.621824 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:17.621948 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":66100,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":18434,"spinlock_wait_cycles":321885824,"thread_start_us":140057,"threads_started":1,"wal-append.queue_time_us":163212}
I20260709 13:13:17.606004  9913 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:17.634100  9941 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:13:17.638347  8061 rpcz_store.cc:276] 0709 13:13:16.130292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.219642 (+ 89350us) service_pool.cc:224] Handling call
0709 13:13:16.219950 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:16.313074 (+ 93124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.598019 (+1284945us) spinlock_profiling.cc:243] Waited 92.3 ms on lock 0x7b5400151154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:17.611140 (+ 13121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212250880}
I20260709 13:13:17.635063  9776 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:17.638827  8726 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:13:17.640743  9905 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:17.554838  9780 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:17.624636  9724 consensus_queue.cc:1059] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:13:17.641525  9731 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:17.633875  9792 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
W20260709 13:13:17.642452  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 309199us
I20260709 13:13:17.667102  9912 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:17.674566  9870 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:17.679061  9913 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:17.691891  8138 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:17.671263  9890 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:13:17.689798  9780 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:17.684682  9776 raft_consensus.cc:697] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:17.714794  9776 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:17.679863  9886 consensus_queue.cc:1175] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" responder_term: 3 status { last_received { term: 3 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 3 index: 3 } }
W20260709 13:13:17.714788  9956 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 13:13:17.708146  9889 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.006s	sys 0.012s
W20260709 13:13:17.716818  9889 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.006s	sys 0.013s
I20260709 13:13:17.693393  9935 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:17.705778  9706 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:17.727438  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 89713.9us
I20260709 13:13:17.728580  9785 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:17.740185  8696 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.705801  9914 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
W20260709 13:13:17.739650  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 113809us
W20260709 13:13:17.688400  9946 consensus_queue.cc:1175] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d9cd2999aa314b39997b423ec1ae509e" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:13:17.744264  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 258855us
I20260709 13:13:17.722527  9938 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:17.775048  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 67859.3us
I20260709 13:13:17.713882  9731 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:17.749905  9858 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:17.737867  9935 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:13:17.754004  9845 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:17.773428  9938 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:17.781445  9950 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:13:17.781601  8138 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:13:17.822724  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 55077.2us
I20260709 13:13:17.784374  8730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 347) took 1329 ms. Trace:
I20260709 13:13:17.786715  9894 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:13:17.786957  9960 log.cc:927] Time spent T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:13:17.780090  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:17.756536  9734 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:13:17.758496  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:17.759459  9963 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.846987  9963 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:17.806941  9919 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:17.839287  9865 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:13:17.838193  9894 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:17.852173  8725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 331) took 1238 ms. Trace:
W20260709 13:13:17.822223  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 60028us
I20260709 13:13:17.852689  8725 rpcz_store.cc:276] 0709 13:13:16.613284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.627866 (+ 14582us) service_pool.cc:224] Handling call
0709 13:13:17.542960 (+915094us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:17.543567 (+   607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.851915 (+308348us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b540060de50. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:17.852103 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2004541568}
I20260709 13:13:17.853977  9894 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:17.823268  8730 rpcz_store.cc:276] 0709 13:13:16.454829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.455012 (+   183us) service_pool.cc:224] Handling call
0709 13:13:17.097173 (+642161us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:13:17.097421 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:17.240194 (+142773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.784092 (+543898us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54001793d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:17.784303 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298569472}
I20260709 13:13:17.856195  8130 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:17.856222  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:17.857393  8131 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:17.825711  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:17.858162  8130 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:17.859108  8133 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b186be823643475982e269b73d886a42"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:13:17.859714  8133 raft_consensus.cc:606] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:17.860119  8133 raft_consensus.cc:614] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:17.860677  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:17.823580  9731 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:17.819962  9845 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:17.754935  9838 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:13:17.750715  9868 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:17.811369  9857 raft_consensus.cc:697] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:17.825006  7961 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37774 (request call id 137) took 5999 ms. Trace:
I20260709 13:13:17.874447  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:17.874606  9916 raft_consensus.cc:3055] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:17.875026  9916 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:17.875329  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:17.876075  8055 raft_consensus.cc:2368] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4.
I20260709 13:13:17.875901  9916 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:17.876503  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 995) took 1021 ms. Trace:
I20260709 13:13:17.876899  8055 rpcz_store.cc:276] 0709 13:13:16.854613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.869797 (+ 15184us) service_pool.cc:224] Handling call
0709 13:13:17.875195 (+1005398us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b5800000c98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:17.876446 (+  1251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3794688}
I20260709 13:13:17.826987  9938 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:17.879455  9938 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:17.879644  9868 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:17.880811  9857 consensus_queue.cc:237] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:17.862929  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.884500  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:17.871927  8102 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:17.851533  9963 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:17.891952  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:17.869841  9845 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:17.892549  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:17.895866  8208 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:17.897518  9870 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.898066  9870 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:17.899591  9938 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:17.900465  9870 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:17.873049  7961 rpcz_store.cc:276] 0709 13:13:11.819812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:11.820121 (+   309us) service_pool.cc:224] Handling call
0709 13:13:12.571094 (+750973us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b5400172d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf80090 00007f6dbb01ef3a 00007f6dc0dd51cc 00007f6dc0ed337e 00007f6dbf3942a4 00007f6dbf394232 00007f6dbf394161 00007f6dbf3940dc 00007f6dbf393392 00007f6db935148c 00007f6db93508c6
0709 13:13:17.819234 (+5248140us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:14.775345 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:14.775995 (+   650us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:14.776047 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:13:14.776065 (+    18us) tablet.cc:662] Decoding operations
0709 13:13:14.797096 (+ 21031us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:14.797197 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:14.797230 (+    33us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:13:14.798026 (+   796us) tablet.cc:701] Row locks acquired
0709 13:13:14.798074 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:13:14.798220 (+   146us) write_op.cc:270] Start()
0709 13:13:14.798376 (+   156us) write_op.cc:276] Timestamp: P: 1783602794798169 usec, L: 0
0709 13:13:14.798400 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:13:14.872660 (+ 74260us) log.cc:844] Serialized 3284 byte log entry
0709 13:13:15.433395 (+560735us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b5400172d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb011e28 00007f6dbea4c581 00007f6dbea4bd58 00007f6dbea509f1 00007f6dbea509a9 00007f6dbea50939 00007f6dbea50901 00007f6dbea4fb2d 000000000038f354 000000000038f199
0709 13:13:15.943768 (+510373us) op_driver.cc:464] REPLICATION: finished
0709 13:13:16.136036 (+192268us) write_op.cc:301] APPLY: starting
0709 13:13:16.146540 (+ 10504us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:16.186129 (+ 39589us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:16.194725 (+  8596us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:16.292838 (+ 98113us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:16.293050 (+   212us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:13:16.293094 (+    44us) write_op.cc:312] APPLY: finished
0709 13:13:16.332959 (+ 39865us) log.cc:844] Serialized 65 byte log entry
0709 13:13:16.333139 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:16.333255 (+   116us) write_op.cc:454] Released schema lock
0709 13:13:16.333305 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:16.333667 (+   362us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:16.333699 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":159216,"reported_tablets":13,"spinlock_wait_cycles":1647310208,"child_traces":[["op",{"apply.queue_time_us":154024,"apply.run_cpu_time_us":3663,"apply.run_wall_time_us":229617,"num_ops":8,"prepare.queue_time_us":167669,"prepare.run_cpu_time_us":44438,"prepare.run_wall_time_us":658230,"raft.queue_time_us":359081,"raft.run_cpu_time_us":21896,"raft.run_wall_time_us":48388,"replication_time_us":1145231,"spinlock_wait_cycles":66535808,"thread_start_us":809327,"threads_started":4,"wal-append.queue_time_us":641072,"wal-append.run_cpu_time_us":27111,"wal-append.run_wall_time_us":1829555}]]}
W20260709 13:13:17.948738  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 76919.9us
I20260709 13:13:17.946017  9917 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:17.895273  9754 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.902436  9747 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.958568  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:17.959702  9974 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.092s
I20260709 13:13:17.961859  8132 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:17.962555  9794 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:17.965401  8059 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:17.965297  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:17.966476  8062 raft_consensus.cc:2410] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:17.968099  8059 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:17.968981  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:17.969954  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:17.969997  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:17.971535  8058 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:17.975454  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:17.973240  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:17.977962  8101 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:17.963745  9754 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:17.970695  9587 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:17.978956  9587 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:17.979477  8138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc8454910b36474c90b678c57cea9773"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:13:17.979780  8721 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:17.980063  9754 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:17.980355  8101 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:17.980823  9780 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:17.981027  9858 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:17.970530  9747 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:17.981447  9858 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.981527  9587 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:17.981873  9858 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:17.981940  9905 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:17.983668  9747 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:17.957448  9917 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:17.984987  9917 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:17.988924  9858 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:17.991472  9858 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:17.994676  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:17.995546  8057 raft_consensus.cc:2410] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:13:17.996296  9952 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:13:17.997145  9980 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:17.997540  9980 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.997861  9980 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:18.002964  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:18.027036  8061 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:18.030489  9919 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:13:18.034938  9780 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.035069  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 196) took 1052 ms. Trace:
I20260709 13:13:18.033701  9858 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.036787  8101 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:18.035483  9780 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.088222  9780 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:18.090255  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 53552.2us
I20260709 13:13:17.980094  8138 raft_consensus.cc:606] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:18.091719  8101 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:18.092805  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:18.094971  8059 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:18.095122  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:18.095659  8058 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:18.096346  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.090022  9780 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:18.092522  9974 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:13:18.108777  9845 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.112962  9745 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:17.983980  9905 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:18.149531  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 57897.4us
I20260709 13:13:18.105751  9980 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.180235  8101 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:18.136157  8138 raft_consensus.cc:614] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:18.149050  9747 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.183511  9979 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:18.183980  9970 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:18.147064  9652 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:18.186374  9798 consensus_meta.cc:287] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.000s
I20260709 13:13:18.147298  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:18.187821  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.173704  8725 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:18.165175  9655 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:18.190546  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:18.191732  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:18.204231  8135 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:18.204918  8137 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:18.182574  8026 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:18.204209  8133 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
W20260709 13:13:18.186983  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 506534us
I20260709 13:13:18.185982  9870 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.186815  9754 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.098558  9858 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.193205  9963 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.209568  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:18.231536  8133 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:18.231437  9963 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.232059  9975 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:18.232458  9975 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:18.233567  9963 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:18.209797  8058 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:18.208961  9870 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.211313  9972 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:18.239774  9972 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.240152  9972 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:18.195179  9970 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.242545  9876 raft_consensus.cc:2764] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:13:18.214569  9858 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:18.217329  9754 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.192783  9747 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.244630  9754 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:18.218952  9609 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.245798  9609 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.246160  9747 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:18.035526  8134 rpcz_store.cc:276] 0709 13:13:16.982195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:16.986253 (+  4058us) service_pool.cc:224] Handling call
0709 13:13:17.004916 (+ 18663us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:17.023363 (+ 18447us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:17.024832 (+  1469us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:17.028552 (+  3720us) log.cc:844] Serialized 26 byte log entry
0709 13:13:17.782376 (+753824us) spinlock_profiling.cc:243] Waited 915 us on lock 0x7b180042c640. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:13:17.890213 (+107837us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:17.893266 (+  3053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.896758 (+  3492us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:17.966507 (+ 69749us) raft_consensus.cc:1685] finished
0709 13:13:17.971204 (+  4697us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:18.035010 (+ 63806us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2395,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":3093,"spinlock_wait_cycles":2104064,"thread_start_us":105596,"threads_started":1,"wal-append.queue_time_us":881081}
I20260709 13:13:18.247743  9609 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:18.219403  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:18.220112  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:18.220873  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:18.248759  8131 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:18.249032  8721 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:18.249877  9845 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:18.238065  8101 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:18.221782  9587 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.199112  9798 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.223819  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:18.254216  9858 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:18.254719  9858 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:18.255052  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:18.255560  9858 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:18.225458  9951 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.759s	user 0.003s	sys 0.000s
I20260709 13:13:18.193512  9979 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.257071  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:18.196614  9652 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.228615  9984 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:18.233258  9975 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.238175  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:18.207772  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:18.236552  9870 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:18.260641  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 348427us
I20260709 13:13:18.282008  8058 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:18.253391  9731 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:18.285037  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:18.285665  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.285220  9731 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.287693  9731 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:18.284451  8175 leader_election.cc:400] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4.
I20260709 13:13:18.288358  8175 leader_election.cc:403] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:13:18.289108  9898 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4.
W20260709 13:13:18.291713  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.291832  8131 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:18.292372  8132 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:18.291867  8135 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:18.293110  9913 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.293670  9913 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.296165  9913 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:18.298643  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:18.298743  9913 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.299589  9913 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.300622  8135 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:18.301991  9913 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:18.304756  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:18.308136  8058 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:18.309175  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:18.310109  8058 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:18.226843  9745 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 13:13:18.342809  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:18.312462  8101 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:18.312842  9876 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:18.313090  9944 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.953s	user 0.002s	sys 0.005s
I20260709 13:13:18.313246  9870 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.313406  9798 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:18.260177  9587 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.307137  8129 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:18.314332  9988 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:18.316221  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:18.318562  9652 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:18.320504  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:18.322180  9972 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.325007  9805 raft_consensus.cc:3055] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:13:18.328191  9958 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.707s	user 0.001s	sys 0.004s
I20260709 13:13:18.258661  9979 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:18.334477  9919 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:18.281160  9754 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.336045  9905 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.280426  9857 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:13:18.339604  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 157128us
I20260709 13:13:18.403167  8026 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:18.339946  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:18.334975  9975 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:18.341126  9889 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.343851  8210 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:13:18.345674  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:18.347236  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:18.358503  8723 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:18.347788  8135 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:18.348922  9914 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.352115  8212 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:18.413609  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:18.353204  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:18.311643  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:18.351588  9970 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:18.358065  9876 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.360993  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:18.363413  8730 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:18.405040  8026 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:18.359766  9780 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:18.408239  8207 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:18.499646  9780 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.343261  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:18.500069  9780 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:18.500456  8137 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:18.359318  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:18.502374  9787 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:13:18.383479  8721 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:18.511574  8696 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:18.357439  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 45066.6us
I20260709 13:13:18.514637  8059 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:18.414444  9979 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.403462  9805 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:18.485026  8058 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:18.529363  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:18.414841  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:18.531675  8208 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
W20260709 13:13:18.511605  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 106643us
W20260709 13:13:18.512660  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.410034  9731 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:18.360881  9870 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:18.515547  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.548461  8101 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:18.516287  8059 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:18.548733  8026 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:18.554797  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:18.542249  8058 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:18.555868  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:18.557018  8026 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:18.409407  9754 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.407204  9905 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.409847  9857 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.560215  8176 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:18.582149  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 376720us
I20260709 13:13:18.563912  9964 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:18.565191  8027 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:18.515538  9986 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.572407  9870 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:18.412992  9889 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.527686  9974 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:18.522097  9845 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:18.466171  9624 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260709 13:13:18.515697  9747 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:18.575125  9805 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.406646  9919 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.577355  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:18.563347  9970 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:18.584668  8026 leader_election.cc:400] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 3. Current term is 4.
I20260709 13:13:18.482998  9914 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.566076  9876 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:18.362449  9988 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.535247  9979 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.594295  8029 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:18.588021  9845 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.598083  8026 leader_election.cc:403] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:13:18.583586  9798 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:18.581619  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 152) took 1461 ms. Trace:
I20260709 13:13:18.584244  9964 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:13:18.651969  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.661612  9916 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:18.795310 10002 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:18.587852  9986 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.598032  9970 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:18.601791  9853 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:18.596398  9870 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:18.594240  9898 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:18.645690  8138 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:18.598383  9996 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.091s
I20260709 13:13:18.956563  9898 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.956972  9898 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:18.957145  9996 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:18.595947  9974 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.959914  9974 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:18.773026  8210 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:18.592088  9887 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:18.617983  9745 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:18.781530  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 187358us
I20260709 13:13:18.964805  9898 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.966576  8054 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:18.967597  9887 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:18.778911  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:18.795734 10002 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.967998  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:18.968662 10002 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:18.779944  9776 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.970651  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:18.971098  9776 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:18.971405  8208 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:18.973106  9776 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:18.974627  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:18.778592  9747 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:18.975350  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:18.783854  9845 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:18.975733  9747 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:18.976733  8054 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:18.977787  9747 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:18.980048  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:18.980170  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:18.783953  9798 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.982002  9798 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:18.982539  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:18.982707  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:18.782985  9991 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.785149  9917 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.784734  9913 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.796269  9805 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:18.887244  9970 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:18.921537  9870 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.974712  8205 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:18.779801  9858 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.796554  9964 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:18.796051  9946 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:18.998732  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.021347  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:18.991962  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.022169  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:19.003145  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:18.981957  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:18.974148  9776 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:19.000675  9992 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
W20260709 13:13:18.999544  9994 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:13:19.000097  9997 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
W20260709 13:13:18.998863  9999 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.353s	user 0.000s	sys 0.000s
I20260709 13:13:19.041127  9870 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:19.003206  8054 raft_consensus.cc:2410] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:13:19.017273  9986 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.036669  9974 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:19.036108  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 451604us
I20260709 13:13:18.946802  9609 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.015301  9913 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.062474  8057 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:18.987804  9655 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.069336  8135 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:19.024072  9678 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:19.047711  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.041932  9990 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.015066  9919 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.014014  9731 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:19.105842  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:19.014216  9780 consensus_meta.cc:287] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.514s	user 0.005s	sys 0.000s
I20260709 13:13:19.016019  9979 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.071128  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.020342  9754 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:19.023887  9975 consensus_meta.cc:287] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.611s	user 0.008s	sys 0.000s
I20260709 13:13:19.021821  9914 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:18.987967  9853 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:19.021215  9988 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:18.968070  9887 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.051712  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.073205  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:19.042022  8212 raft_consensus.cc:2410] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:19.018422  9857 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.030316  9916 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.077471  8026 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:19.132910  8137 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:19.099315  9890 raft_consensus.cc:3055] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:19.068943 10008 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:18.986622  9991 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.030717  9917 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.143658  9624 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.146809  9624 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.147114  9917 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.082165  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.148037  8132 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:19.083293  9913 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:19.083820  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2011) took 1111 ms. Trace:
I20260709 13:13:19.083314  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2009) took 1114 ms. Trace:
I20260709 13:13:19.149000 10008 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.115531  9990 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.149226  8056 rpcz_store.cc:276] 0709 13:13:17.969200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.970867 (+  1667us) service_pool.cc:224] Handling call
0709 13:13:18.034429 (+ 63562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:18.034789 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:18.034823 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:18.035264 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 13:13:18.346159 (+310895us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:18.348005 (+  1846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:18.353055 (+  5050us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:19.083095 (+730040us) raft_consensus.cc:1685] finished
0709 13:13:19.083114 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.083258 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2308,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":20071,"thread_start_us":304944,"threads_started":1,"wal-append.queue_time_us":556843}
I20260709 13:13:19.151216 10008 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.151432  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:19.152048  8056 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:19.152650  9990 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.079771  9609 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.086634 10011 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.070644  9946 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.087204  9798 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.088380  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.050467  9858 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.090754  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.068552  9776 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.091493  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:19.093056  9996 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 3. Current term is 4.
I20260709 13:13:19.165174  8129 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:19.165443  9798 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.165940  9858 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.166582  8205 raft_consensus.cc:2393] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:19.167402  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.167987  8205 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:19.169821  9776 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:19.076853  9986 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:13:19.170773  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.171425  8134 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
I20260709 13:13:19.094498  9655 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.170734  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.173525  9889 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:19.166172  9853 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.079113  9974 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.174630  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:18.786824  8063 rpcz_store.cc:276] 0709 13:13:17.120358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.166086 (+ 45728us) service_pool.cc:224] Handling call
0709 13:13:17.182619 (+ 16533us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:17.183061 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:17.183109 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:17.183680 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:13:17.241408 (+ 57728us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:17.262997 (+ 21589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:17.273023 (+ 10026us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:18.312460 (+1039437us) spinlock_profiling.cc:243] Waited 8.96 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:18.515596 (+203136us) raft_consensus.cc:1685] finished
0709 13:13:18.515618 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:18.576095 (+ 60477us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":20607872,"thread_start_us":35309,"threads_started":1,"wal-append.queue_time_us":95814}
I20260709 13:13:19.175952  9974 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.175912  8056 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:19.176530  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:19.174235  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.090229 10002 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:19.177275  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:19.177992  8136 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:19.169778  7959 catalog_manager.cc:5697] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e reported cstate change: leader changed from <none> to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 3 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:13:19.110356  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 117717us
I20260709 13:13:19.131428  8208 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:19.132104  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:19.136004 10005 log.cc:927] Time spent T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:13:19.101233  9964 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.110522  9678 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.139866  9887 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.118945  9845 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:19.133571  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 56194.7us
I20260709 13:13:19.148798  9624 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:19.148941  8062 rpcz_store.cc:276] 0709 13:13:17.972541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.974212 (+  1671us) service_pool.cc:224] Handling call
0709 13:13:17.974562 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:17.974797 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:17.974831 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:18.029963 (+ 55132us) log.cc:844] Serialized 26 byte log entry
0709 13:13:18.347969 (+318006us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:18.351811 (+  3842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:18.352102 (+   291us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:19.083634 (+731532us) raft_consensus.cc:1685] finished
0709 13:13:19.083655 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.083771 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":199080,"threads_started":1,"wal-append.queue_time_us":585051}
I20260709 13:13:19.216702  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.217015  8057 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:13:19.158725  8207 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:19.142287  9890 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:19.097318  9870 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.159117 10011 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.161022  9609 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.140813  9754 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:19.145977  9970 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.164186  8131 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:19.171870  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.238507  8137 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:19.189153  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:19.239336  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:19.239583  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 50514.2us
I20260709 13:13:19.201084  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2007) took 1232 ms. Trace:
I20260709 13:13:19.240862  8061 rpcz_store.cc:276] 0709 13:13:17.968577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.968707 (+   130us) service_pool.cc:224] Handling call
0709 13:13:19.200913 (+1232206us) spinlock_profiling.cc:243] Waited 57.3 ms on lock 0x7b540001b154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:19.201031 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131789696}
I20260709 13:13:19.241849  8061 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:19.242956  8061 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:19.244055  8061 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:19.177402  8063 raft_consensus.cc:2410] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:13:19.202544  8059 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:13:19.202272  7959 catalog_manager.cc:5697] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:19.204056  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.219120  9974 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.204826  9805 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.206288  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.211203  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:18.967913  9898 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.143891  9916 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.217366  8026 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:19.217389  8062 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:19.218202  8057 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:13:19.290028  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 72738us
I20260709 13:13:19.173525  9655 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.231891  8102 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:19.159328  9780 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.201798  9973 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:19.211570  9889 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:19.234318  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.261333  9952 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.244817  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.246482  8136 raft_consensus.cc:2410] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:13:19.204439 10016 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.250622  9917 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:19.251485  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.251917  8063 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:19.254660  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.308614  8132 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:19.256407  9870 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.255473  7959 catalog_manager.cc:5697] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 2 to 3, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 3 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:19.310016  7959 catalog_manager.cc:5697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 2 to 3, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 3 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:19.311303  7959 catalog_manager.cc:5697] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 2 to 3, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 3 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:19.216298  9787 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:13:19.257587  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:19.259616  9964 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.216269  9946 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.287853  9914 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:13:19.278980  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.231031  9853 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:19.236671  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:19.227161  9887 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:13:19.291280  8133 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:19.291865  8026 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:19.230471  9771 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:13:19.246986  9731 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:19.287627  8726 raft_consensus.cc:2393] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:19.244184  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:19.295251  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 63433.9us
I20260709 13:13:19.312687  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:19.375283  8130 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:19.375586  8102 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:19.314030  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.377293  8102 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:19.246847  9912 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:13:19.247257  9990 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:13:19.315930  8727 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:19.336869  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.294886  9754 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:19.225597 10008 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:19.298767  9991 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.312747  7959 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 2, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:19.379921  7959 catalog_manager.cc:5697] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:19.284040  9974 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:19.381927  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 90126.9us
I20260709 13:13:19.359720  8212 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:19.313849  9917 raft_consensus.cc:697] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:19.366669  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2129) took 1007 ms. Trace:
I20260709 13:13:19.301981  9889 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.286548  9805 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.381032  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1016) took 1179 ms. Trace:
I20260709 13:13:19.387399  8130 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:19.387924  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.391561  8210 rpcz_store.cc:276] 0709 13:13:18.359699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.413301 (+ 53602us) service_pool.cc:224] Handling call
0709 13:13:19.366529 (+953228us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b540010c554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:19.366619 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":531143552}
I20260709 13:13:19.343443  9652 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:19.392750  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.393509  8059 rpcz_store.cc:276] 0709 13:13:18.202044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.549453 (+347409us) service_pool.cc:224] Handling call
0709 13:13:19.202040 (+652587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:19.245970 (+ 43930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.365120 (+119150us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b540001b150. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:19.380980 (+ 15860us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1500212096}
I20260709 13:13:19.359102  9780 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.342900 10012 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.105s
I20260709 13:13:19.393929  8133 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:19.398142  8727 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:19.404549  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:19.395030  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 112383us
I20260709 13:13:19.400920  8137 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:19.391391  7959 catalog_manager.cc:5697] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 2 to 3, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:19.401423  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.405475  8133 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:19.398964  8132 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:19.406662  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:19.363485  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.381839  9754 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.374295  8722 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:19.374591  9731 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.390635  9858 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:19.382241  9968 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:19.306233  9952 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.407338  8130 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:19.310497 10016 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:19.414119  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 210756us
I20260709 13:13:19.416807  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.432472  8132 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:19.396085  9747 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:19.363677  9853 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.305995  9973 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.409081  7959 catalog_manager.cc:5697] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:19.440537  7959 catalog_manager.cc:5697] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 2 to 3, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 3 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:19.400346  9986 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:13:19.403131  9652 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.403498 10002 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.395838  9988 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:13:19.394435 10008 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.442595  7959 catalog_manager.cc:5697] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 2 to 3, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 3 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:19.445477  9889 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:19.421598  9678 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:19.412295  9754 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:19.408317  8134 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:19.397920  9946 consensus_queue.cc:1059] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:13:19.414786  9731 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:19.587952  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.588726  8132 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:19.448150  9780 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.469647  7959 catalog_manager.cc:5697] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:19.411078  9905 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:19.432210  9845 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.459049  8175 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:19.402776 10009 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:13:19.449245  9747 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:19.583323  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 186568us
I20260709 13:13:19.603483  8026 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:19.603629  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.458813  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:19.458381  9655 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:19.542867  8130 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:19.605563  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:19.604209  8132 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:19.470953  9970 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.587083  9889 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.458585  9917 consensus_queue.cc:237] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.606823  9889 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.596177  9780 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.609391  9652 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.593902  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.609754  9889 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.610107  8131 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:19.610229  9780 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:19.610874 10004 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.830s	user 0.000s	sys 0.004s
I20260709 13:13:19.611609  9747 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:19.601881  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 142905us
I20260709 13:13:19.591831  9887 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:19.584844  9858 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:19.631716  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:19.615666 10016 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.617980  9973 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.479444  9991 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.126s
I20260709 13:13:19.619736  9952 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:19.620978 10026 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
W20260709 13:13:19.621577 10003 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.633s	user 0.000s	sys 0.006s
W20260709 13:13:19.643028 10020 log.cc:927] Time spent T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 13:13:19.642592  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:19.597951  9964 raft_consensus.cc:3055] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:19.643800  9964 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:19.643877 10028 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.162s
I20260709 13:13:19.644678  9964 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:19.645258 10024 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 13:13:19.623036 10008 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.481462  9776 consensus_queue.cc:1059] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.123s
I20260709 13:13:19.611939  8725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 342) took 1276 ms. Trace:
I20260709 13:13:19.658514  8725 rpcz_store.cc:276] 0709 13:13:18.335586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.336413 (+   827us) service_pool.cc:224] Handling call
0709 13:13:18.336808 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:18.337257 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:18.337299 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:18.337874 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 13:13:18.559924 (+222050us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:18.589328 (+ 29404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:18.594530 (+  5202us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:19.610228 (+1015698us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:19.611680 (+  1452us) raft_consensus.cc:1685] finished
0709 13:13:19.611698 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.611884 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2190,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":961,"spinlock_wait_cycles":6383488,"thread_start_us":189641,"threads_started":1,"wal-append.queue_time_us":275056}
I20260709 13:13:19.598196  9916 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:19.662722  9776 raft_consensus.cc:3055] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:19.663249  9776 raft_consensus.cc:740] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:19.664161  9776 consensus_queue.cc:260] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.665632  9776 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:19.635336 10019 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:19.639284  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:19.605175 10002 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.605243 10012 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:19.645740  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1013) took 1444 ms. Trace:
I20260709 13:13:19.648111  9905 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:13:19.642540 10023 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:13:19.648885  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:19.624377  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:19.658017  9964 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:19.612447  9996 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:19.696904  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.672735  9805 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.700847  9984 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:19.701476  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:19.704360  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 215) took 1354 ms. Trace:
I20260709 13:13:19.707064  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:19.767853  8207 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:19.691088  9980 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.655838  9991 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.105s
I20260709 13:13:19.708132  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.709623  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.745784  9946 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.715190  9890 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.717846  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.776139  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:19.776827  8209 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:19.722395  9734 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:19.724531  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:19.731557  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.789000  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:19.733323  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:19.789942  8133 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:19.735342  8056 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:47112
W20260709 13:13:19.735577  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 491753us
I20260709 13:13:19.737510  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 210) took 1392 ms. Trace:
I20260709 13:13:19.641340  9876 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:19.676005  8060 rpcz_store.cc:276] 0709 13:13:18.201153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.201947 (+   794us) service_pool.cc:224] Handling call
0709 13:13:18.202261 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:18.202599 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:18.202634 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:18.203130 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 13:13:18.527955 (+324825us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:18.601013 (+ 73058us) log.cc:844] Serialized 14 byte log entry
0709 13:13:18.612296 (+ 11283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:18.612386 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:19.619894 (+1007508us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b18003a6120. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:13:19.620957 (+  1063us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:19.645560 (+ 24603us) raft_consensus.cc:1685] finished
0709 13:13:19.645580 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.645690 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14756,"prepare.queue_time_us":88428,"prepare.run_cpu_time_us":6583,"prepare.run_wall_time_us":119253,"raft.queue_time_us":40091,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":136,"spinlock_wait_cycles":60838016,"thread_start_us":247598,"threads_started":1,"wal-append.queue_time_us":529554}
I20260709 13:13:19.634100  9887 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.636533  9888 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:13:19.636708  9652 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
W20260709 13:13:19.755887  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 107122us
I20260709 13:13:19.758266  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.817602  8212 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:19.698536  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.637837  9858 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.707651 10028 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:19.753160 10019 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.767131  8206 rpcz_store.cc:276] 0709 13:13:18.350214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.350334 (+   120us) service_pool.cc:224] Handling call
0709 13:13:18.350644 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.782079 (+431435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:18.782106 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:18.782145 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:18.782174 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.704087 (+921913us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54002c8b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:19.704144 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.704283 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":622079232}
I20260709 13:13:19.712178 10011 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.744861  9905 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:19.759073  9996 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.774482  8724 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:19.767880 10012 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:19.767565  9974 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.744593 10002 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.795516  8029 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:19.803360  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.853955  8208 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:19.829095  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:19.855338  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 59914us
I20260709 13:13:19.797468  9853 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.818670  8210 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:19.856782  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:19.817649  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:19.785408  9734 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:19.843005  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.792642  8721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 354) took 1199 ms. Trace:
I20260709 13:13:19.798488  9913 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.771039  9984 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.786370  9946 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.820582  8056 raft_consensus.cc:606] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:19.826453  8204 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:19.855396  8212 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
W20260709 13:13:19.822361  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:19.861279  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 43701.9us
I20260709 13:13:19.862397  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:19.863129  8212 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:19.863588  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:19.810021  8207 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:19.857821  8206 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:19.859532  8029 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:19.820277  9876 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.844348  9905 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:19.849201  9996 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:19.857672  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:19.824625  9890 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:19.849759 10029 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:19.877079  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 438132us
I20260709 13:13:19.906960  8208 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:13:19.860445  8056 raft_consensus.cc:614] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:19.872916  9988 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:19.840926 10028 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.827397  9754 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:19.812984  8209 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:19.851800 10012 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.856559 10009 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:19.904711  9747 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:19.955837  9747 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.876983  9925 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:19.873008  9652 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:19.909456  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.860991  9986 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
W20260709 13:13:19.879573 10037 log.cc:927] Time spent T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 13:13:19.961607  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 102183us
I20260709 13:13:19.876678  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:19.961867 10030 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:13:19.859274  8721 rpcz_store.cc:276] 0709 13:13:18.589130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.591799 (+  2669us) service_pool.cc:224] Handling call
0709 13:13:19.624235 (+1032436us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:19.788144 (+163909us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13428480}
I20260709 13:13:19.937101  9754 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.960788  9747 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:19.981948  8133 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:19.961369  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:19.907925  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:20.022790  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 114936us
I20260709 13:13:20.023619  8693 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:20.024106  9946 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.025184  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:19.923467 10029 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.958016 10012 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:19.871062  9952 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:19.980657  9652 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:19.871074  9913 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
W20260709 13:13:20.038842  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:19.997637  8132 raft_consensus.cc:2410] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:20.017300  8212 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
W20260709 13:13:20.018940  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 280191us
I20260709 13:13:20.019025  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:19.873545 10010 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 13:13:19.969645  9925 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.033545  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.021258  9858 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:19.957036  9734 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:19.793709  8203 rpcz_store.cc:276] 0709 13:13:18.345271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.345978 (+   707us) service_pool.cc:224] Handling call
0709 13:13:19.097290 (+751312us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b5800168398. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1eee 0000000000375939 00007f6dbff651e0 00007f6dc002e0c4 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:13:19.097459 (+   169us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:19.146838 (+ 49379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:19.146875 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:19.147326 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:13:19.278400 (+131074us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:19.285808 (+  7408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.737087 (+451279us) spinlock_profiling.cc:243] Waited 49 ms on lock 0x7b540018a454. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:19.737159 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:19.737231 (+    72us) raft_consensus.cc:1685] finished
0709 13:13:19.737251 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:19.737435 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":876,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"spinlock_wait_cycles":115125376,"thread_start_us":83453,"threads_started":1,"wal-append.queue_time_us":153251}
I20260709 13:13:20.217108  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:20.222847  8059 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:19.995364  8134 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:20.253921  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.143464  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:20.255201  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:20.136090  8061 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:20.261745  8133 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:20.127511  8211 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:13:20.131063  8136 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:20.025918  9988 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:20.139364  9731 consensus_meta.cc:287] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.541s	user 0.008s	sys 0.000s
I20260709 13:13:20.261968  8059 raft_consensus.cc:2393] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:20.133406 10034 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:13:20.119916  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.037446  9652 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.131639  8027 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:20.122473  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:20.297904  8210 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:20.027640 10029 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.034538 10040 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:20.127101 10015 raft_consensus.cc:3055] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:20.275115  8137 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:20.037207  9609 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.263s
I20260709 13:13:20.109535  9876 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:20.204293  9986 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:20.300199  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 168615us
I20260709 13:13:20.199003  9655 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.255821  9946 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.030369  9845 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:20.167476 10041 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:13:20.130885  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1018) took 1165 ms. Trace:
I20260709 13:13:20.034009  9899 raft_consensus.cc:2764] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:13:20.341305  8027 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:20.307179  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.190943  8055 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:20.345945  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:20.117581 10019 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.033628  9754 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:20.133494 10002 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.120986  9984 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.096619 10028 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:20.107553  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.325933  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:20.264513  9925 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:20.039745  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.139986  9972 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.117506  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:20.128962  8726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 346) took 1155 ms (client timeout 1536 ms). Trace:
I20260709 13:13:20.137300  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 867) took 1846 ms. Trace:
I20260709 13:13:20.188664  9952 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:20.301308  9734 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:20.311164 10009 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:20.300356  9913 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:20.135203  9996 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.125761  9905 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.309813 10011 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.118432  9745 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:20.346493  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2020) took 1827 ms. Trace:
I20260709 13:13:20.346493  8729 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:20.335211 10015 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:13:20.349871  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.344075  8054 rpcz_store.cc:276] 0709 13:13:18.965876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.967719 (+  1843us) service_pool.cc:224] Handling call
0709 13:13:20.130701 (+1162982us) spinlock_profiling.cc:243] Waited 7.78 ms on lock 0x7b5400219b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:20.130832 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17897344}
I20260709 13:13:20.309708  8129 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:20.352741  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:20.338259  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.301038  9988 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:20.392880  8726 rpcz_store.cc:269] 0709 13:13:18.973931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.975113 (+  1182us) service_pool.cc:224] Handling call
0709 13:13:20.128727 (+1153614us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b540060f754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:20.128893 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":599367040}
I20260709 13:13:20.416121  9905 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:20.416277  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:20.356741  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:20.417361  8726 raft_consensus.cc:2393] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:20.417613  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:20.418287  8728 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:20.356873  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.419143 10009 raft_consensus.cc:697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:20.419530 10010 consensus_queue.cc:1059] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:20.420019 10009 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:20.420506 10010 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:20.421779 10010 raft_consensus.cc:3055] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:20.308735  9731 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.423656  9731 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.363507  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:20.368680  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.424511  8730 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.427804  8176 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:20.369557  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.429411  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:20.429937  9914 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.430313  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.404016  9952 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:20.430564  9914 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.431100 10028 raft_consensus.cc:2764] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:13:20.431380  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.395576  8138 rpcz_store.cc:276] 0709 13:13:18.290624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.290744 (+   120us) service_pool.cc:224] Handling call
0709 13:13:18.291042 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.956335 (+665293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.137073 (+1180738us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400409cd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:20.137244 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736694656}
I20260709 13:13:20.432564  9914 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:20.433040  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.435539  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:20.369434  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.436548  8211 raft_consensus.cc:2376] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6. Re-sending same reply.
W20260709 13:13:20.371737  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.438546  9968 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:20.438961  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:20.439554  9996 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.372144  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:20.440966  8059 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:20.347414  9870 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:20.343426  9986 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:20.378835  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 204) took 1246 ms. Trace:
I20260709 13:13:20.342864  9655 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.358199 10002 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:20.358423  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2005) took 2381 ms. Trace:
I20260709 13:13:20.407889 10040 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:20.375082  9747 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.406093  8058 rpcz_store.cc:276] 0709 13:13:18.519082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:18.556679 (+ 37597us) service_pool.cc:224] Handling call
0709 13:13:18.556922 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:18.557121 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:20.166050 (+1608929us) spinlock_profiling.cc:243] Waited 4.98 ms on lock 0x7b1800398b00. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf
0709 13:13:20.166217 (+   167us) log.cc:844] Serialized 14 byte log entry
0709 13:13:20.345900 (+179683us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:20.345952 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:20.345993 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.346286 (+   293us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.346432 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40219,"prepare.queue_time_us":226797,"prepare.run_cpu_time_us":7265,"prepare.run_wall_time_us":62145,"raft.queue_time_us":2530,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":6273,"spinlock_wait_cycles":11458304,"thread_start_us":504052,"threads_started":2,"wal-append.queue_time_us":226832}
I20260709 13:13:20.463460  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.464046  8058 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:20.464823  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.465174  8058 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:20.465667  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.466009  8058 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:20.349992  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.376158  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:20.469974  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 885) took 1222 ms. Trace:
I20260709 13:13:20.410914  8026 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:20.470386  8136 rpcz_store.cc:276] 0709 13:13:19.247286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.709330 (+462044us) service_pool.cc:224] Handling call
0709 13:13:20.469826 (+760496us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b540015b154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:20.469926 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":826292096}
W20260709 13:13:20.471344  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 60514.3us
I20260709 13:13:20.471539  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:20.472026  8026 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:20.473116  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:20.413888  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:20.375268  8056 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:20.432353  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.373243  8104 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:20.417456  9913 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
W20260709 13:13:20.477062  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 103854us
I20260709 13:13:20.477842  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:20.429972  8212 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.444576  8176 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:20.478837  8104 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:20.445040  8730 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:20.446448  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:20.403275  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:20.449178  8135 rpcz_store.cc:276] 0709 13:13:19.132052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.132773 (+   721us) service_pool.cc:224] Handling call
0709 13:13:19.133044 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:19.179988 (+ 46944us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:19.180030 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:19.205746 (+ 25716us) log.cc:844] Serialized 26 byte log entry
0709 13:13:19.678553 (+472807us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:19.678591 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:19.742232 (+ 63641us) spinlock_profiling.cc:243] Waited 46.5 ms on lock 0x7b54003c8cd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:19.742297 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:20.374418 (+632121us) raft_consensus.cc:1685] finished
0709 13:13:20.374439 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.378766 (+  4327us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1275,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":10165,"spinlock_wait_cycles":107035392,"thread_start_us":441200,"threads_started":1,"wal-append.queue_time_us":684850}
I20260709 13:13:20.410396 10011 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.422127 10010 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:20.452447  9587 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:20.410590  9745 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.410279  9886 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.416739  9988 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.344801  9946 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:20.466698  8057 raft_consensus.cc:2393] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:20.468065  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:20.468361  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.468974  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.446472  9887 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.473434  8026 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:20.355422  9754 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.475328  8136 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:20.526894  8059 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:20.527707  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:20.481655  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:20.527982  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.508034  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:20.528693  8059 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:20.528920  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:20.529399  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.529779  8059 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:20.481233  8104 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:20.484500  8176 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:20.439028  9968 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:20.456338  9972 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:20.489413  8729 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:20.500954  8058 raft_consensus.cc:2410] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:13:20.539701  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 66330.7us
I20260709 13:13:20.540340  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:20.540406  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.541682  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.501477  8212 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:20.530335  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.543285  8059 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:20.543591  9887 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:20.543650  9988 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:20.530961  8136 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:20.487226  9974 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:20.480818 10002 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:20.534323  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 49904.1us
I20260709 13:13:20.484910 10052 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:20.538872  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:20.557919  8135 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:20.486557 10019 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:20.503410  8726 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:13:20.544059  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.547178  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.549696  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:20.492326  9561 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:20.531412  8725 raft_consensus.cc:2368] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4.
W20260709 13:13:20.503513 10051 log.cc:927] Time spent T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:13:20.544046  9887 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.499220  9776 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:20.555637  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.470237 10040 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:20.490875  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.535989  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:20.563951 10019 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.557853  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.593433  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.558913  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:20.559382  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.560683  8209 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:20.595578  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:20.596012  8209 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.563019  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:20.455458  8055 rpcz_store.cc:276] 0709 13:13:17.967963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:17.976175 (+  8212us) service_pool.cc:224] Handling call
0709 13:13:19.724340 (+1748165us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b5800000c98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:20.349107 (+624767us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119202176}
I20260709 13:13:20.469729  9747 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.550276  9988 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:20.609251  9988 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:20.610443  9747 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.555009 10010 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.576092  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:20.555173  9587 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.622802  9587 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.624480  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.624765  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:20.625248  9587 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.625341  8725 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:20.551941  9974 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.552626  9917 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:20.498996 10014 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.076s
I20260709 13:13:20.626101  9974 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.578692  8101 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:13:20.627414  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 48790us
I20260709 13:13:20.584918 10053 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:20.556694 10002 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.579262  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:20.536664  9914 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:13:20.536504 10028 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:13:20.536859  9928 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:13:20.566064  9853 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:20.545945  8026 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:13:20.639765  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 93890.1us
I20260709 13:13:20.640758  8026 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:13:20.591205  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 55291.3us
I20260709 13:13:20.642078  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:20.643064  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:20.644089  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:20.645071  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:20.483649  9858 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:20.603345  9968 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:20.555228  9996 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:20.564980  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:20.557610 10052 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.596698  8136 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:20.598244  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.599066 10027 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:20.600030  9652 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:20.604503  9655 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.605242  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:20.597447 10019 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:20.607021  8063 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
W20260709 13:13:20.667675 10050 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:13:20.668635  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1204) took 1279 ms. Trace:
I20260709 13:13:20.668977  8130 rpcz_store.cc:276] 0709 13:13:19.389199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.606080 (+216881us) service_pool.cc:224] Handling call
0709 13:13:19.606399 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:19.624986 (+ 18587us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:19.625023 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:19.633542 (+  8519us) log.cc:844] Serialized 26 byte log entry
0709 13:13:20.138217 (+504675us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b18003a0720. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:13:20.344761 (+206544us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:20.344800 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.363441 (+ 18641us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b540008d254. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:20.363494 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:20.668423 (+304929us) raft_consensus.cc:1685] finished
0709 13:13:20.668449 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:20.668584 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3225,"raft.queue_time_us":1187,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":14856,"spinlock_wait_cycles":84916352,"thread_start_us":206397,"threads_started":1,"wal-append.queue_time_us":772618}
W20260709 13:13:20.670773 10012 consensus_meta.cc:287] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.630s	user 0.003s	sys 0.004s
I20260709 13:13:20.671134 10012 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.672106  8212 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:20.673027 10012 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:20.586140  9887 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:20.674482  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:20.674147 10019 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.676687 10019 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.586481  9776 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.677872  9776 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:20.678308  9974 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.615460  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:20.615499  8693 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:20.502688  9886 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:20.682914  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 67481us
I20260709 13:13:20.683815  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:20.683974  9886 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:20.552512  9972 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.616950  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:20.618314  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:20.616535  8209 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:20.685699  9972 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.620709 10029 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.622128  9952 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:20.628054  8027 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:20.628474  8101 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:13:20.687017  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 58605us
I20260709 13:13:20.687601  8101 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:20.687783 10029 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.688398  8101 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:20.631107  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 51940.2us
I20260709 13:13:20.545317  8054 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:20.646620  8026 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:20.694955  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 48436.2us
I20260709 13:13:20.695847  8026 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:20.647320  9998 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:20.476696 10015 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.633315 10002 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:20.701601  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 73648.2us
I20260709 13:13:20.633955  9980 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:20.633131  9587 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:20.654840  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:20.651801  9919 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:20.651669  9561 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:20.657150  9898 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:20.650527  9858 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:13:20.657646 10056 log.cc:927] Time spent T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:13:20.650712 10014 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:20.653865  9986 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:20.659471 10011 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.706306  9886 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:20.660349  9988 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:20.643453 10053 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:20.660993  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:20.662567  9964 consensus_meta.cc:287] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.902s	user 0.015s	sys 0.014s
I20260709 13:13:20.663170 10040 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.663662 10027 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:20.685166  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:20.688525 10052 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:20.642189  9853 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.689229  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:20.626147  9917 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.690480  8058 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:20.698987  8101 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:20.680243  9974 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:20.702499  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 221316us
I20260709 13:13:20.807338  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:20.704449  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 49670.4us
I20260709 13:13:20.689021  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:20.697458  9952 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:20.703655  9655 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:20.704690 10019 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:20.705008  9972 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:20.728819  8136 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:20.826743  8104 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:20.827142  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:20.700068  9998 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:20.703620  9980 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:20.659642  9652 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:20.827805  8104 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:20.715576 10029 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:20.716639  9561 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.712715  9919 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:20.717099 10015 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.721292  9858 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:20.716138  9898 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:20.723866  9747 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:20.839834  9561 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.787519  8132 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:20.768999  9946 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:20.734043  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:20.805819  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 116667us
I20260709 13:13:20.768251  9587 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:20.807775  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 108864us
I20260709 13:13:21.015969  8101 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:13:20.808478  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 123376us
I20260709 13:13:21.018882  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:20.776134  9754 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.776485 10028 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:21.025257 10028 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:20.777240  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.026216  8054 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:21.027155  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.027485 10028 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:21.027732  8054 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:21.028781  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.029451  8054 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:20.656226  9996 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:20.828445  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 125231us
I20260709 13:13:21.031059  9754 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:13:20.811280 10059 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:13:21.031509  9754 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.031994  9996 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:20.829531  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:21.032783  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 203317us
I20260709 13:13:21.033427  9754 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:21.034307  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.034430  9734 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:13:21.034755  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:21.035311  8175 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.616s (SENT) [suppressed 17 similar messages]
I20260709 13:13:20.805382  9853 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.036142  8726 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:21.036495  9988 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:20.831236  8104 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:20.806293  9917 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:20.806100 10027 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.806372 10053 raft_consensus.cc:697] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:21.042092  9814 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:20.837121  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:20.813854  9968 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:20.786417  9925 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:20.814383  9986 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.824379  9655 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:20.811940 10040 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:20.818460 10064 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:20.833808  9980 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:20.830142  9845 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:20.830914  9998 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.826536  9972 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:20.824076  9914 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:20.722805 10014 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.830056 10029 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:20.834227  9964 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:20.838363  9865 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:20.842620  9919 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:20.826083 10011 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:13:20.831368 10067 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:20.968412  9776 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:20.977893  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.112264  8060 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:21.113317  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:20.963990  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:20.969756 10068 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:13:20.970217  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:20.985898  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:20.864940  9898 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.128309  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 892) took 1165 ms. Trace:
I20260709 13:13:21.128768  8132 rpcz_store.cc:276] 0709 13:13:19.963543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.298395 (+334852us) service_pool.cc:224] Handling call
0709 13:13:21.128096 (+829701us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b5400062054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:21.128226 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":972794624}
W20260709 13:13:21.006888 10071 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:13:21.045404  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:21.013147  9745 consensus_meta.cc:287] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.002s
I20260709 13:13:20.979061 10019 raft_consensus.cc:697] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:13:21.041704  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 210522us
I20260709 13:13:21.137595  8104 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:21.138537 10081 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:21.143666  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.144313  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:21.146727  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:21.147415  8723 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.148073  9917 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.150214  9917 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:21.150694  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:21.151515  8203 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.154796  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:21.156035  8211 raft_consensus.cc:2368] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5.
I20260709 13:13:21.157370  8723 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:21.034042  9887 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.168146  9887 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:21.102169  8136 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:21.104359  8101 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:13:21.170001  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 65732.4us
I20260709 13:13:21.170816  8101 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:21.171868  8101 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:21.114163  8060 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:21.174110  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.174971  8060 raft_consensus.cc:2410] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:21.175829  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.176648  8060 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:21.177438  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.177771  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:21.044399 10047 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.772s	user 0.003s	sys 0.000s
W20260709 13:13:21.052502 10042 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.752s	user 0.000s	sys 0.002s
I20260709 13:13:21.041102  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.119208  8207 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:21.053622 10028 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:21.034799  9734 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:21.103040  9814 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:21.107319  9986 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.124569  9609 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:21.126374  9776 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:21.036209  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 895) took 1071 ms. Trace:
I20260709 13:13:21.106135 10029 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.103929  9972 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.105990  9655 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:20.962133  8029 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:21.157754  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.158540  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.218760  8210 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:21.160151  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.046871  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:21.219637  8058 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:21.219704  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 257657us
I20260709 13:13:21.220762  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:21.223742  8029 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:21.112798  9865 raft_consensus.cc:697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:21.115906  9968 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:21.117125 10064 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:21.089553  9587 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.204332  8203 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:21.238077  8058 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:21.103293 10014 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.136415  8131 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.075039  9988 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
I20260709 13:13:21.190946  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:21.188171  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2138) took 1491 ms. Trace:
I20260709 13:13:21.107080  9914 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:21.201733  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 166456us
I20260709 13:13:21.206903  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 216) took 1169 ms. Trace:
I20260709 13:13:21.184122 10081 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:21.185493  9964 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:21.191190  9734 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.250029  9925 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:21.250494  9968 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.201923  9814 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.183223  9745 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.212430 10028 raft_consensus.cc:697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:21.190081  8054 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:21.188803  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.267112  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.267854  8060 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:21.268929  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:21.269565  8060 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:21.235625  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 57922.5us
I20260709 13:13:21.213970 10080 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:21.191918 10002 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.237370  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:21.127676  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:21.225162  9870 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:21.230886 10011 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:21.225394 10009 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:13:21.239426  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 94956.3us
I20260709 13:13:21.043066 10053 consensus_queue.cc:237] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:21.244908  8206 rpcz_store.cc:276] 0709 13:13:19.696371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:19.865123 (+168752us) service_pool.cc:224] Handling call
0709 13:13:19.865467 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:19.907160 (+ 41693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:19.907186 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:19.962204 (+ 55018us) log.cc:844] Serialized 26 byte log entry
0709 13:13:20.194706 (+232502us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:20.194749 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.194821 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:21.043824 (+849003us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:21.179690 (+135866us) raft_consensus.cc:1685] finished
0709 13:13:21.179718 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.188114 (+  8396us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41290,"raft.queue_time_us":42787,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":316591360,"thread_start_us":173951,"threads_started":1,"wal-append.queue_time_us":225636}
I20260709 13:13:21.264380  9853 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.247869  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:21.239696  9876 consensus_queue.cc:1059] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:13:21.237952  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.254818  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.271173  9858 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.331634  8058 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:21.255611  8104 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:21.258512 10064 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:21.261417  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.333957  8058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e397a319b11487fbbacec77723b5739"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:13:21.209784  8134 rpcz_store.cc:276] 0709 13:13:19.964301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.420095 (+455794us) service_pool.cc:224] Handling call
0709 13:13:20.420519 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:20.420964 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:20.421008 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:20.421595 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 13:13:20.538186 (+116591us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:20.551421 (+ 13235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.563839 (+ 12418us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:21.035981 (+472142us) raft_consensus.cc:1685] finished
0709 13:13:21.035998 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.036144 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12475,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":91,"thread_start_us":90119,"threads_started":1,"wal-append.queue_time_us":152342}
I20260709 13:13:21.334503  8058 raft_consensus.cc:606] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:21.334873  8058 raft_consensus.cc:614] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:21.335000  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:21.335281  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:47112 (request call id 10) took 1222 ms. Trace:
I20260709 13:13:21.263446  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.335672  8058 rpcz_store.cc:276] 0709 13:13:20.113131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.333834 (+1220703us) service_pool.cc:224] Handling call
0709 13:13:21.335199 (+  1365us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:21.244501 10093 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.336926  8058 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:21.337433  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 169) took 1218 ms. Trace:
I20260709 13:13:21.337736  8058 rpcz_store.cc:276] 0709 13:13:20.118985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.336402 (+1217417us) service_pool.cc:224] Handling call
0709 13:13:21.336645 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.337234 (+   589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.337388 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:21.342227  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 170) took 1223 ms. Trace:
I20260709 13:13:21.342720  8058 rpcz_store.cc:276] 0709 13:13:20.119224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.341369 (+1222145us) service_pool.cc:224] Handling call
0709 13:13:21.341672 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.341873 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:21.341911 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:21.341962 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:21.342002 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.342067 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.342179 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:21.344794  8058 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:21.345510  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 171) took 1226 ms. Trace:
I20260709 13:13:21.247537  8130 rpcz_store.cc:276] 0709 13:13:20.037418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.670494 (+633076us) service_pool.cc:224] Handling call
0709 13:13:20.670747 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:20.670928 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:20.671176 (+   248us) log.cc:844] Serialized 14 byte log entry
0709 13:13:20.865642 (+194466us) log.cc:844] Serialized 14 byte log entry
0709 13:13:21.058847 (+193205us) spinlock_profiling.cc:243] Waited 8.87 ms on lock 0x7b180042cbe0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbb00e060 00007f6dbb020716 00007f6dbb031287 00007f6dbb031208
0709 13:13:21.206581 (+147734us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:21.206630 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:21.206667 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.206727 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.206842 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7682,"prepare.queue_time_us":1008,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":144252,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":43424,"spinlock_wait_cycles":24401408,"thread_start_us":333211,"threads_started":3,"wal-append.queue_time_us":300997}
I20260709 13:13:21.248706 10081 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:13:21.288568  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 51257.5us
I20260709 13:13:21.285462  8060 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:21.293092  9952 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:21.308251  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.360096  8057 raft_consensus.cc:2376] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4. Re-sending same reply.
I20260709 13:13:21.360976  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.361836  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.234197  9731 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:21.326537  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:21.363530  8728 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.290978  9865 consensus_queue.cc:237] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.328547  8729 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:13:21.313331 10082 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:13:21.338682  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:21.335131  9914 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:21.338580  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:13:21.356439  9988 raft_consensus.cc:3055] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:21.250094  9964 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:21.340197  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 84657.9us
I20260709 13:13:21.348059  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:21.345988  8058 rpcz_store.cc:276] 0709 13:13:20.119489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.344302 (+1224813us) service_pool.cc:224] Handling call
0709 13:13:21.344598 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.345269 (+   671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.345457 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:21.356542  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:21.309837  8059 raft_consensus.cc:2393] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:21.311465  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:21.235775 10019 consensus_queue.cc:237] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.323596  9986 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:21.357748  8696 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:21.365423  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2021) took 1245 ms. Trace:
I20260709 13:13:21.366577  8135 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:21.337810  9980 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.324575  8131 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
W20260709 13:13:21.328593  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 80803.3us
I20260709 13:13:21.331899  9925 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:21.370591  8133 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:21.308482 10027 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.399199  9998 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.426743  9980 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:21.426920  8138 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:21.427285  9998 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:21.430984  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.431186  9858 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:21.339360  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 168) took 1220 ms. Trace:
I20260709 13:13:21.431816  8061 rpcz_store.cc:276] 0709 13:13:20.118685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.356683 (+237998us) service_pool.cc:224] Handling call
0709 13:13:21.339160 (+982477us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b540025f8d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:21.339299 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255856896}
I20260709 13:13:21.432205  8062 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:21.432621  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1034) took 1056 ms. Trace:
I20260709 13:13:21.432934  8062 rpcz_store.cc:276] 0709 13:13:20.376490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.431753 (+1055263us) service_pool.cc:224] Handling call
0709 13:13:21.431995 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.432478 (+   483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.432589 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:21.433179  8061 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:13:21.433831  8062 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:21.359719 10092 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:21.385447  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:21.373090 10012 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:13:21.385691  8104 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:21.389757  9919 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.390589  8722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 507) took 1038 ms. Trace:
W20260709 13:13:21.393273 10058 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.754s	user 0.002s	sys 0.000s
I20260709 13:13:21.359557  9561 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.383790  9964 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.400665  9972 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.377506  9988 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:21.404392 10052 consensus_meta.cc:287] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.593s	user 0.007s	sys 0.002s
I20260709 13:13:21.375622 10028 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.408248  8057 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:21.411468  8728 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:21.412012  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:21.443733  9747 consensus_meta.cc:287] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.000s
I20260709 13:13:21.369577 10009 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:13:21.420214  8060 rpcz_store.cc:276] 0709 13:13:20.120271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.363422 (+1243151us) service_pool.cc:224] Handling call
0709 13:13:21.363661 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.365127 (+  1466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:21.365156 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:21.365195 (+    39us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:21.365222 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.365277 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.365372 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:21.433805  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1040) took 1056 ms. Trace:
I20260709 13:13:21.359833 10081 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:21.434759  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1041) took 1056 ms. Trace:
I20260709 13:13:21.462586  8062 rpcz_store.cc:276] 0709 13:13:20.393264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.433432 (+1040168us) service_pool.cc:224] Handling call
0709 13:13:21.433688 (+   256us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:21.434596 (+   908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.434725 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:21.462980  8204 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:21.464062  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1042) took 1060 ms. Trace:
I20260709 13:13:21.464392  8062 rpcz_store.cc:276] 0709 13:13:20.403660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.463377 (+1059717us) service_pool.cc:224] Handling call
0709 13:13:21.463645 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.463788 (+   143us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:21.463811 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:21.463854 (+    43us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:21.463881 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.463932 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.464017 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:21.465766  8062 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:13:21.466250 10009 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:21.466485  8063 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:21.466712 10009 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:21.467270  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1044) took 1012 ms. Trace:
I20260709 13:13:21.467641  8063 rpcz_store.cc:276] 0709 13:13:20.454891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.465938 (+1011047us) service_pool.cc:224] Handling call
0709 13:13:21.466228 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:21.466988 (+   760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.467216 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:21.467573 10009 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:21.435724  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 78051us
I20260709 13:13:21.470021  8696 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:21.471122  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 890) took 1601 ms. Trace:
I20260709 13:13:21.383221  9745 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:21.471527  8137 rpcz_store.cc:276] 0709 13:13:19.869445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.335622 (+466177us) service_pool.cc:224] Handling call
0709 13:13:20.336033 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:20.336385 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:20.336413 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:20.339446 (+  3033us) log.cc:844] Serialized 26 byte log entry
0709 13:13:20.404305 (+ 64859us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:20.404332 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:20.404375 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:21.392719 (+988344us) spinlock_profiling.cc:243] Waited 5.75 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:21.470845 (+ 78126us) raft_consensus.cc:1685] finished
0709 13:13:21.470866 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.471066 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1081,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":4221,"spinlock_wait_cycles":13230976,"thread_start_us":40098,"threads_started":1,"wal-append.queue_time_us":221809}
I20260709 13:13:21.472960  9587 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:21.473268  9890 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:21.473481  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:21.437837  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 52211.8us
I20260709 13:13:21.474632  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2028) took 1038 ms. Trace:
I20260709 13:13:21.456806 10052 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.474972  8059 rpcz_store.cc:276] 0709 13:13:20.435725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.543854 (+108129us) service_pool.cc:224] Handling call
0709 13:13:21.474475 (+930621us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b54003ba454. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:21.474585 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1698649856}
I20260709 13:13:21.475399  9919 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:21.475777  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:21.441285  8722 rpcz_store.cc:276] 0709 13:13:20.352136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.353944 (+  1808us) service_pool.cc:224] Handling call
0709 13:13:21.338575 (+984631us) spinlock_profiling.cc:243] Waited 8.4 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1eee 0000000000375939 00007f6dbff651e0 00007f6dc002e0c4 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:21.390528 (+ 51953us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19309952}
I20260709 13:13:21.476425 10052 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:21.441402  9858 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:21.477015  9858 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:21.477022  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:21.477800  9858 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.476545  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:21.438293  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 155339us
I20260709 13:13:21.480531  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:21.484795  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:21.486295  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:21.466259  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2038) took 1027 ms. Trace:
I20260709 13:13:21.488312  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:21.488926  8136 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.488502  8062 rpcz_store.cc:276] 0709 13:13:20.441894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.465231 (+1023337us) service_pool.cc:224] Handling call
0709 13:13:21.465495 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.466052 (+   557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.466213 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:21.491356  8101 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:21.438409  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 149503us
I20260709 13:13:21.430111  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:21.418002 10011 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:21.422591  8054 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.421897 10093 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:21.435681 10092 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:21.440160 10087 raft_consensus.cc:2764] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:13:21.420022  9734 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.456314  8058 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:21.565318  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:21.373584  9952 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:21.457130  8176 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:21.458072  8061 rpcz_store.cc:276] 0709 13:13:20.377852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.432721 (+1054869us) service_pool.cc:224] Handling call
0709 13:13:21.432983 (+   262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:21.433594 (+   611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.433758 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:21.454720  9898 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.459537  9561 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:21.460588  9972 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:21.504889  8101 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:21.468924 10009 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.560431  8136 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:21.480175  9747 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:21.567464  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 96211.5us
W20260709 13:13:21.570392  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 113336us
I20260709 13:13:21.572089  8061 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:13:21.578605  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 73792.5us
I20260709 13:13:21.580789  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:21.584935  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2044) took 1047 ms. Trace:
I20260709 13:13:21.585646  8101 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:21.584925  8136 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.568495  8063 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:21.563689  9968 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:21.585337  8061 rpcz_store.cc:276] 0709 13:13:20.537351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.571352 (+1034001us) service_pool.cc:224] Handling call
0709 13:13:21.571699 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.584636 (+ 12937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.584873 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:21.589159  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.561543 10092 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:21.577292 10105 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:13:21.589689  8061 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:21.507498 10025 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:13:21.594419  8136 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:21.577301 10012 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:21.605141  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:21.606007 10104 log.cc:927] Time spent T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:13:21.582700 10100 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:21.610787 10011 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.638221  8212 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:21.613482  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:21.613786  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.592664  9968 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:21.482764  9988 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.639216  8206 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:21.637323  8132 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:21.640074  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.612843  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.571070 10039 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:13:21.594620 10096 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:21.642764  8208 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:21.643771  8212 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:21.627833  9655 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.648708 10039 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:21.649230 10039 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:21.647431  9952 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.650324  9655 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:21.650038 10039 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:21.650874 10097 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:13:21.639540  9913 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:21.652360  9913 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.652712  9734 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:21.654673  9913 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:21.655623 10102 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:13:21.656050  9964 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.627470 10014 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.651186 10039 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.642808 10028 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:21.640686  9968 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:21.679821  8730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 355) took 1122 ms. Trace:
I20260709 13:13:21.683856  8730 rpcz_store.cc:276] 0709 13:13:20.557206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.557388 (+   182us) service_pool.cc:224] Handling call
0709 13:13:20.577056 (+ 19668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:20.627832 (+ 50776us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:20.627861 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:20.627898 (+    37us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:20.627926 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.679673 (+1051747us) spinlock_profiling.cc:243] Waited 50.6 ms on lock 0x7b540052c454. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:21.679702 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.679781 (+    79us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116312832}
I20260709 13:13:21.685616  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1165) took 1124 ms. Trace:
I20260709 13:13:21.641273  9853 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:21.686085  8211 rpcz_store.cc:276] 0709 13:13:20.561053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.593132 (+ 32079us) service_pool.cc:224] Handling call
0709 13:13:21.679141 (+1086009us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b54001531d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:21.685542 (+  6401us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1291833600}
I20260709 13:13:21.643229 10012 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:21.644582 10100 raft_consensus.cc:697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:21.648205  9988 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:21.657990  9964 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:21.682802  9913 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:21.689121 10028 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.663897  9898 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:21.658052 10029 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.661770  9814 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.791548  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1214) took 1349 ms. Trace:
I20260709 13:13:21.597565  8061 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:13:21.792011  8135 rpcz_store.cc:276] 0709 13:13:20.442084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.615232 (+173148us) service_pool.cc:224] Handling call
0709 13:13:21.791084 (+1175852us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b54003c96d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:21.791320 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1572376832}
I20260709 13:13:21.793045  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2045) took 1247 ms. Trace:
I20260709 13:13:21.793550  8061 rpcz_store.cc:276] 0709 13:13:20.545012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.597038 (+1052026us) service_pool.cc:224] Handling call
0709 13:13:21.597305 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.792366 (+195061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.792826 (+   460us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:21.748667  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.749027  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1048) took 1216 ms. Trace:
I20260709 13:13:21.750962  8058 raft_consensus.cc:2410] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:21.752192  8130 raft_consensus.cc:2393] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:21.757669  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:21.762317  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.763000  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:21.822454  9919 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:21.822799  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 65418.1us
W20260709 13:13:21.825021  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.825065  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:21.766077  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.766357  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.767792  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.828521  8212 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:21.820792  8062 rpcz_store.cc:276] 0709 13:13:20.532725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.537689 (+1004964us) service_pool.cc:224] Handling call
0709 13:13:21.555243 (+ 17554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.555436 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:21.576308 (+ 20872us) log.cc:844] Serialized 14 byte log entry
0709 13:13:21.747737 (+171429us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:21.747904 (+   167us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:21.748118 (+   214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.748308 (+   190us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.748678 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4723,"prepare.queue_time_us":21796,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":4419,"raft.queue_time_us":33340,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":250,"thread_start_us":170537,"threads_started":1,"wal-append.queue_time_us":124179}
I20260709 13:13:21.829551 10028 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:21.768265  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:21.833698  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.835175  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:21.836621  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:21.838711 10085 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:21.840528  9886 consensus_queue.cc:1059] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:21.772199  8026 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:21.841727  9561 raft_consensus.cc:3055] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:13:21.842058  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 70304.8us
I20260709 13:13:21.842176  9561 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:21.843067  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:21.843497  8026 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:21.844151  9886 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:21.844620  9886 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:21.845141  9886 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.775858  9898 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:21.846909  9898 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.848554  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:21.849200  8059 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.849787  9898 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:21.698717  9913 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.852492  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:21.852807  9913 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:21.776880  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:21.853799  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:21.780524  9974 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:21.779722  9853 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:21.854588  9974 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.854964  9853 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.856930  9974 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:21.857761  9853 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:21.782051 10012 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.704716 10029 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:21.859233  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:21.859534 10029 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:21.782516 10014 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:21.860049 10029 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:21.773463  9609 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:21.658180 10092 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.860965  8210 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:21.861135  9609 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.861389 10029 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.711799 10080 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:21.862800 10080 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.863083  8136 raft_consensus.cc:2368] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5.
I20260709 13:13:21.863874  8204 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:21.864444 10092 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:21.865650  8210 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:21.645907 10081 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.787603  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 894) took 1823 ms. Trace:
I20260709 13:13:21.796658  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 217) took 1758 ms. Trace:
I20260709 13:13:21.867503  8137 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:21.867631  8133 rpcz_store.cc:276] 0709 13:13:20.038175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.785159 (+746984us) service_pool.cc:224] Handling call
0709 13:13:20.785502 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.423458 (+637956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.795028 (+371570us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b54000d20d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:21.796122 (+  1094us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1345068288}
I20260709 13:13:21.868731 10081 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:21.869235  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:21.790119  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:21.871282  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:21.871904  8132 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.802913  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.804073  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:21.873479  8134 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:21.799291  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1045) took 1321 ms. Trace:
I20260709 13:13:21.801621  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.874722  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:21.875231  8060 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:21.876000  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:21.874473  8063 rpcz_store.cc:276] 0709 13:13:20.477711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.468612 (+990901us) service_pool.cc:224] Handling call
0709 13:13:21.468921 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:21.594804 (+125883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.798745 (+203941us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b540001ac54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:21.799057 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28576128}
W20260709 13:13:21.813542  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 1.2032e+06us
I20260709 13:13:21.880730  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:21.804737  9814 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:21.763998  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:21.882124 10092 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.793620  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:21.882865  8056 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:21.827049  9919 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:21.883112  8135 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.882982 10092 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.883778  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2043) took 1395 ms. Trace:
I20260709 13:13:21.843631  9561 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.884200  8056 rpcz_store.cc:276] 0709 13:13:20.488830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.667059 (+178229us) service_pool.cc:224] Handling call
0709 13:13:21.763447 (+1096388us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b5800000c98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:21.883695 (+120248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52725120}
I20260709 13:13:21.885419  9561 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.885561  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:21.687414  9890 consensus_queue.cc:1059] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:13:21.886211  8056 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.842372  9887 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:21.887140 10089 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:21.887252  9887 raft_consensus.cc:697] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:21.887713 10089 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:21.695053 10100 consensus_queue.cc:237] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.888784  9887 consensus_queue.cc:237] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.889025 10089 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.853979  8730 raft_consensus.cc:2368] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5.
I20260709 13:13:21.891060 10089 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.891234  8026 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:21.866840  8129 rpcz_store.cc:276] 0709 13:13:19.964096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.416974 (+452878us) service_pool.cc:224] Handling call
0709 13:13:20.417275 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:20.527361 (+110086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:20.532725 (+  5364us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:20.592102 (+ 59377us) log.cc:844] Serialized 26 byte log entry
0709 13:13:20.670893 (+ 78791us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:20.670935 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.786736 (+1115801us) spinlock_profiling.cc:243] Waited 55.5 ms on lock 0x7b540023d2d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:21.786830 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:21.786936 (+   106us) raft_consensus.cc:1685] finished
0709 13:13:21.786956 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:21.787353 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9649,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":7339,"spinlock_wait_cycles":127599360,"thread_start_us":51766,"threads_started":1,"wal-append.queue_time_us":231033}
I20260709 13:13:21.892226  8027 leader_election.cc:400] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5.
I20260709 13:13:21.892340  9952 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:21.892565  8027 leader_election.cc:403] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:13:21.893184  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1159) took 1338 ms. Trace:
I20260709 13:13:21.893549  8207 rpcz_store.cc:276] 0709 13:13:20.555045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.555361 (+   316us) service_pool.cc:224] Handling call
0709 13:13:21.892769 (+1337408us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b540019ebd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:21.893006 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1223879552}
I20260709 13:13:21.894730  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.896592  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:21.897158  8210 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.899621  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:21.901253 10010 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.901680  8133 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:21.837888  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:21.808563  9968 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.908911  8059 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:21.910142  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1207) took 1463 ms. Trace:
I20260709 13:13:21.910656  8133 rpcz_store.cc:276] 0709 13:13:20.439193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.900589 (+1461396us) service_pool.cc:224] Handling call
0709 13:13:21.901297 (+   708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:21.902166 (+   869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.902857 (+   691us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:21.914490  9952 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5.
I20260709 13:13:21.922009  9814 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:21.922976  9814 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.923816  8102 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:21.925141  9964 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:21.932149  8132 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:21.940524  8054 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:21.949313 10092 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:21.951552  9853 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:21.952054  9609 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:21.952368  9853 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.954422 10081 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:21.954826  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.955854  8205 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:21.956267  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:21.957834  9988 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:13:21.958791  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 116782us
I20260709 13:13:21.959981  9919 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:21.961231 10014 raft_consensus.cc:3055] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:21.961584 10014 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:21.961629  9919 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.962323  9919 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.962671  8055 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:21.962754 10014 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.964250 10014 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.964627  9919 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:21.960145  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:21.965893  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 218) took 1607 ms. Trace:
I20260709 13:13:21.967217  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2032) took 1530 ms. Trace:
W20260709 13:13:21.967725  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:21.967607  8054 rpcz_store.cc:276] 0709 13:13:20.436784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.308959 (+872175us) service_pool.cc:224] Handling call
0709 13:13:21.967116 (+658157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:21.968550  8209 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:13:21.967949 10010 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:21.968991  8209 pending_rounds.cc:85] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 1
I20260709 13:13:21.969146 10010 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:21.969707 10010 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:21.970445 10027 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:21.969287  8209 pending_rounds.cc:107] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:13:21.974892  8209 raft_consensus.cc:2889] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:21.969539  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:13:21.978252  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 54647.6us
I20260709 13:13:21.966241  8138 rpcz_store.cc:276] 0709 13:13:20.358004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.802142 (+444138us) service_pool.cc:224] Handling call
0709 13:13:20.802458 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.427264 (+624806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:21.965433 (+538169us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b54003c91d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:21.965723 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435751296}
I20260709 13:13:21.935679  9964 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:13:21.983495  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:21.983886  9814 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:21.975455  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:21.990116  8056 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:21.991394 10010 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:21.992502  8059 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:21.993551  9853 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:22.001230  8205 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:22.002975 10028 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.004078  8102 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:22.005818  8138 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:22.006983  9609 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:22.002158  8135 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:22.007623  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2027) took 1572 ms. Trace:
I20260709 13:13:22.010705  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:22.011272  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:22.011770  8207 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:22.011768  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:22.013276  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:22.014398  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 54819.2us
W20260709 13:13:22.015239  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.015583  9968 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:22.016053  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:22.005839  8101 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:22.019758  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50445.6us
I20260709 13:13:22.020336  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:22.022109 10002 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:22.032088  8210 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:22.043404  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1211) took 1595 ms. Trace:
I20260709 13:13:22.046437  8138 rpcz_store.cc:276] 0709 13:13:20.440133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.981253 (+1541120us) service_pool.cc:224] Handling call
0709 13:13:22.005457 (+ 24204us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.033684 (+ 28227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.035929 (+  2245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.051596  8104 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:22.036306  9814 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
W20260709 13:13:22.055142  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 51199us
I20260709 13:13:22.057185  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:22.055390  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:22.044003  9561 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:22.061712  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:22.046825  8204 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:22.097919  8136 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:22.098963  8026 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:22.099888  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:22.037190  8055 rpcz_store.cc:276] 0709 13:13:20.435456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:20.872874 (+437418us) service_pool.cc:224] Handling call
0709 13:13:22.007421 (+1134547us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b5400334354. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:22.007563 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1112928512}
W20260709 13:13:22.103147 10040 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.003s	sys 0.013s
W20260709 13:13:22.103564 10040 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.004s	sys 0.013s
I20260709 13:13:22.104024  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:22.105217  8055 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:22.043757 10080 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.107949 10080 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:22.109341  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 103755us
I20260709 13:13:22.104272  8204 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:13:22.113459  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50685.4us
I20260709 13:13:22.060437  9988 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:22.118026  8029 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:22.060937  9561 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:22.037456 10028 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.098505  9853 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:22.122211  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:22.115687  8102 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.113166  8138 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:22.063305  9952 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:22.114624 10081 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:22.127185  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:22.126577  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:22.117733  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:22.129415  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2054) took 1097 ms. Trace:
I20260709 13:13:22.136299  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:22.136993  8138 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:22.123680  9561 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:22.139071  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 80069.6us
I20260709 13:13:22.141947  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:22.138674  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:22.136655  8060 rpcz_store.cc:276] 0709 13:13:21.027063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.031407 (+1004344us) service_pool.cc:224] Handling call
0709 13:13:22.111882 (+ 80475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.119855 (+  7973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.119880 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.124044 (+  4164us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:22.124093 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.124160 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.124272 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.152724  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:22.155929  8059 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:22.157935  8060 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:22.157783  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:22.158993  8208 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:13:22.121970  9988 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.148152 10089 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:22.164580  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:22.172251  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:22.170591  8136 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:22.174091  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 76035.9us
I20260709 13:13:22.165081 10010 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:22.164032  9887 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:22.168431  9984 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.006s	sys 0.011s
I20260709 13:13:22.176162 10081 raft_consensus.cc:697] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:22.178720  9974 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.178391  9952 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.178929  7959 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34146 (request call id 256) took 7534 ms. Trace:
I20260709 13:13:22.179137  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:22.179587  9964 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:22.180547  8729 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:22.180850  9876 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:13:22.181563  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 118682us
I20260709 13:13:22.172199  9858 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:22.182070 10089 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:22.187548  9984 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.984s	user 0.007s	sys 0.011s
I20260709 13:13:22.168202  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:22.189600  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:22.169987  9747 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.192828  8029 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.185528  9887 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.176833  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2047) took 1600 ms. Trace:
I20260709 13:13:22.200577  8135 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:22.203011  9952 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:22.205175  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.205430  9890 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.206913  9609 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.207626  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 220) took 1847 ms. Trace:
I20260709 13:13:22.211766  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:22.215939  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2159) took 1070 ms. Trace:
I20260709 13:13:22.211689 10089 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:22.225262  8209 rpcz_store.cc:276] 0709 13:13:21.142179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.156765 (+ 14586us) service_pool.cc:224] Handling call
0709 13:13:21.157114 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.002813 (+845699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.189197 (+186384us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b54002bf554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:22.212196 (+ 22999us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1864796416}
I20260709 13:13:22.230726  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:22.215232  9887 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:22.234766  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:22.216792  9876 raft_consensus.cc:697] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:22.216559  9858 raft_consensus.cc:697] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:22.238909  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 174) took 1548 ms. Trace:
I20260709 13:13:22.239871  8026 leader_election.cc:400] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5.
I20260709 13:13:22.240187  8026 leader_election.cc:403] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:22.240762 10052 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:13:22.241778  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.242437  9905 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:13:22.243304  8060 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:22.243525  8134 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:22.244207  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:22.244580  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 56187.1us
I20260709 13:13:22.244886  8060 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:22.245471  8029 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.230822 10010 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:22.251418  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1221) took 1226 ms. Trace:
I20260709 13:13:22.228139  9609 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.259945  9609 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:22.260259  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 55386.8us
I20260709 13:13:22.261303  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1209) took 1821 ms. Trace:
I20260709 13:13:22.262036  9609 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:22.262661  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:22.263249  9609 raft_consensus.cc:3055] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:22.263528  8132 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:22.263661  9609 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:22.264096  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:22.264657  9609 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.265261  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:22.266103  8058 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:22.264252  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 897) took 1759 ms. Trace:
I20260709 13:13:22.228143  9890 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.229265  9964 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.239312  8057 rpcz_store.cc:276] 0709 13:13:20.611968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.023228 (+1411260us) service_pool.cc:224] Handling call
0709 13:13:22.023575 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.023888 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:22.024169 (+   281us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.151171 (+127002us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.151220 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:22.158381 (+  7161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.158448 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.160377 (+  1929us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15383,"prepare.queue_time_us":680,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":4009,"raft.queue_time_us":38899,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":6000,"thread_start_us":105743,"threads_started":1,"wal-append.queue_time_us":214436}
I20260709 13:13:22.254662  8130 rpcz_store.cc:276] 0709 13:13:21.025334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.347672 (+322338us) service_pool.cc:224] Handling call
0709 13:13:22.233947 (+886275us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b54001aac54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:22.251367 (+ 17420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":779061120}
I20260709 13:13:22.255847  8102 leader_election.cc:400] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4.
I20260709 13:13:22.291646  8137 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:22.290663  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1216) took 1848 ms. Trace:
I20260709 13:13:22.273242  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:22.281121  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:22.306131  8102 leader_election.cc:403] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:13:22.307188  8135 rpcz_store.cc:276] 0709 13:13:20.442586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.250885 (+1808299us) service_pool.cc:224] Handling call
0709 13:13:22.290000 (+ 39115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.290272 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.290306 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.290372 (+    66us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:22.290421 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.290492 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.290602 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.275717  8175 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:22.310830  8136 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:22.288028  8101 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.309557  8138 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:22.306092  8132 rpcz_store.cc:276] 0709 13:13:20.504893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.262849 (+1757956us) service_pool.cc:224] Handling call
0709 13:13:22.263248 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.263985 (+   737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.264172 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.215229  8061 rpcz_store.cc:276] 0709 13:13:20.576680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.993805 (+1417125us) service_pool.cc:224] Handling call
0709 13:13:21.994254 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.994608 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:21.994943 (+   335us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.168586 (+173643us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.168634 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:22.175761 (+  7127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.175894 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.176723 (+   829us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":344,"prepare.queue_time_us":619,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":4465,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":2312,"thread_start_us":166571,"threads_started":2,"wal-append.queue_time_us":162547}
I20260709 13:13:22.253464 10089 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:22.317858  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 62728.4us
I20260709 13:13:22.265938  9609 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:22.316748  8101 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:22.272531 10119 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.089s	user 0.002s	sys 0.000s
I20260709 13:13:22.319254  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 219) took 1958 ms. Trace:
W20260709 13:13:22.325791  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 96046.7us
I20260709 13:13:22.326427  8029 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.321193  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 898) took 1807 ms. Trace:
I20260709 13:13:22.318908  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:22.320248  8059 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:22.266429 10052 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:22.324859  8134 rpcz_store.cc:276] 0709 13:13:20.360374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.877033 (+1516659us) service_pool.cc:224] Handling call
0709 13:13:22.238360 (+361327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.243933 (+  5573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.319065 (+ 75132us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b54001aac50. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:22.319193 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825468672}
I20260709 13:13:22.287940  8722 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 3 LEARNER]: Advancing to term 4
I20260709 13:13:22.261677  8133 rpcz_store.cc:276] 0709 13:13:20.439670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.937923 (+1498253us) service_pool.cc:224] Handling call
0709 13:13:21.987006 (+ 49083us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.012599 (+ 25593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.012629 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:22.013198 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.134838 (+121640us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:22.134893 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.134983 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:22.261068 (+126085us) raft_consensus.cc:1685] finished
0709 13:13:22.261092 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.261209 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":120732,"threads_started":1,"wal-append.queue_time_us":157851}
I20260709 13:13:22.273227  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1210) took 1833 ms. Trace:
I20260709 13:13:22.277076 10028 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:22.286254  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:22.291646  9988 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:22.278910 10010 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:22.331099  8136 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:13:22.249884  9747 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.375761  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:22.376596  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:22.378099  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 901) took 1840 ms. Trace:
I20260709 13:13:22.316258 10014 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:22.334678  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:22.335992  8134 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:22.336546  8062 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:22.336650  8135 raft_consensus.cc:1240] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:22.381898  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 905) took 1830 ms. Trace:
I20260709 13:13:22.382102  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 908) took 1742 ms. Trace:
I20260709 13:13:22.382402  8134 rpcz_store.cc:276] 0709 13:13:20.551252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.328054 (+1776802us) service_pool.cc:224] Handling call
0709 13:13:22.335641 (+  7587us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.381550 (+ 45909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.381832 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.382670  8135 rpcz_store.cc:276] 0709 13:13:20.639329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.336156 (+1696827us) service_pool.cc:224] Handling call
0709 13:13:22.336488 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.381808 (+ 45320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.382039 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.192432  7959 rpcz_store.cc:276] 0709 13:13:14.644388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.644551 (+   163us) service_pool.cc:224] Handling call
0709 13:13:22.178705 (+7534154us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:19.955938 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:19.956521 (+   583us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:19.956562 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:13:19.956583 (+    21us) tablet.cc:662] Decoding operations
0709 13:13:19.960262 (+  3679us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:19.960326 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:19.960357 (+    31us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:13:20.122032 (+161675us) tablet.cc:701] Row locks acquired
0709 13:13:20.122057 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:13:20.122209 (+   152us) write_op.cc:270] Start()
0709 13:13:20.122320 (+   111us) write_op.cc:276] Timestamp: P: 1783602800122168 usec, L: 0
0709 13:13:20.122357 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:13:20.123154 (+   797us) log.cc:844] Serialized 4869 byte log entry
0709 13:13:20.804336 (+681182us) op_driver.cc:464] REPLICATION: finished
0709 13:13:21.008223 (+203887us) write_op.cc:301] APPLY: starting
0709 13:13:21.008307 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:21.008610 (+   303us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:21.009851 (+  1241us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:21.012456 (+  2605us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:21.012688 (+   232us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:13:21.012730 (+    42us) write_op.cc:312] APPLY: finished
0709 13:13:21.013000 (+   270us) log.cc:844] Serialized 89 byte log entry
0709 13:13:21.013152 (+   152us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:21.013278 (+   126us) write_op.cc:454] Released schema lock
0709 13:13:21.013323 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:21.013658 (+   335us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:21.013683 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2958,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":203650,"apply.run_cpu_time_us":3336,"apply.run_wall_time_us":5416,"num_ops":12,"prepare.queue_time_us":200992,"prepare.run_cpu_time_us":26158,"prepare.run_wall_time_us":390206,"raft.queue_time_us":160772,"raft.run_cpu_time_us":11812,"raft.run_wall_time_us":122040,"replication_time_us":652750,"thread_start_us":473987,"threads_started":4,"wal-append.queue_time_us":279078,"wal-append.run_cpu_time_us":17158,"wal-append.run_wall_time_us":1608909}]]}
I20260709 13:13:22.337296  8101 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.338030 10080 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:13:22.338294  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 111568us
I20260709 13:13:22.338748  9655 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4.
I20260709 13:13:22.340966  7956 catalog_manager.cc:5697] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 3 to 4, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 4 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
W20260709 13:13:22.343533 10133 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:13:22.344301  8132 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:22.380976  8057 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:22.317725 10100 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.346596  9984 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:13:22.346922  9888 raft_consensus.cc:3055] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:22.342170  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 176) took 1251 ms. Trace:
I20260709 13:13:22.347559  8203 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:22.348539  9890 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:22.348677  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:22.266950  9858 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.225221  8129 rpcz_store.cc:276] 0709 13:13:20.360701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.892306 (+1531605us) service_pool.cc:224] Handling call
0709 13:13:21.892714 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:21.893027 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:22.050219 (+157192us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.193145 (+142926us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.194178 (+  1033us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:22.207333 (+ 13155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.207415 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.207538 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34468,"prepare.queue_time_us":159387,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":36952,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":18072,"thread_start_us":250922,"threads_started":2,"wal-append.queue_time_us":234174}
I20260709 13:13:22.244251 10081 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.267007  9876 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.364288  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:22.365271  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:22.365757  9974 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:22.368237 10039 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:22.371025  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:22.372282  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:22.358450 10009 consensus_queue.cc:1059] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:13:22.373375  8063 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:22.373886 10121 log.cc:927] Time spent T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:13:22.375123  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:22.332206  8131 rpcz_store.cc:276] 0709 13:13:20.439902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:21.971099 (+1531197us) service_pool.cc:224] Handling call
0709 13:13:21.971639 (+   540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:21.972216 (+   577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:21.972261 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:21.972882 (+   621us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.135819 (+162937us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:22.138584 (+  2765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.145696 (+  7112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:22.272899 (+127203us) raft_consensus.cc:1685] finished
0709 13:13:22.272933 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.273068 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17280,"raft.queue_time_us":5845,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":12510,"thread_start_us":148716,"threads_started":1,"wal-append.queue_time_us":293802}
I20260709 13:13:22.357748 10002 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.271602  8726 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:22.280731  9886 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.327456  8137 rpcz_store.cc:276] 0709 13:13:20.505926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.290907 (+1784981us) service_pool.cc:224] Handling call
0709 13:13:22.291373 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.307023 (+ 15650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.313342 (+  6319us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.380419  8136 rpcz_store.cc:276] 0709 13:13:20.537826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.314311 (+1776485us) service_pool.cc:224] Handling call
0709 13:13:22.317957 (+  3646us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.376949 (+ 58992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.377989 (+  1040us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.397750  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:22.398391  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 61191.8us
I20260709 13:13:22.513276  8060 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:22.513454  8101 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.400346  8133 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:22.400849  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:22.397713  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 71389.6us
I20260709 13:13:22.402031  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:22.402366  8207 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:22.404503  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 903) took 1854 ms. Trace:
I20260709 13:13:22.515863  8132 rpcz_store.cc:276] 0709 13:13:20.545701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.320798 (+1775097us) service_pool.cc:224] Handling call
0709 13:13:22.344089 (+ 23291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.399143 (+ 55054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.400037 (+   894us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.414296  8026 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:22.418262  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:22.418274  7956 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:22.387219 10014 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:22.517432 10014 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:22.421941  9887 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.422762  8134 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:13:22.422744  8061 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:22.522089  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 907) took 1956 ms. Trace:
I20260709 13:13:22.522537  8133 rpcz_store.cc:276] 0709 13:13:20.558429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.330744 (+1772315us) service_pool.cc:224] Handling call
0709 13:13:22.400165 (+ 69421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.514130 (+113965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.514310 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.422973  9946 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:22.422843  8130 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:22.423758  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2046) took 1854 ms. Trace:
I20260709 13:13:22.411381  8056 rpcz_store.cc:276] 0709 13:13:21.086602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.033191 (+946589us) service_pool.cc:224] Handling call
0709 13:13:22.110589 (+ 77398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.235585 (+124996us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.236563 (+   978us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.253393 (+ 16830us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:22.297434 (+ 44041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.338063 (+ 40629us) spinlock_profiling.cc:243] Waited 99.5 ms on lock 0x7b540028aad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:22.338120 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.338275 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228877568}
I20260709 13:13:22.543304  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:22.542495  8054 rpcz_store.cc:276] 0709 13:13:20.569556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.266300 (+1696744us) service_pool.cc:224] Handling call
0709 13:13:22.266726 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.267130 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:22.267607 (+   477us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.414091 (+146484us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.423447 (+  9356us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:22.423507 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.423573 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.423699 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10051,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":14351,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"thread_start_us":119268,"threads_started":1,"wal-append.queue_time_us":242925}
I20260709 13:13:22.543898  8056 raft_consensus.cc:2376] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3. Re-sending same reply.
I20260709 13:13:22.544720  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:22.544895  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:22.545382  8054 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:22.545789  8056 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:22.428045 10010 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.542508  8176 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:22.546730  9974 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:22.547196 10085 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:22.547709 10085 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:22.548087  8101 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.548616 10085 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.549361  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.550758 10096 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:22.466117 10080 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:13:22.551302  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 173549us
I20260709 13:13:22.468449  9964 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:22.469228  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:22.553406  8062 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:22.553503  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:22.471925  9561 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.554610 10002 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:22.555172 10002 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:22.555454  9561 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:22.475608  9898 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.463352 10040 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:22.556456 10040 consensus_queue.cc:1243] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:22.556285  9898 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.476023  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 222) took 1830 ms. Trace:
W20260709 13:13:22.500753  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 135556us
I20260709 13:13:22.537606  7956 catalog_manager.cc:5697] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 2 to 3, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 3 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:22.557132  8135 rpcz_store.cc:276] 0709 13:13:20.645346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.383924 (+1738578us) service_pool.cc:224] Handling call
0709 13:13:22.403100 (+ 19176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.475717 (+ 72617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.475756 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.475796 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:22.475821 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.475868 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.475960 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.477586  8205 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:13:22.476898  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.517995  8212 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:22.503621  9655 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:22.529525  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:22.535718  9887 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:22.538223  8721 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:13:22.577107  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 899) took 2070 ms. Trace:
I20260709 13:13:22.550833  9814 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:22.556059 10002 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.404356  9888 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:22.440110  9747 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:22.522567  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 221) took 1877 ms. Trace:
I20260709 13:13:22.493981  8057 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:22.583690  8101 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.561060  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:22.584698  8134 rpcz_store.cc:276] 0709 13:13:20.645126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.383637 (+1738511us) service_pool.cc:224] Handling call
0709 13:13:22.422523 (+ 38886us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.522222 (+ 99699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.522500 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.561053 10096 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:22.562186  9898 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:22.556079  9988 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:22.588901  9988 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:22.589432  8133 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:22.590044  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1230) took 1486 ms. Trace:
I20260709 13:13:22.589948  9988 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.590483  8133 rpcz_store.cc:276] 0709 13:13:21.104072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.584395 (+1480323us) service_pool.cc:224] Handling call
0709 13:13:22.589091 (+  4696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.589787 (+   696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.589977 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.591297  9988 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:22.591849  8133 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:22.593263  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:22.593946  8057 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:22.588778  9980 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.594627  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1055) took 1150 ms. Trace:
I20260709 13:13:22.594709  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:22.595067  8057 rpcz_store.cc:276] 0709 13:13:21.444553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.585011 (+1140458us) service_pool.cc:224] Handling call
0709 13:13:22.593663 (+  8652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.594362 (+   699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.594549 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.595772  8134 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:36038
I20260709 13:13:22.596290  9776 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.596464  8134 raft_consensus.cc:606] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Received request to transfer leadership
I20260709 13:13:22.596854  8134 raft_consensus.cc:614] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:22.597268  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:36038 (request call id 82) took 1563 ms. Trace:
I20260709 13:13:22.597400  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:22.597618  8134 rpcz_store.cc:276] 0709 13:13:21.034214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.595595 (+1561381us) service_pool.cc:224] Handling call
0709 13:13:22.597205 (+  1610us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:22.563174  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 150108us
I20260709 13:13:22.598278  8726 raft_consensus.cc:2393] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:22.598803  8102 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:22.599668  8729 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:22.593304  8133 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:13:22.602044  7956 catalog_manager.cc:5697] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 2 to 3, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 3 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:22.603070  9888 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.604317  9888 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:22.563537  8029 leader_election.cc:400] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5.
I20260709 13:13:22.594094  8206 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:13:22.563850  8056 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:13:22.567353 10100 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:22.567963  9853 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:22.565956  9968 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:13:22.558804  8209 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:22.571111  8211 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:22.572333  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 904) took 2026 ms. Trace:
I20260709 13:13:22.575445  8061 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:22.545698  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:22.614959  8130 rpcz_store.cc:276] 0709 13:13:20.545938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.324579 (+1778641us) service_pool.cc:224] Handling call
0709 13:13:22.339045 (+ 14466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.526475 (+187430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.572141 (+ 45666us) spinlock_profiling.cc:243] Waited 66.1 ms on lock 0x7b54004092d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:22.572282 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151960448}
I20260709 13:13:22.578136  8138 rpcz_store.cc:276] 0709 13:13:20.506257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.313290 (+1807033us) service_pool.cc:224] Handling call
0709 13:13:22.344692 (+ 31402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.345138 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:22.345856 (+   718us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.573998 (+228142us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.574041 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:22.576881 (+  2840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.576957 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.577061 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":338,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":25751,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"thread_start_us":156932,"threads_started":1}
I20260709 13:13:22.518528 10028 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.594024  8208 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:22.558204  8205 pending_rounds.cc:85] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 1
W20260709 13:13:22.562714  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 231413us
I20260709 13:13:22.564059  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:22.573210  9655 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:22.578027 10040 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:22.620918  8205 pending_rounds.cc:107] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:13:22.621331  8205 raft_consensus.cc:2889] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:22.629102  8060 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:13:22.629941  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1054) took 1206 ms. Trace:
W20260709 13:13:22.630681  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 67341.5us
I20260709 13:13:22.632015  8137 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:22.632077  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1052) took 1258 ms. Trace:
I20260709 13:13:22.633059  8056 rpcz_store.cc:276] 0709 13:13:21.373858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.563248 (+1189390us) service_pool.cc:224] Handling call
0709 13:13:22.563625 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.614087 (+ 50462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.632005 (+ 17918us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.633745 10028 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.634605  8056 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:22.635264  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1060) took 1031 ms. Trace:
I20260709 13:13:22.635553  8056 rpcz_store.cc:276] 0709 13:13:21.604248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.633935 (+1029687us) service_pool.cc:224] Handling call
0709 13:13:22.634322 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.635008 (+   686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.635195 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.636705 10052 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:22.636924  8723 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:22.653033  8029 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.621739  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 177) took 1528 ms. Trace:
I20260709 13:13:22.655342  8062 rpcz_store.cc:276] 0709 13:13:21.086881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.574965 (+1488084us) service_pool.cc:224] Handling call
0709 13:13:22.607752 (+ 32787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.614856 (+  7104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.614880 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.614917 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:22.614962 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.615015 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.615106 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.622604  9655 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:22.621002 10040 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:22.627701  9890 raft_consensus.cc:3055] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:22.636935  8056 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:13:22.659564  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 60876.9us
I20260709 13:13:22.630409  8060 rpcz_store.cc:276] 0709 13:13:21.423632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.580223 (+1156591us) service_pool.cc:224] Handling call
0709 13:13:22.628830 (+ 48607us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.629646 (+   816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.629883 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.666306  8696 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:22.666723  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:22.667084  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 92376.9us
I20260709 13:13:22.668438  9776 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:22.668972  9776 raft_consensus.cc:697] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:22.669607  9609 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:22.670017  9609 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.670457  9609 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:22.670106  9776 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:22.670758  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 93133.4us
I20260709 13:13:22.670948  9886 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:22.671609  9886 consensus_queue.cc:1059] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:22.672704  9917 raft_consensus.cc:3055] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:22.673041  9917 raft_consensus.cc:740] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:22.673862  9917 consensus_queue.cc:260] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.674865  9917 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:22.621713  9898 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:13:22.668916  8696 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:22.679240 10039 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:22.679610 10039 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.679641  9814 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.680085 10039 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:22.680501  8054 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:22.681452 10028 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:22.681699  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1061) took 1076 ms. Trace:
I20260709 13:13:22.682083  8056 rpcz_store.cc:276] 0709 13:13:21.605603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.636294 (+1030691us) service_pool.cc:224] Handling call
0709 13:13:22.636677 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.681419 (+ 44742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.681644 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:22.682587  8132 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:22.683552  8056 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:22.684136  9898 consensus_queue.cc:1059] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:22.684926  8056 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:22.684993  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:22.685302  9905 consensus_queue.cc:1059] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:22.685638  8725 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:22.686306 10027 raft_consensus.cc:3055] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:22.656656 10080 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:22.686700 10027 raft_consensus.cc:740] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:22.686959  8176 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:22.687844 10012 raft_consensus.cc:3055] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:22.687538 10027 consensus_queue.cc:260] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.686182  8056 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
W20260709 13:13:22.691244  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45703.1us
I20260709 13:13:22.688203 10012 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:13:22.695927  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.696080  8138 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:22.697675  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 913) took 1646 ms. Trace:
I20260709 13:13:22.698402  8132 rpcz_store.cc:276] 0709 13:13:21.051114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.578579 (+1527465us) service_pool.cc:224] Handling call
0709 13:13:22.586003 (+  7424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.682982 (+ 96979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.697438 (+ 14456us) spinlock_profiling.cc:243] Waited 96.4 ms on lock 0x7b54001f09d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:22.697621 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":221759872}
W20260709 13:13:22.703727  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 63319.9us
I20260709 13:13:22.664430  9890 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:22.693092 10027 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:22.686996 10080 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:22.696856 10052 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.695423  9561 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:22.724184 10096 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:22.725361  9814 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:22.726307  9853 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:22.700134 10012 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.727573 10080 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.729288  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 224) took 1651 ms. Trace:
W20260709 13:13:22.784958  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 63900.3us
I20260709 13:13:22.730118  8056 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:22.685119  9898 consensus_queue.cc:1059] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:13:22.731211  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.785223  8136 rpcz_store.cc:276] 0709 13:13:21.077451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.575165 (+1497714us) service_pool.cc:224] Handling call
0709 13:13:22.596757 (+ 21592us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.597012 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:22.597269 (+   257us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.728993 (+131724us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:22.729047 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:22.729088 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.729149 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.729242 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16177,"prepare.queue_time_us":597,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":16290,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":3197,"thread_start_us":115066,"threads_started":1}
I20260709 13:13:22.729246 10081 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.793471  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.797497  8102 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.798885  8102 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.733333  9561 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.822568 10100 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.823356 10100 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:13:22.824751  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.832279  9845 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.832535  9776 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:22.833470  9776 raft_consensus.cc:3055] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:22.833770  9776 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:22.834610  9776 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.835726  9776 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:22.800181  9853 raft_consensus.cc:3055] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:22.801062 10089 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.808768  9561 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:22.839010  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 120578us
I20260709 13:13:22.808598  9952 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.826545 10100 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:13:22.868245  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 284568us
I20260709 13:13:22.801606 10014 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.901403  8212 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:22.902635 10012 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:22.902745  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.903879  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.905802 10029 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:22.907388  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.908142 10029 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:22.908922  9952 raft_consensus.cc:3055] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:13:22.874099  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 75305.7us
I20260709 13:13:22.843425  9853 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:22.875257  8101 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:22.866536  9845 consensus_queue.cc:1059] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:13:22.860424  9747 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:22.881165  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1059) took 1284 ms. Trace:
W20260709 13:13:22.881294  8722 consensus_meta.cc:287] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.503s	user 0.005s	sys 0.005s
I20260709 13:13:22.882936 10010 consensus_queue.cc:1059] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:22.884111  9917 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:22.884601  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:22.886243  8132 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:22.887117  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:22.888736  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:22.830293  9890 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.827962 10040 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.889367  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:22.890802  8208 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:22.827486 10081 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.861595 10089 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:22.895668 10153 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:13:22.899045  8063 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:22.899261  9984 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5.
I20260709 13:13:22.899927  9974 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.813277  9609 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.918082  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:22.923451  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.924427  8102 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:22.909286  9952 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:22.948853  8061 rpcz_store.cc:276] 0709 13:13:21.596512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.615411 (+1018899us) service_pool.cc:224] Handling call
0709 13:13:22.632760 (+ 17349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.633136 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.633166 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:22.633651 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.659511 (+ 25860us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:22.659546 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.659610 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:22.880952 (+221342us) raft_consensus.cc:1685] finished
0709 13:13:22.880973 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:22.881111 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1094,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":25517,"threads_started":1,"wal-append.queue_time_us":155499}
I20260709 13:13:22.986938  8061 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
W20260709 13:13:22.952164  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44889.3us
I20260709 13:13:22.929066  9887 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:22.965006  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 89836.9us
W20260709 13:13:22.976011  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:22.976219 10010 raft_consensus.cc:3055] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:22.995824 10010 raft_consensus.cc:740] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:22.996354  8061 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:13:22.996975 10010 consensus_queue.cc:260] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:22.961494 10014 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:22.999282  8131 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:23.000093  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:23.000095  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:23.024853  8101 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:23.029791 10134 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:22.993853 10002 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.957646  9747 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:22.977598 10039 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:23.039623  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 115402us
I20260709 13:13:23.041141  8133 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:23.012903  8722 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 4 LEARNER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:13:23.016975 10040 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.048667  8056 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:13:23.030880 10009 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.012676  9858 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:13:23.062031 10147 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:23.055413  9952 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.063345  9858 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:23.063951  9952 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:23.065078  8134 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:23.047654 10080 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:23.066993 10080 raft_consensus.cc:697] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:23.068008 10080 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.066026  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:23.058027 10157 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:23.026751  9853 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.049759 10029 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:23.072620  9853 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.075205 10089 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:23.080951 10100 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:23.087881  8135 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:23.039083  9887 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:23.031961  9974 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:23.021678  9814 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.029s
W20260709 13:13:23.091856  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 61504.3us
I20260709 13:13:23.030499  9968 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 13:13:23.103199  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 66169.3us
I20260709 13:13:23.053932 10010 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:23.118878  8101 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:23.121865  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 178) took 1873 ms. Trace:
I20260709 13:13:23.043092 10002 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.130110  8059 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
W20260709 13:13:23.132175  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:23.134017  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 44958.8us
I20260709 13:13:23.130506  9974 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:23.143733  9974 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:23.145843  9952 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.146425  9952 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:23.147778  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 52803.5us
I20260709 13:13:23.148344  9952 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:23.149461 10148 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:13:23.123586  8058 rpcz_store.cc:276] 0709 13:13:21.246509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.560680 (+1314171us) service_pool.cc:224] Handling call
0709 13:13:22.619611 (+ 58931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.654508 (+ 34897us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:22.693679 (+ 39171us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.115516 (+421837us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.119570 (+  4054us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:23.119616 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.119694 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.119830 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10665,"prepare.queue_time_us":34406,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":5518,"thread_start_us":255958,"threads_started":1,"wal-append.queue_time_us":341517}
I20260709 13:13:23.151916  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:23.151180 10158 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:13:23.151612  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:23.161324  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 252199us
I20260709 13:13:23.091375 10009 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.164680  8058 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:23.127976  9609 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:23.167945  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:23.174857  8134 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:23.132711  9845 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:23.177565 10151 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:23.179152 10039 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:23.179781 10175 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:23.180130 10175 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:23.180927 10175 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:23.181268  9655 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.182919  9845 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:23.185892  9561 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:23.187951  9561 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:23.189684 10009 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:23.191648  8211 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:13:23.167701  9952 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:13:23.190585  9845 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:23.193646 10170 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:23.194190  9747 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:23.182142 10175 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.196245 10002 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:23.193647  9952 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:23.197465  9952 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.198495  9968 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:13:23.199044 10179 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:13:23.199494  9952 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:23.200244  9845 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:23.201769  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 64861us
I20260709 13:13:23.202059  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2068) took 1073 ms. Trace:
I20260709 13:13:23.202095  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:23.208840  8062 rpcz_store.cc:276] 0709 13:13:22.134854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.693688 (+558834us) service_pool.cc:224] Handling call
0709 13:13:22.711783 (+ 18095us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.735306 (+ 23523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.735340 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:22.803960 (+ 68620us) log.cc:844] Serialized 26 byte log entry
0709 13:13:23.049556 (+245596us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:23.049608 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.049857 (+   249us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.201870 (+152013us) raft_consensus.cc:1685] finished
0709 13:13:23.201887 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.202007 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1640,"raft.queue_time_us":8856,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":7714,"thread_start_us":211292,"threads_started":1,"wal-append.queue_time_us":294648}
I20260709 13:13:23.208076  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:23.214030  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:23.214623  8133 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
W20260709 13:13:23.206807 10162 log.cc:927] Time spent T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:13:23.220031  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:23.220570  8058 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:23.199038 10151 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:23.223542  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:23.227768  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:23.218663  9952 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:13:23.221630 10009 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:13:23.231187  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:23.231914  8134 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:23.233021  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:23.233870  8134 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:23.215909  9609 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:23.234954  9609 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:23.232807  9747 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:23.242731  9747 consensus_queue.cc:1243] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 1.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:13:23.247890 10151 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:23.251447  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:23.260147  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.261094  8063 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:23.262295  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:23.265143  8063 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:23.265693  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:23.238493  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:23.236058  9609 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:23.268088  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:23.270016  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 48056.8us
I20260709 13:13:23.273334  9876 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:13:23.274400  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 51313.1us
I20260709 13:13:23.282860  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:23.287631  8058 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:23.222524  9561 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:23.289793  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:23.293352  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 131036us
I20260709 13:13:23.294135  8693 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:23.298691  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2069) took 1144 ms. Trace:
I20260709 13:13:23.303864  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:23.305848  8134 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:23.308449  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:23.309013  8210 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.312976  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:23.299057  8060 rpcz_store.cc:276] 0709 13:13:22.155481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.690612 (+535131us) service_pool.cc:224] Handling call
0709 13:13:22.696083 (+  5471us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:22.734930 (+ 38847us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.734958 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:22.790834 (+ 55876us) log.cc:844] Serialized 52 byte log entry
0709 13:13:22.900097 (+109263us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b48000801d8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:13:22.985848 (+ 85751us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:23.046829 (+ 60981us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.119673 (+ 72844us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.224217 (+104544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.224391 (+   174us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.298478 (+ 74087us) raft_consensus.cc:1685] finished
0709 13:13:23.298502 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.298632 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68497,"prepare.queue_time_us":61754,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":3561,"raft.queue_time_us":141257,"raft.run_cpu_time_us":2641,"raft.run_wall_time_us":162372,"spinlock_wait_cycles":5710720,"thread_start_us":252419,"threads_started":2,"wal-append.queue_time_us":258293}
I20260709 13:13:23.314059  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:23.310962  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:23.315789  8209 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:23.319128  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:23.370368  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:23.373612  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.319658  8058 raft_consensus.cc:2376] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4. Re-sending same reply.
W20260709 13:13:23.394869  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 68902.5us
I20260709 13:13:23.303650 10010 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.305564 10012 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.401487  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:23.408856  8060 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:23.425354  8210 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:23.427706  8696 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:23.417835 10153 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:13:23.441087  8058 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:13:23.447924  8136 pending_rounds.cc:85] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 2
I20260709 13:13:23.411478 10010 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:23.413403 10012 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.455291  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:23.466180  9946 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:23.447181  9655 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:23.450548 10173 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:23.453002  9988 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.453464  8136 pending_rounds.cc:107] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 13:13:23.455569 10039 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:23.482693  8136 raft_consensus.cc:2917] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 13:13:23.484191 10177 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.484911 10177 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:23.487735  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 88642.3us
I20260709 13:13:23.467890  9561 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:23.469297  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1229) took 2365 ms. Trace:
I20260709 13:13:23.486987 10180 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:23.527829  8136 raft_consensus.cc:2889] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:23.509176 10002 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.487254 10173 raft_consensus.cc:697] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:23.509852 10039 raft_consensus.cc:697] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:23.512714  9561 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:23.530591  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 102997us
I20260709 13:13:23.482318  9988 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.547653  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:23.527334 10187 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:23.555521  8729 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:13:23.559947 10184 log.cc:927] Time spent T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:13:23.562872 10039 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:23.566740  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:23.567915  9988 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:23.535964  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:23.568953  8722 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:23.567848  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:23.570163 10010 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:23.531160 10180 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.532212 10002 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:23.572582 10180 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:23.573053  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 181) took 1287 ms. Trace:
I20260709 13:13:23.573455  8056 rpcz_store.cc:276] 0709 13:13:22.285879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.804173 (+518294us) service_pool.cc:224] Handling call
0709 13:13:22.977937 (+173764us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:23.049086 (+ 71149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.572166 (+523080us) spinlock_profiling.cc:243] Waited 70.5 ms on lock 0x7b540016a654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:23.572868 (+   702us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b540016a650. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:23.572991 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502779136}
I20260709 13:13:23.574905  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 246) took 1004 ms. Trace:
W20260709 13:13:23.542383 10176 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 13:13:23.563618 10177 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:23.534085  9561 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:23.575296  8212 rpcz_store.cc:276] 0709 13:13:22.570691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.577947 (+  7256us) service_pool.cc:224] Handling call
0709 13:13:22.606398 (+ 28451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.901763 (+295365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.574681 (+672918us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b540023f5d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:23.574847 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":678005504}
I20260709 13:13:23.588999  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:23.593439  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 372) took 1118 ms. Trace:
I20260709 13:13:23.593910 10010 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:23.606503  8722 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
W20260709 13:13:23.607437 10191 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:13:23.608201 10173 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:23.610580 10012 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:23.612514  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:23.612540  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 64970.9us
I20260709 13:13:23.615108  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:23.615999  8056 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:23.617136  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 184) took 1297 ms. Trace:
I20260709 13:13:23.614737 10010 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:23.620873 10010 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.623034 10027 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:23.617511  8054 rpcz_store.cc:276] 0709 13:13:22.319812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.900860 (+581048us) service_pool.cc:224] Handling call
0709 13:13:22.901148 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.103091 (+201943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.119855 (+ 16764us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.121930 (+  2075us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:23.123957 (+  2027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.616914 (+492957us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b540009bad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:23.616956 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.617077 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436522752}
I20260709 13:13:23.625300  9561 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.625833  8062 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:23.597971  8727 rpcz_store.cc:276] 0709 13:13:22.475481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.475707 (+   226us) service_pool.cc:224] Handling call
0709 13:13:22.476013 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.476239 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:22.581980 (+105741us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.592828 (+1010848us) spinlock_profiling.cc:243] Waited 36.8 ms on lock 0x7b18001d3ce0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb623a
0709 13:13:23.593022 (+   194us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.593080 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:23.593145 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.593225 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.593349 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":229362,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":4718,"spinlock_wait_cycles":84553088,"thread_start_us":201845,"threads_started":2,"wal-append.queue_time_us":343936}
I20260709 13:13:23.575090 10187 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.634598  9561 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:23.635182  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 205902us
I20260709 13:13:23.637331  8026 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:23.641249  8132 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:13:23.628505 10189 consensus_queue.cc:1059] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:13:23.631623 10012 consensus_queue.cc:1059] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:13:23.644670  8056 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:23.644972  8026 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:23.646062  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:23.646811  8062 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:23.647461  9747 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:23.648218  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:23.614382 10200 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:13:23.651515  9655 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:23.654099  9747 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.655426 10009 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:23.513010  8129 rpcz_store.cc:276] 0709 13:13:21.103792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.579394 (+1475602us) service_pool.cc:224] Handling call
0709 13:13:22.632623 (+ 53229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:22.632920 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:22.633194 (+   274us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.726749 (+ 93555us) log.cc:844] Serialized 14 byte log entry
0709 13:13:22.897026 (+170277us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b180035d460. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:13:23.289682 (+392656us) spinlock_profiling.cc:243] Waited 7.17 ms on lock 0x7b180035d460. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ea3c0a 00007f6dc15e3219 00007f6dbabccd70 00007f6dbabc0293 00007f6dbabb5cea 00007f6dbabb623a 00007f6dbb020a2a 00007f6dbb0300bf 00007f6dbb030038
0709 13:13:23.465843 (+176161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:23.465917 (+    74us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:23.465993 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.469066 (+  3073us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.469220 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":325094,"prepare.queue_time_us":8698,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":154102,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":23000,"spinlock_wait_cycles":22774400,"thread_start_us":180749,"threads_started":2,"wal-append.queue_time_us":81655}
I20260709 13:13:23.658404 10180 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:23.660315  8725 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:23.660946  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:23.650818 10189 raft_consensus.cc:3055] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:23.663970  9964 raft_consensus.cc:3055] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:23.665504  9561 raft_consensus.cc:3055] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:23.675189  8062 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:23.685767  9561 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:23.672061  9747 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.684577  8054 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:23.675161 10180 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.693550  9561 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.696911  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:23.692343  9886 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:23.682298 10189 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:23.669065 10134 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:13:23.703596 10189 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:23.704758 10189 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.705279 10187 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:23.685341  9964 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:23.707098  9964 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.708389  9964 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:23.698616 10180 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:23.715386  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 94328.6us
I20260709 13:13:23.718454  8062 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:13:23.699337  9886 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:23.724774  8062 raft_consensus.cc:1240] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:23.726969  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.726840  8062 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:13:23.733649  8209 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:23.738154 10207 consensus_queue.cc:1059] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:23.742067  9747 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:23.745134 10160 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.650s	user 0.000s	sys 0.004s
I20260709 13:13:23.747341 10002 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:23.748184  8723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 375) took 1070 ms. Trace:
I20260709 13:13:23.748579  8723 rpcz_store.cc:276] 0709 13:13:22.677711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.677826 (+   115us) service_pool.cc:224] Handling call
0709 13:13:22.678122 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:22.678509 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:22.678540 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:22.678972 (+   432us) log.cc:844] Serialized 26 byte log entry
0709 13:13:22.829472 (+150500us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:22.842048 (+ 12576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:22.849894 (+  7846us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:23.744646 (+894752us) spinlock_profiling.cc:243] Waited 4.69 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:23.747874 (+  3228us) raft_consensus.cc:1685] finished
0709 13:13:23.747907 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:23.748120 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":617,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":82,"spinlock_wait_cycles":10791680,"thread_start_us":124728,"threads_started":1,"wal-append.queue_time_us":354626}
W20260709 13:13:23.751693 10182 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.314s	user 0.002s	sys 0.000s
I20260709 13:13:23.753031  9747 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:23.754137 10081 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:23.763971  8722 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:23.722867 10027 consensus_queue.cc:1059] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:13:23.775672 10200 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:23.778604  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:23.779610  8054 raft_consensus.cc:2393] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:23.819389  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:23.845952  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:23.874775 10207 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:13:23.884262  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 147606us
I20260709 13:13:23.885594  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:23.870602 10200 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:23.864130 10027 consensus_queue.cc:1243] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:13:23.891490 10027 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:23.897389  9886 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.881923 10180 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.874257  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:23.900694  8723 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 4 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:23.886787 10200 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:23.889612  8101 leader_election.cc:336] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 2.045s (SENT)
I20260709 13:13:23.903404  8101 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:23.887331 10217 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.926975  8212 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:13:23.934834  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 47150.9us
I20260709 13:13:23.937245  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:23.939402  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:23.931910  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:23.904194 10180 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:23.944940  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:23.945690  8129 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.946600 10200 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:23.924793  9964 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:23.950786  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:23.952322  8207 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:23.953595  8207 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.961161  8207 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:23.970988  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:23.973917  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:23.975988  9913 raft_consensus.cc:3055] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:23.976397  9913 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:23.977855  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:23.979667  9913 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:23.980656  9913 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.980896  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:23.981464  8137 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:23.982638  8132 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:13:23.983693  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:23.984247  8132 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:23.985445  8693 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:23.986766  9890 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:23.987965  8137 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:23.988862  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:23.989348  8137 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:23.990705  8102 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:23.992194 10189 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:23.992594 10189 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.992921 10189 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:23.996296  9734 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:23.996955  9734 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:24.001518  8061 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:24.002490  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:24.004475  8137 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:24.006861  8056 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:13:24.009068  9886 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:24.009394  8056 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:13:24.013311  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 917) took 2015 ms. Trace:
I20260709 13:13:24.013401  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:24.016388  9734 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:24.024504 10081 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:24.025072 10081 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:13:24.025413 10195 log.cc:927] Time spent T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.419s	user 0.003s	sys 0.000s
I20260709 13:13:24.029031  8101 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:24.013886  8131 rpcz_store.cc:276] 0709 13:13:21.998363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.590993 (+592630us) service_pool.cc:224] Handling call
0709 13:13:22.591266 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:23.014735 (+423469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.013039 (+998304us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b540035cfd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:24.013229 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937997056}
I20260709 13:13:24.044638  8026 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:24.038028  8205 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:24.055277  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:24.083051  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 197562us
I20260709 13:13:24.084389  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:24.078915  8727 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:24.094599  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:24.095470  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50947.4us
I20260709 13:13:24.095935  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:24.097257  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.100649  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:24.095225  8205 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 1.
W20260709 13:13:24.102671  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 73714.1us
I20260709 13:13:24.119186  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2196) took 1403 ms. Trace:
I20260709 13:13:24.119707  8211 rpcz_store.cc:276] 0709 13:13:22.716118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.716241 (+   123us) service_pool.cc:224] Handling call
0709 13:13:22.726967 (+ 10726us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:23.192091 (+465124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.118884 (+926793us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b54002be654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:24.119108 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1068098304}
W20260709 13:13:24.126966  8059 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2063) took 2018 ms (client timeout 2430 ms). Trace:
W20260709 13:13:24.127511  8059 rpcz_store.cc:269] 0709 13:13:22.108901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.346518 (+237617us) service_pool.cc:224] Handling call
0709 13:13:22.380244 (+ 33726us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b5800000c98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:24.126768 (+1746524us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b540027b854. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:24.126895 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1186785408}
I20260709 13:13:24.132803 10010 raft_consensus.cc:3055] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:24.133296 10010 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:24.135030  8058 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:24.121153  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.147217  8211 raft_consensus.cc:2410] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 5 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:13:24.148926 10010 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.150612 10010 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:24.155946  8130 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:13:24.162498  8130 pending_rounds.cc:85] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 6
I20260709 13:13:24.163031  8130 pending_rounds.cc:107] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:13:24.163416  8130 raft_consensus.cc:2889] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:24.170789  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.171602  8205 raft_consensus.cc:2393] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:24.104605 10029 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:13:24.172410 10167 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:13:24.172590  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:24.172932  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:24.173367  8060 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:24.078150  9952 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.096s
I20260709 13:13:24.174577  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:24.175937 10216 log.cc:927] Time spent T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
W20260709 13:13:24.176301 10208 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:13:23.941612 10217 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:24.178830 10217 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:24.025967 10081 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:24.182103 10039 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:24.182714 10147 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:24.078157 10172 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:24.186964  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 83465.5us
I20260709 13:13:24.089057 10009 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:24.204830  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 101343us
W20260709 13:13:24.209178  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.218151  8057 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:24.184115  8129 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:24.186640  9913 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:24.204538 10170 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:24.202553  8721 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:24.213989 10217 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:24.248713  8009 auto_rebalancer.cc:893] Removing replace marker failed: Replica 8a968cada1e4446ebb2a5e167176e649 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:24.224071  9964 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:24.252537  8009 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:13:24.213780 10029 raft_consensus.cc:697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:24.257612  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 161770us
I20260709 13:13:24.258543  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:24.232908  9747 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:24.214619 10186 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
W20260709 13:13:24.252875  8009 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:13:24.262290  8026 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:24.262200  8176 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:24.243913 10089 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.205289 10172 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:24.216053 10009 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:24.305008  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 62132.1us
W20260709 13:13:24.244187  8055 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2056) took 2134 ms (client timeout 2045 ms). Trace:
I20260709 13:13:24.238600  9655 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:13:24.253042  9876 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:24.294728 10174 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:24.329284  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 70839.8us
I20260709 13:13:24.340395  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.311067 10089 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:24.332711 10213 log.cc:927] Time spent T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:13:24.308455 10052 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:24.348783  8059 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:24.317782  9913 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:24.360258  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.366991  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:24.293792 10217 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.378633  9913 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:24.379242 10218 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:24.379948 10218 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:24.379635  9913 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:24.254827 10170 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:24.307878  9964 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.302662  9747 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:24.382368  8175 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:24.383774  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.342671  9876 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.374068 10087 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:24.344110 10174 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.396765  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:24.400202  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.401335  8054 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:24.404527  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:24.391287  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 181089us
W20260709 13:13:24.330785  8055 rpcz_store.cc:269] 0709 13:13:22.099033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.106121 (+  7088us) service_pool.cc:224] Handling call
0709 13:13:24.168693 (+2062572us) spinlock_profiling.cc:243] Waited 720 ms on lock 0x7b54003ba454. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:24.233254 (+ 64561us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1655870208}
I20260709 13:13:24.376741 10052 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:24.398030  8211 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:24.392460 10087 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:24.408476  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:24.411067  8060 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:24.413823  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.397914 10039 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:24.394704 10223 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:24.407680 10212 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:24.444766  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:24.409015 10218 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.450637 10174 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:24.451529  9876 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:24.451728 10052 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.454998  9964 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:24.455793  8205 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:24.456794  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.457747  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.458639 10039 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:24.459605  9747 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:24.416662 10087 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:24.462801 10218 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.465600 10218 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:24.466205 10218 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:24.470116  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.471256  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:24.472378  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.473269  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2082) took 1276 ms. Trace:
I20260709 13:13:24.473644  8063 rpcz_store.cc:276] 0709 13:13:23.196375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.414818 (+218443us) service_pool.cc:224] Handling call
0709 13:13:23.415176 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:23.447602 (+ 32426us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:23.460720 (+ 13118us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.735078 (+274358us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:23.755427 (+ 20349us) log.cc:844] Serialized 26 byte log entry
0709 13:13:23.755547 (+   120us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:23.755567 (+    20us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.755608 (+    41us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:24.466209 (+710601us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b18002ae4a0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:13:24.472957 (+  6748us) raft_consensus.cc:1685] finished
0709 13:13:24.473072 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.473189 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29369,"prepare.queue_time_us":13089,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":50481,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":17430,"spinlock_wait_cycles":2946432,"thread_start_us":247197,"threads_started":2,"wal-append.queue_time_us":227094}
I20260709 13:13:24.476262  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:24.477423 10196 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.883s	user 0.003s	sys 0.000s
I20260709 13:13:24.479533  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2202) took 1186 ms. Trace:
I20260709 13:13:24.479955  8204 rpcz_store.cc:276] 0709 13:13:23.293032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.298693 (+  5661us) service_pool.cc:224] Handling call
0709 13:13:23.428725 (+130032us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:23.447380 (+ 18655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.453181 (+  5801us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:23.475450 (+ 22269us) log.cc:844] Serialized 26 byte log entry
0709 13:13:23.521750 (+ 46300us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:23.521789 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.521876 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:24.476866 (+954990us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:24.479220 (+  2354us) raft_consensus.cc:1685] finished
0709 13:13:24.479246 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.479467 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3493,"raft.queue_time_us":1674,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":6193,"spinlock_wait_cycles":43671936,"thread_start_us":37778,"threads_started":1,"wal-append.queue_time_us":110786}
I20260709 13:13:24.482944 10172 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:24.483383 10193 log.cc:927] Time spent T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.888s	user 0.002s	sys 0.000s
I20260709 13:13:24.483503  8062 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:24.484787  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:24.485345  8062 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:24.485656  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1249) took 1186 ms. Trace:
I20260709 13:13:24.486007  8135 rpcz_store.cc:276] 0709 13:13:23.298836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.317308 (+ 18472us) service_pool.cc:224] Handling call
0709 13:13:23.317616 (+   308us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:23.439919 (+122303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.439950 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:23.447973 (+  8023us) log.cc:844] Serialized 52 byte log entry
0709 13:13:23.481591 (+ 33618us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:23.486825 (+  5234us) log.cc:844] Serialized 14 byte log entry
0709 13:13:23.518785 (+ 31960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:23.518887 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:24.482675 (+963788us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:24.485419 (+  2744us) raft_consensus.cc:1685] finished
0709 13:13:24.485495 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.485605 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36479,"prepare.queue_time_us":4804,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":29,"raft.queue_time_us":168636,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":15904,"spinlock_wait_cycles":24879872,"thread_start_us":60500,"threads_started":2,"wal-append.queue_time_us":136768}
I20260709 13:13:24.487167  8696 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:24.488898 10190 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:24.489059  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2084) took 1292 ms. Trace:
I20260709 13:13:24.489430  8058 rpcz_store.cc:276] 0709 13:13:23.196804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.452365 (+255561us) service_pool.cc:224] Handling call
0709 13:13:24.134730 (+682365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.135438 (+   708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.488884 (+353446us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b540027b850. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:24.489004 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1568528640}
W20260709 13:13:24.490533  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.482172  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.491690  8204 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:24.492604  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:24.493708  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.494304  8204 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:24.495440  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:24.476692 10226 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:13:24.531052  9561 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:24.531864  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.539525  8206 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:24.389483 10029 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.540241  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:24.494161 10087 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:24.548220  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:24.549635  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:24.551964  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:24.558522  8029 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:24.440836  9890 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 13:13:24.578076  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:24.563311  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:24.576366  8203 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:13:24.576830  8204 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:13:24.541260 10181 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:24.541097 10212 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:24.586113  8206 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:13:24.590540  8206 pending_rounds.cc:85] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 1
I20260709 13:13:24.590843  8206 pending_rounds.cc:107] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:13:24.591193  8206 raft_consensus.cc:2889] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:24.552968  9952 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:24.554265  9747 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:24.561913  9988 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:24.590297  8204 pending_rounds.cc:85] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 2
I20260709 13:13:24.596323  8212 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:13:24.613907  8212 pending_rounds.cc:85] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 1
I20260709 13:13:24.614883  8212 pending_rounds.cc:107] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:13:24.615908  8212 raft_consensus.cc:2889] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:24.551970  9734 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:13:24.595439  9876 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:24.597194  8723 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:24.598925  8726 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:24.599354 10134 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:24.665864  8102 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:24.607831  8204 pending_rounds.cc:107] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:13:24.602144 10180 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.632095  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:24.604791 10231 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:24.633742  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 225358us
I20260709 13:13:24.617986 10027 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:24.621419 10009 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:24.638967  8207 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:24.648221  8728 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:24.630685 10089 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:24.602713 10181 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:24.633742 10217 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:24.662467  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:24.678411  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:24.712113  8136 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:24.659106 10241 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:24.721050  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 182240us
I20260709 13:13:24.678145  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:24.677656  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:24.719621 10183 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 1.277s	user 0.001s	sys 0.003s
W20260709 13:13:24.710893 10160 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:13:24.638245  9853 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:13:24.669250 10198 log.cc:927] Time spent T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.961s	user 0.004s	sys 0.000s
I20260709 13:13:24.721865  8104 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:24.725561  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:24.732266  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 54237.3us
I20260709 13:13:24.744060  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:24.698065 10010 consensus_meta.cc:287] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.547s	user 0.006s	sys 0.002s
I20260709 13:13:24.672595 10173 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:13:24.708267 10230 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:13:24.699636 10189 consensus_meta.cc:287] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.706s	user 0.010s	sys 0.000s
W20260709 13:13:24.690083 10224 log.cc:927] Time spent T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:13:24.741266  8207 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:24.692238 10096 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:24.673244  8204 raft_consensus.cc:2889] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:24.707526 10170 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:24.746206  8212 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:24.721061  9980 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:24.695717 10087 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:24.700307 10200 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:24.761488  8104 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:24.769340  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:24.695196  9964 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:13:24.773164  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1056) took 3309 ms. Trace:
I20260709 13:13:24.756848 10012 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:24.691293 10081 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:13:24.789312  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:24.691865 10174 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 13:13:24.779383  8210 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:24.693043 10180 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:24.796258 10180 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:24.735106 10175 consensus_queue.cc:1059] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:13:24.786582  8057 rpcz_store.cc:276] 0709 13:13:21.457535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:22.595832 (+1138297us) service_pool.cc:224] Handling call
0709 13:13:24.217927 (+1622095us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.218655 (+   728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.766971 (+548316us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54003ba450. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:24.767144 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3703771392}
I20260709 13:13:24.739971 10089 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:13:24.743494 10217 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:24.798378 10217 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:24.798758 10217 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:24.807019 10174 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:24.774516 10189 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:24.811208 10189 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:24.811993  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.812610  8211 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:24.813417  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.813948  8211 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:24.814266 10052 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:24.814823 10052 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:24.815261 10052 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:24.815356  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:24.818683  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:24.819886 10200 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.821848 10181 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:24.819886  9886 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:24.822919  9886 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:24.823774  9886 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.824649 10052 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.738131 10241 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.826522 10010 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:24.827116 10052 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:24.777776  8205 raft_consensus.cc:2410] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:13:24.828125 10241 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:24.829432  9964 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:24.792238  9980 raft_consensus.cc:697] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:24.831036  9980 consensus_queue.cc:237] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.832221 10081 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.832868 10081 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.792711 10245 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:24.792650 10192 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:24.793503  8029 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:24.792135 10243 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.772470 10096 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:24.841096  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.835448  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:24.835552  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 74167.4us
I20260709 13:13:24.807670 10188 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:24.837832  8211 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:24.848318  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:24.838564  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.838423  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.854077  8205 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:24.844331  8696 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:24.814118  9988 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:24.846805  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:24.859411  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:24.856218  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:24.854568 10029 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:13:24.859942  8029 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:24.858744  8130 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:24.692647 10231 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:24.844688  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:24.861734  8135 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:24.859601  8131 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:13:24.677995 10212 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.865059  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:24.791062 10170 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:24.865748  8134 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:24.821870 10200 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:24.847342  9964 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:24.854760 10009 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:24.872288  9655 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.784397  9734 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:13:24.873206  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2094) took 1148 ms. Trace:
I20260709 13:13:24.774255  9853 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:24.847204 10096 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:24.873852  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:24.846902 10180 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:24.880622  8056 rpcz_store.cc:276] 0709 13:13:23.724425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.010250 (+285825us) service_pool.cc:224] Handling call
0709 13:13:24.038673 (+ 28423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:24.103822 (+ 65149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:24.103866 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:24.183926 (+ 80060us) log.cc:844] Serialized 26 byte log entry
0709 13:13:24.608226 (+424300us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:24.608278 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.608359 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:24.873012 (+264653us) raft_consensus.cc:1685] finished
0709 13:13:24.873032 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:24.873154 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15109,"raft.queue_time_us":22606,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"thread_start_us":423890,"threads_started":1,"wal-append.queue_time_us":686793}
I20260709 13:13:24.865198 10189 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:24.865815  9887 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:24.864034 10181 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
W20260709 13:13:24.971045  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:24.854701 10002 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:13:24.866732  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:24.865130  9876 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:13:24.866297  9988 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.846822 10243 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.872786 10239 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.155s
I20260709 13:13:24.973377 10052 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:25.024324  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 229895us
I20260709 13:13:24.895855 10180 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:24.978870 10189 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:24.884584  9655 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:24.863412 10188 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:25.017830  8129 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: []
W20260709 13:13:25.058274  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 194553us
I20260709 13:13:25.062291  9561 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.062817  9561 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.063712  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:25.070528  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 210674us
I20260709 13:13:25.074146  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:25.075538  9561 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:25.056574 10180 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:25.058621 10189 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:25.089830 10252 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:13:25.092252  9853 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:25.094614 10081 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:25.101927 10248 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.327s	user 0.003s	sys 0.003s
I20260709 13:13:25.103575 10243 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:25.104254  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:25.109664 10173 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:25.110112 10173 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:25.119004  8722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 526) took 1339 ms. Trace:
I20260709 13:13:25.119383  8722 rpcz_store.cc:276] 0709 13:13:23.779383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.779562 (+   179us) service_pool.cc:224] Handling call
0709 13:13:23.779937 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:23.780386 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:23.780428 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:24.701029 (+920601us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b2c000c0e58. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db990fa11 00007f6db991f372 00007f6dbb051a01 00007f6dbb01643a 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2
0709 13:13:24.701582 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:13:24.778913 (+ 77331us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:24.778972 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:24.779063 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:25.118679 (+339616us) raft_consensus.cc:1685] finished
0709 13:13:25.118698 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:25.118940 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":513,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"spinlock_wait_cycles":8196992,"thread_start_us":72090,"threads_started":1,"wal-append.queue_time_us":355727}
W20260709 13:13:25.121762  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58918us
W20260709 13:13:25.122892 10255 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.239s	user 0.000s	sys 0.000s
W20260709 13:13:25.124051 10240 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:13:25.124186 10192 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:25.124789 10192 consensus_queue.cc:1243] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:13:25.126111 10253 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
W20260709 13:13:25.124053  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 51264.7us
I20260709 13:13:25.057615 10175 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:25.129247  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:25.062450 10010 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.129405  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:25.134119  8131 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:25.136523  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:25.137228  8138 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:25.137059  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:25.144394  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.148159  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:25.076611 10218 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:25.055538 10052 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.094251 10246 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:25.155700  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:25.187937  8132 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:25.204257 10257 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:13:25.206946 10231 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:25.208329  9655 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:25.158612 10192 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.208987 10192 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.209964 10052 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:25.155941 10175 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:25.153744 10010 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.212649 10247 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.213125 10247 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.215067 10247 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:25.216580  8210 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:25.214107 10010 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:25.215981 10175 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:25.219607 10089 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:25.217137  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:25.217505 10192 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:25.228704  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:25.229492  8132 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:25.230608  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:25.230816  9734 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:25.231673  9845 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:25.232049  9845 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:25.232386  9845 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:25.234083  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:25.235162  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:25.235872  8134 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:25.236282  8104 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:25.238003 10211 raft_consensus.cc:1064] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:13:25.249197 10263 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:25.249670 10263 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:25.250926  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 57412us
I20260709 13:13:25.210136  9964 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:25.186453 10243 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:25.250025 10263 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:25.194667 10246 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:13:25.278491  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 207014us
I20260709 13:13:25.243147 10188 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:25.240175 10211 consensus_queue.cc:237] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:25.255265 10243 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:25.241348  9890 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:13:25.294844  8056 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:25.197662 10245 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 13:13:25.296227  8055 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:25.297307  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:25.291424  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:25.298574  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:25.298838  8722 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:25.298756  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:25.240140  9845 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:25.299753  8057 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:25.250325 10257 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:13:25.300762  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:13:25.260190  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 196545us
I20260709 13:13:25.286429 10039 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:25.419864  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:25.458588  8059 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:25.325928  8057 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:13:25.641516  8131 consensus_meta.cc:287] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.001s
W20260709 13:13:25.321544  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 266232us
I20260709 13:13:25.691339  8131 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:25.549059  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:25.355727  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.347728  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:25.692487  8211 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:25.727094  8104 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:25.636595  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.304347  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:25.325789 10188 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:25.504519 10081 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:25.398666 10212 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.000s	sys 0.029s
I20260709 13:13:25.363863  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:25.498232  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:25.716027  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:25.334693 10217 consensus_meta.cc:287] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.506s	user 0.006s	sys 0.003s
I20260709 13:13:25.300562 10243 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:25.327244 10257 raft_consensus.cc:3055] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:25.562311 10147 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:25.487789 10192 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:25.325335 10170 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.651672  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:25.576133 10100 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.009s	sys 0.022s
I20260709 13:13:25.689900 10231 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.690716 10258 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:25.629881 10268 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:25.743860  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:25.744757  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:25.805083  8211 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:25.463734 10087 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:25.808531 10010 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:25.809042 10087 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:25.809221 10258 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:25.809393 10010 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.545249  9845 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:25.530579 10241 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:25.811158 10258 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:25.810930 10241 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.811682 10258 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:25.812026 10258 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:25.812353 10258 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:25.812927 10241 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:25.420476  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:25.813755  8055 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:25.611021  9734 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.325137 10249 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.486150  9853 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:25.814457 10268 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.815187  9853 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.814692  9734 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:25.816516 10268 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:25.817211 10268 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:25.464362 10173 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:25.817842  9734 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:25.741766  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:25.822742  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:25.823752  8722 raft_consensus.cc:2393] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:13:25.783779 10212 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.904s	user 0.000s	sys 0.045s
I20260709 13:13:25.694769 10039 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:25.828990 10039 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:25.833428  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:25.834591  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:25.835016  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:25.836252  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:25.777302 10259 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.733330 10188 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:25.808490 10231 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:25.741602 10081 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.817252  9853 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:25.529042 10175 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:25.729310  9913 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.732268 10269 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.779055 10217 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.785271 10257 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:25.818961  8723 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:13:25.819209 10173 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:25.805650 10192 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.877931  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:25.819492  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:25.878743 10275 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:13:25.819252 10087 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:25.879143 10192 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:25.814970 10249 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:25.880816 10263 consensus_meta.cc:287] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.000s
W20260709 13:13:25.802012 10100 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.010s	sys 0.030s
I20260709 13:13:25.882856 10081 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:25.882969 10272 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:13:25.837152  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 109954us
I20260709 13:13:25.882371 10263 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.860167 10002 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:25.837524  8129 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:25.791831 10147 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:25.863337 10259 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.488229 10246 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:25.870819  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:25.741683  9952 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:25.858421 10188 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:25.876579  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 160666us
I20260709 13:13:25.814795  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:25.805157 10170 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.745708 10207 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.814661 10010 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:25.886144  8104 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:25.937628  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 51597.6us
I20260709 13:13:25.938738  8104 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:25.896764 10217 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:25.878328 10277 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:13:25.880932 10249 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:25.942518  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:25.926210  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2209) took 1992 ms. Trace:
I20260709 13:13:25.955506  8203 rpcz_store.cc:276] 0709 13:13:23.933628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.934264 (+   636us) service_pool.cc:224] Handling call
0709 13:13:23.946745 (+ 12481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.590526 (+643781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.919108 (+1328582us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b540006efd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:25.926117 (+  7009us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1412912000}
I20260709 13:13:25.957340  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:25.959334 10284 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:13:25.874238  9913 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.885975 10175 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.958102  8203 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:25.951705 10081 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:25.876085 10269 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:25.944509 10207 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.133189  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:26.244168  8027 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:26.267215  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 329656us
I20260709 13:13:26.272620  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:26.235704  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:26.276532  8131 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
W20260709 13:13:26.276844  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.082304  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 266) took 1359 ms. Trace:
I20260709 13:13:26.277594  8207 rpcz_store.cc:276] 0709 13:13:24.723442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.841991 (+118549us) service_pool.cc:224] Handling call
0709 13:13:26.081632 (+1239641us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b5800168398. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:13:26.081818 (+   186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:26.081994 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:26.082019 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:26.082061 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:26.082097 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.082141 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.082240 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6586624}
W20260709 13:13:26.234663  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 870871us
I20260709 13:13:26.271364  8212 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:25.951973 10283 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:25.989959  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:26.294015  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.238940 10299 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.961499  9952 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:25.971036  9747 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.077803  9845 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.300549  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:26.301230  8212 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
W20260709 13:13:26.073320 10254 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 13:13:26.313024 10189 consensus_meta.cc:287] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 1.178s	user 0.004s	sys 0.005s
I20260709 13:13:26.087301 10257 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.313417 10189 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:26.263104 10217 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:26.314503 10217 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.315565 10027 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:26.316010 10027 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:26.316515 10217 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:26.317083 10096 consensus_meta.cc:287] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 1.298s	user 0.009s	sys 0.000s
I20260709 13:13:26.318974 10259 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.319585  8727 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:26.320181  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 384) took 1132 ms. Trace:
I20260709 13:13:26.320497  8727 rpcz_store.cc:276] 0709 13:13:25.187760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.187891 (+   131us) service_pool.cc:224] Handling call
0709 13:13:26.319254 (+1131363us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:13:26.319422 (+   168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:26.319959 (+   537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.320130 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11862912}
W20260709 13:13:26.320950 10180 consensus_meta.cc:287] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 1.187s	user 0.011s	sys 0.000s
I20260709 13:13:26.268774 10218 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:26.321421 10180 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.321695 10218 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.298019  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:26.239627 10175 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.006044 10081 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:26.303364  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:26.296633  8724 raft_consensus.cc:2393] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:26.310663  9845 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:26.284302  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2223) took 1043 ms. Trace:
I20260709 13:13:26.311185 10170 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.311616  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:26.301121 10247 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:26.312484 10263 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:25.966838 10029 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:26.337649  8207 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:26.317502 10096 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:26.338766 10002 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.339118 10081 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:26.338402  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 95629us
I20260709 13:13:25.960394 10294 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.351s
I20260709 13:13:26.274571 10243 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.298454 10010 raft_consensus.cc:2764] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:13:26.323385 10207 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:26.353363  8212 raft_consensus.cc:2410] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:26.302098 10231 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.353883  8728 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:13:26.324043  9747 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:26.354354  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:25.977029 10286 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.372s
I20260709 13:13:26.379842  8136 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
W20260709 13:13:26.355535  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 57571.2us
I20260709 13:13:26.320200  9913 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:26.357317  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2230) took 1113 ms. Trace:
I20260709 13:13:26.347304 10218 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:26.352563 10217 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:26.370860  8207 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:26.384318  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2231) took 1136 ms. Trace:
I20260709 13:13:26.384563  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:26.384649  8207 rpcz_store.cc:276] 0709 13:13:25.247942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.358756 (+1110814us) service_pool.cc:224] Handling call
0709 13:13:26.370696 (+ 11940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:26.384105 (+ 13409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.384268 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:26.368320  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:26.382535  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2225) took 1140 ms. Trace:
I20260709 13:13:26.385830  8212 rpcz_store.cc:276] 0709 13:13:25.241909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.311377 (+1069468us) service_pool.cc:224] Handling call
0709 13:13:26.382480 (+ 71103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:26.381232  8208 rpcz_store.cc:276] 0709 13:13:25.243889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.356527 (+1112638us) service_pool.cc:224] Handling call
0709 13:13:26.356827 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:26.356993 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:26.357024 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:26.357068 (+    44us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:26.357104 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.357158 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.357266 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:26.365929 10180 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.388249  8132 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:26.324167 10299 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:26.367406 10296 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:26.389755  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:26.394698  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 150640us
I20260709 13:13:26.358769 10247 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.378991 10291 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:26.365567 10029 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.423558  8026 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:26.393415  9913 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:26.373317 10002 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.395444 10211 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.397559 10290 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:26.432075  8026 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:26.397837 10217 raft_consensus.cc:697] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:26.421816 10304 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:26.432106 10010 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:26.419175 10188 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:26.507798  8135 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.005s	sys 0.020s
W20260709 13:13:26.508160  8135 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.005s	sys 0.020s
I20260709 13:13:26.490936  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 530) took 2050 ms. Trace:
I20260709 13:13:26.424964 10296 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:26.504784 10010 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:26.474086 10207 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:13:26.516599 10312 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:26.496675 10294 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:13:26.473519 10290 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:26.510196 10170 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:26.493052 10308 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:13:26.427090 10291 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:26.523310 10299 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:26.523928 10096 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:26.511904 10189 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:26.525470 10258 consensus_meta.cc:287] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.713s	user 0.007s	sys 0.007s
I20260709 13:13:26.527705  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:26.525542 10312 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:26.520192  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1264) took 1439 ms. Trace:
I20260709 13:13:26.557253  8133 rpcz_store.cc:276] 0709 13:13:25.081086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.131484 (+ 50398us) service_pool.cc:224] Handling call
0709 13:13:25.146481 (+ 14997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:25.211651 (+ 65170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:25.211694 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:25.211747 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:25.211795 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.518240 (+1306445us) spinlock_profiling.cc:243] Waited 64.9 ms on lock 0x7b5400001754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:26.518292 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.520131 (+  1839us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149376384}
I20260709 13:13:26.509789  8726 rpcz_store.cc:276] 0709 13:13:24.422740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.427073 (+  4333us) service_pool.cc:224] Handling call
0709 13:13:24.452888 (+ 25815us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:24.668884 (+215996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.426586 (+1757702us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54001793d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:26.472939 (+ 46353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304479488}
I20260709 13:13:26.535012  9845 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.537230  8723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 533) took 1336 ms. Trace:
I20260709 13:13:26.521364 10217 consensus_queue.cc:237] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.470553 10211 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.513566 10180 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:26.516705  9876 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.516782 10010 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:26.516855  9952 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.556775  8205 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:26.351540  8204 rpcz_store.cc:276] 0709 13:13:25.241318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.246861 (+  5543us) service_pool.cc:224] Handling call
0709 13:13:25.248922 (+  2061us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:25.249329 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:25.249358 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:25.249817 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:13:25.722740 (+472923us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:25.725623 (+  2883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.738305 (+ 12682us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:26.234332 (+496027us) raft_consensus.cc:1685] finished
0709 13:13:26.247352 (+ 13020us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.281057 (+ 33705us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3027,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":7927,"thread_start_us":440404,"threads_started":1,"wal-append.queue_time_us":442946}
I20260709 13:13:26.561658  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:26.571254  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 139309us
I20260709 13:13:26.572391  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2228) took 1329 ms. Trace:
I20260709 13:13:26.572710  8727 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:26.572749  8205 rpcz_store.cc:276] 0709 13:13:25.243274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.808623 (+565349us) service_pool.cc:224] Handling call
0709 13:13:26.527572 (+718949us) spinlock_profiling.cc:243] Waited 9.61 ms on lock 0x7b5800168398. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:26.572333 (+ 44761us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22098304}
I20260709 13:13:26.575932  8205 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:13:26.576978  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2238) took 1087 ms. Trace:
I20260709 13:13:26.577315  8205 rpcz_store.cc:276] 0709 13:13:25.490072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.575444 (+1085372us) service_pool.cc:224] Handling call
0709 13:13:26.575754 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:26.576757 (+  1003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.576933 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:26.552517 10170 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:26.568892  8724 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:26.569826  9747 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.569376 10303 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:26.575464  8129 consensus_meta.cc:287] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.656s	user 0.010s	sys 0.000s
I20260709 13:13:26.560947 10258 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.587001 10180 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:26.590476 10308 consensus_queue.cc:1059] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:26.580432 10263 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:26.593372 10027 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.579046  9952 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.574879  9876 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.595265 10170 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:26.598582 10278 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.952s	user 0.008s	sys 0.009s
I20260709 13:13:26.603145 10029 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.603636 10269 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:26.589218  8129 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:26.606549 10290 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:26.606834 10207 consensus_queue.cc:1059] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:26.566754  8723 rpcz_store.cc:276] 0709 13:13:25.200833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.202086 (+  1253us) service_pool.cc:224] Handling call
0709 13:13:25.236636 (+ 34550us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:25.877474 (+640838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.536942 (+659468us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b54005802d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:26.537168 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1338544896}
I20260709 13:13:26.608229 10303 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.609822  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:26.610934 10296 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:26.635437 10180 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:26.658936 10319 raft_consensus.cc:3055] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:26.612681 10247 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:26.580494 10305 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:26.614799 10304 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:26.629415  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 77434.2us
I20260709 13:13:26.671097  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 248) took 1082 ms. Trace:
W20260709 13:13:26.635035  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 200042us
I20260709 13:13:26.636320  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1091) took 1341 ms. Trace:
I20260709 13:13:26.640647 10258 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.602035 10081 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.666024  8205 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:26.687546  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:26.634835 10288 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:26.639459 10002 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.639268 10283 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:26.643497  9952 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:26.612411 10292 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:13:26.659623 10188 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:26.640036 10263 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.672590 10291 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:26.675593  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 101841us
I20260709 13:13:26.633877 10174 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:26.681048  8058 rpcz_store.cc:276] 0709 13:13:25.291918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.293976 (+  2058us) service_pool.cc:224] Handling call
0709 13:13:25.294296 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:25.301620 (+  7324us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:26.595889 (+1294269us) spinlock_profiling.cc:243] Waited 9.2 ms on lock 0x7b18002ec4e0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf
0709 13:13:26.596024 (+   135us) log.cc:844] Serialized 14 byte log entry
0709 13:13:26.633355 (+ 37331us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:26.633404 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:26.633446 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.633510 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.633656 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":423996,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":108,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"spinlock_wait_cycles":21157760,"thread_start_us":459178,"threads_started":2,"wal-append.queue_time_us":61010}
I20260709 13:13:26.686643 10200 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:26.686810  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:26.666391  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 277) took 1444 ms. Trace:
I20260709 13:13:26.688277  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:26.741859  8057 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:13:26.748893  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.764721  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2226) took 1522 ms. Trace:
I20260709 13:13:26.920467  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2236) took 1623 ms. Trace:
W20260709 13:13:26.988272  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.688586  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 241) took 1615 ms. Trace:
I20260709 13:13:26.991530  8137 rpcz_store.cc:276] 0709 13:13:25.073105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.073230 (+   125us) service_pool.cc:224] Handling call
0709 13:13:25.104359 (+ 31129us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:25.136279 (+ 31920us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:25.136308 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:25.149473 (+ 13165us) log.cc:844] Serialized 26 byte log entry
0709 13:13:25.562117 (+412644us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:25.562173 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:25.562434 (+   261us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:26.598012 (+1035578us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:26.688276 (+ 90264us) raft_consensus.cc:1685] finished
0709 13:13:26.688297 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.688512 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31537,"raft.queue_time_us":32111,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":27465,"spinlock_wait_cycles":3103104,"thread_start_us":302423,"threads_started":1,"wal-append.queue_time_us":340766}
I20260709 13:13:26.676134 10308 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:26.995270  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:26.758599  8101 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.710s (SENT) [suppressed 7 similar messages]
I20260709 13:13:26.854354  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:26.998083  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:26.998307  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:26.677341 10319 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:26.999060  8063 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:26.999100  8057 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:27.076992  8133 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:27.001497  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:26.739851  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:26.680501 10305 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:27.626958  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:27.105943  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:27.547446  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 121534us
W20260709 13:13:27.628574  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:27.369578  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 610976us
I20260709 13:13:27.444797  8104 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:27.630909  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 186171us
I20260709 13:13:27.496486  8102 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:27.332937  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 534) took 2077 ms. Trace:
I20260709 13:13:26.802414 10211 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.762765 10027 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:26.749759 10325 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:27.447376 10257 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:27.645788  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2222) took 2387 ms. Trace:
I20260709 13:13:26.989352  8210 rpcz_store.cc:276] 0709 13:13:25.221948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.222113 (+   165us) service_pool.cc:224] Handling call
0709 13:13:25.222549 (+   436us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:25.258549 (+ 36000us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:25.258578 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:25.259153 (+   575us) log.cc:844] Serialized 52 byte log entry
0709 13:13:26.084840 (+825687us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b48002301d8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:13:26.321500 (+236660us) spinlock_profiling.cc:243] Waited 4.66 ms on lock 0x7b1800473980. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:13:26.377432 (+ 55932us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:26.435092 (+ 57660us) log.cc:844] Serialized 14 byte log entry
0709 13:13:26.666051 (+230959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.666123 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:26.666161 (+    38us) raft_consensus.cc:1685] finished
0709 13:13:26.666173 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.666309 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58090,"prepare.queue_time_us":136180,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":63222,"raft.run_cpu_time_us":15234,"raft.run_wall_time_us":833319,"spinlock_wait_cycles":13846656,"thread_start_us":287096,"threads_started":3,"wal-append.queue_time_us":1215107}
I20260709 13:13:27.634683 10305 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:27.439035  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1274) took 1562 ms. Trace:
I20260709 13:13:27.688148  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:27.688079  8132 rpcz_store.cc:276] 0709 13:13:25.876907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.877022 (+   115us) service_pool.cc:224] Handling call
0709 13:13:27.438795 (+1561773us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b540035cad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:27.438963 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1161738112}
I20260709 13:13:26.756042 10180 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:27.382671 10188 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.307076 10170 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.167466 10002 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:27.218387 10010 consensus_meta.cc:287] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.640s	user 0.007s	sys 0.003s
I20260709 13:13:27.441298  8729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 537) took 1876 ms. Trace:
I20260709 13:13:26.739486 10315 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.308524 10319 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:27.713838 10315 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:27.715045  8056 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:27.715137 10319 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:26.988517  8209 rpcz_store.cc:276] 0709 13:13:25.242200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.340680 (+1098480us) service_pool.cc:224] Handling call
0709 13:13:26.341086 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:26.354985 (+ 13899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:26.355015 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:26.380278 (+ 25263us) log.cc:844] Serialized 26 byte log entry
0709 13:13:26.639441 (+259163us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:26.639479 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.639543 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:26.764459 (+124916us) raft_consensus.cc:1685] finished
0709 13:13:26.764486 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.764654 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11218,"raft.queue_time_us":14106,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":12366,"thread_start_us":246485,"threads_started":1,"wal-append.queue_time_us":281079}
I20260709 13:13:26.730407  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:27.638259  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:27.717020  8063 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:27.718014  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:27.718683  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:27.719060  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:27.719630  8063 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:27.721128  8727 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 4 LEARNER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:13:27.721715  8056 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:27.638502 10263 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.845710 10187 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:26.989589  8208 rpcz_store.cc:276] 0709 13:13:25.296925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.386790 (+1089865us) service_pool.cc:224] Handling call
0709 13:13:26.387169 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:26.394502 (+  7333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:26.394535 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:26.394950 (+   415us) log.cc:844] Serialized 26 byte log entry
0709 13:13:26.668568 (+273618us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:26.668625 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.668711 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:26.920174 (+251463us) raft_consensus.cc:1685] finished
0709 13:13:26.920208 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.920393 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3378,"raft.queue_time_us":7662,"spinlock_wait_cycles":9997184,"thread_start_us":237280,"threads_started":1,"wal-append.queue_time_us":325023}
I20260709 13:13:27.638991  9952 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:27.454072  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2235) took 2158 ms. Trace:
I20260709 13:13:27.691906  9853 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.719434  8722 raft_consensus.cc:2393] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:27.435433  8730 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:13:27.059629  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:26.729755 10247 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:27.105947  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:27.735208  8131 raft_consensus.cc:2410] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:27.001498 10174 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:27.047029  8135 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:27.042189 10283 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:27.274823  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 937) took 2425 ms. Trace:
I20260709 13:13:26.740090 10303 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:27.215848  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:27.465662  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2239) took 1889 ms. Trace:
I20260709 13:13:27.752298  8205 rpcz_store.cc:276] 0709 13:13:25.575852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.577980 (+1002128us) service_pool.cc:224] Handling call
0709 13:13:26.578740 (+   760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:26.688094 (+109354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:27.465390 (+777296us) spinlock_profiling.cc:243] Waited 87.1 ms on lock 0x7b54002be154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:27.465594 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":200273280}
W20260709 13:13:27.387372  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.639220  8058 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:27.754190  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:27.060338  8138 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:27.755949  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:27.756611  8138 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:27.643882  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:27.041854 10317 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:27.644968  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 148566us
I20260709 13:13:27.774362 10317 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:27.633652  8728 rpcz_store.cc:276] 0709 13:13:25.255151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.255993 (+   842us) service_pool.cc:224] Handling call
0709 13:13:26.318817 (+1062824us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:13:26.318998 (+   181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:26.373848 (+ 54850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:27.332606 (+958758us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b540041e6d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:27.332862 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84045568}
I20260709 13:13:27.776432 10317 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:27.776706  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:27.777887  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:27.753340  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:27.781014  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:27.781661  8205 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:27.662653  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:26.680783  8129 rpcz_store.cc:276] 0709 13:13:25.587017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.610902 (+ 23885us) service_pool.cc:224] Handling call
0709 13:13:26.669104 (+1058202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:27.634908 10246 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:26.688417 10269 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:26.921557  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:26.740703 10258 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:27.784049  8055 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:27.784099 10258 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:26.734207 10207 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:27.734606  9853 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:27.324792 10311 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.814s	user 0.008s	sys 0.000s
I20260709 13:13:27.786156 10258 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:27.382899  9988 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:27.692279  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:26.995760 10308 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:27.740393 10247 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:27.789224 10308 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:27.790442 10308 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:27.722484 10002 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:27.410403  8212 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.000s	sys 0.008s
W20260709 13:13:27.797360  8212 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.127s	user 0.001s	sys 0.009s
I20260709 13:13:27.690915  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:27.393854 10241 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.703830  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:27.051194 10189 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:27.704111  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1090) took 2413 ms. Trace:
I20260709 13:13:27.834012  8061 rpcz_store.cc:276] 0709 13:13:25.290602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.292565 (+  1963us) service_pool.cc:224] Handling call
0709 13:13:25.292944 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:26.351662 (+1058718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:26.351700 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:26.358176 (+  6476us) log.cc:844] Serialized 26 byte log entry
0709 13:13:27.449521 (+1091345us) spinlock_profiling.cc:243] Waited 9.5 ms on lock 0x7b1800461260. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:13:27.637892 (+188371us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:27.638145 (+   253us) log.cc:844] Serialized 14 byte log entry
0709 13:13:27.665534 (+ 27389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:27.703195 (+ 37661us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b1800461260. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ef72a6 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419
0709 13:13:27.703766 (+   571us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540043d854. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:27.703820 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:27.703874 (+    54us) raft_consensus.cc:1685] finished
0709 13:13:27.703891 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:27.704045 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77316,"prepare.queue_time_us":4280,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":79812,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":15338,"spinlock_wait_cycles":2445792512,"thread_start_us":188206,"threads_started":1,"wal-append.queue_time_us":1261059}
I20260709 13:13:27.623359 10173 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:27.437820  9913 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:27.836305  9913 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:27.722316 10170 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:27.327059 10177 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:27.462294  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:27.499481  9734 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:27.699465  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 202) took 1023 ms. Trace:
I20260709 13:13:27.460896  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:26.766073  9876 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.995081 10299 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:27.371955 10217 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.685868  8206 rpcz_store.cc:276] 0709 13:13:25.240999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.251500 (+ 10501us) service_pool.cc:224] Handling call
0709 13:13:25.262903 (+ 11403us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:25.280922 (+ 18019us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:25.280957 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:25.299665 (+ 18708us) log.cc:844] Serialized 52 byte log entry
0709 13:13:25.729759 (+430094us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:25.774920 (+ 45161us) log.cc:844] Serialized 14 byte log entry
0709 13:13:26.191639 (+416719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.191741 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:27.416830 (+1225089us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b180006c580. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:13:27.428433 (+ 11603us) raft_consensus.cc:1685] finished
0709 13:13:27.428529 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:27.627883 (+199354us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44981,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":59711,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":5809,"spinlock_wait_cycles":5988608,"thread_start_us":459495,"threads_started":2,"wal-append.queue_time_us":333440}
I20260709 13:13:27.216607 10081 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:27.319607 10147 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:27.661969 10192 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:27.661962 10320 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:27.059144 10087 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:27.716989  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:27.109786  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.711181  8729 rpcz_store.cc:276] 0709 13:13:25.565295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.565436 (+   141us) service_pool.cc:224] Handling call
0709 13:13:25.624237 (+ 58801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:25.837041 (+212804us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:25.926855 (+ 89814us) log.cc:844] Serialized 14 byte log entry
0709 13:13:25.948027 (+ 21172us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:25.948098 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:25.948162 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:27.440157 (+1491995us) spinlock_profiling.cc:243] Waited 8.36 ms on lock 0x7b1800423040. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ef72a6 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419
0709 13:13:27.441025 (+   868us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b5400198f54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:27.441080 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:27.441232 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":530328,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":20992,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":4395,"spinlock_wait_cycles":508023424,"thread_start_us":55980,"threads_started":1}
I20260709 13:13:27.723223  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:27.688982 10218 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.725881  8063 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:27.725867  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:27.728160  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:27.897413  8208 raft_consensus.cc:2410] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:13:27.446663 10243 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:27.729619  8207 rpcz_store.cc:276] 0709 13:13:25.295374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.385543 (+1090169us) service_pool.cc:224] Handling call
0709 13:13:26.415440 (+ 29897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:26.469723 (+ 54283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:26.469753 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:26.469795 (+    42us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:26.469822 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:27.453818 (+983996us) spinlock_profiling.cc:243] Waited 54.1 ms on lock 0x7b5400189054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:27.453871 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:27.454008 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124434816}
I20260709 13:13:27.639873  8204 raft_consensus.cc:2410] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:13:27.882936 10299 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:27.910120  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1210) took 1173 ms. Trace:
I20260709 13:13:27.911599 10299 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:27.736024  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:27.911881 10218 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:26.802779 10304 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.034548 10200 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:27.913743 10218 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:27.914022 10200 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:26.803015  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:27.914574 10323 raft_consensus.cc:670] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } [suppressed 3 similar messages]
I20260709 13:13:27.914908  8723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 406) took 1234 ms. Trace:
I20260709 13:13:27.914184  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:27.740417  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:27.915630  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:27.915993  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 175676us
I20260709 13:13:27.916185  8207 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:27.916944  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:27.917505  8063 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 3.
I20260709 13:13:27.740805  8730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 385) took 2532 ms. Trace:
I20260709 13:13:27.870718 10177 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:27.918000  8730 rpcz_store.cc:276] 0709 13:13:25.202548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.202678 (+   130us) service_pool.cc:224] Handling call
0709 13:13:27.434806 (+2232128us) spinlock_profiling.cc:243] Waited 9.6 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:13:27.435026 (+   220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:27.733517 (+298491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:27.735264 (+  1747us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22080768}
I20260709 13:13:27.773317  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:27.754773  8058 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:27.786511 10269 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:27.786585 10207 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:26.742535  8721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 400) took 1049 ms. Trace:
I20260709 13:13:27.782287  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:27.688977 10325 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:27.869911 10173 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:27.869583 10170 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:27.712580 10188 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:27.732309 10187 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:27.854272 10337 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:13:27.868120  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:27.725385 10010 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:27.871285  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:27.869314  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2237) took 2447 ms. Trace:
I20260709 13:13:27.868346  9913 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:27.870725  8054 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:27.867935 10189 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:27.874297  8135 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:13:27.874545  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:27.876660  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:27.876688  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:27.878527  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 103414us
I20260709 13:13:27.875233  8062 rpcz_store.cc:276] 0709 13:13:26.675717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.675826 (+   109us) service_pool.cc:224] Handling call
0709 13:13:26.676106 (+   280us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:26.679148 (+  3042us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:26.679177 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:26.855756 (+176579us) log.cc:844] Serialized 52 byte log entry
0709 13:13:27.320021 (+464265us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:27.683517 (+363496us) log.cc:844] Serialized 14 byte log entry
0709 13:13:27.699090 (+ 15573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:27.699191 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:27.699246 (+    55us) raft_consensus.cc:1685] finished
0709 13:13:27.699263 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:27.699416 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":130128,"prepare.queue_time_us":315446,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":16869,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":55762,"thread_start_us":737675,"threads_started":2,"wal-append.queue_time_us":779889}
I20260709 13:13:27.877518  8728 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
W20260709 13:13:27.882454  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:27.892325  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1224) took 1097 ms. Trace:
I20260709 13:13:27.898252  8725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 409) took 1122 ms. Trace:
I20260709 13:13:27.898445 10087 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:27.899684  8729 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:13:27.886060 10260 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:27.900213  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1221) took 1106 ms. Trace:
I20260709 13:13:27.892632 10192 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:27.879868  9734 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:27.898185 10320 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:27.901049  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:27.901929  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:27.704753 10027 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:27.749287  8136 rpcz_store.cc:276] 0709 13:13:24.844050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:24.860309 (+ 16259us) service_pool.cc:224] Handling call
0709 13:13:27.269323 (+2409014us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b540036e554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:27.269456 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3434249984}
I20260709 13:13:27.919415  8131 raft_consensus.cc:2410] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:13:27.883483 10333 log.cc:927] Time spent T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:13:27.880524 10319 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:27.910485  8204 rpcz_store.cc:276] 0709 13:13:26.737222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.737385 (+   163us) service_pool.cc:224] Handling call
0709 13:13:27.910059 (+1172674us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:27.923197  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 236354us
I20260709 13:13:27.923707  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:27.926450  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:27.927443  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:27.883114 10217 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:27.894501 10081 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:27.934384 10039 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:27.915454  8723 rpcz_store.cc:276] 0709 13:13:26.680048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.688303 (+  8255us) service_pool.cc:224] Handling call
0709 13:13:27.914850 (+1226547us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:27.934737  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:27.936494 10177 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:27.989503  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 298638us
I20260709 13:13:27.873032 10241 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:28.006170  9853 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:28.007853  8055 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:28.008438  8203 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:28.008265 10218 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:28.009629  8205 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:28.014645  8207 raft_consensus.cc:2410] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:13:28.023244  8059 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:27.890570 10258 consensus_queue.cc:1059] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.128s
I20260709 13:13:28.029369  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 203) took 1352 ms. Trace:
I20260709 13:13:28.031123 10247 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:27.922156 10304 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:28.236763  8211 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:28.236794  8056 rpcz_store.cc:276] 0709 13:13:26.676912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.677017 (+   105us) service_pool.cc:224] Handling call
0709 13:13:28.029158 (+1352141us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b540009c254. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:28.029299 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1675995392}
I20260709 13:13:28.153301  8134 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:28.238668  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:28.257807  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.257984  8056 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
W20260709 13:13:28.258921  8101 leader_election.cc:336] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 2.430s (SENT)
I20260709 13:13:28.238611  8104 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:28.259970  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.261464  8061 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:28.261634  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 947) took 1217 ms. Trace:
I20260709 13:13:28.262014  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 208) took 1279 ms. Trace:
I20260709 13:13:28.237396  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1230) took 1255 ms. Trace:
I20260709 13:13:28.262672  8138 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:28.110911  8057 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:28.264521  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:28.263257  8211 rpcz_store.cc:276] 0709 13:13:26.982286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.837516 (+855230us) service_pool.cc:224] Handling call
0709 13:13:28.237332 (+399816us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.261552  8210 rpcz_store.cc:276] 0709 13:13:26.794562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.693670 (+899108us) service_pool.cc:224] Handling call
0709 13:13:27.892279 (+198609us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.265097  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1223) took 1470 ms. Trace:
I20260709 13:13:28.266870  8206 raft_consensus.cc:2410] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:13:28.237560  8208 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:28.267130  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:28.264662  8058 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:28.267465  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1215) took 1528 ms. Trace:
I20260709 13:13:28.267722  8132 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:28.267961  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2105) took 1282 ms. Trace:
W20260709 13:13:28.227787  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 305006us
I20260709 13:13:28.266247  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 954) took 1079 ms. Trace:
I20260709 13:13:28.260628  8062 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:28.266978  8205 rpcz_store.cc:276] 0709 13:13:26.794271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.782090 (+987819us) service_pool.cc:224] Handling call
0709 13:13:28.265045 (+482955us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.281140  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 953) took 1094 ms. Trace:
I20260709 13:13:28.281629  8130 rpcz_store.cc:276] 0709 13:13:27.186354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.752149 (+565795us) service_pool.cc:224] Handling call
0709 13:13:28.281080 (+528931us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.281920  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:28.219481 10257 consensus_meta.cc:287] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.000s
I20260709 13:13:28.282927  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:28.292629  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.209370  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:28.261922  8212 rpcz_store.cc:276] 0709 13:13:25.422432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.420854 (+998422us) service_pool.cc:224] Handling call
0709 13:13:26.431688 (+ 10834us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:26.431923 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:26.669882 (+237959us) log.cc:844] Serialized 14 byte log entry
0709 13:13:27.837957 (+1168075us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:27.869051 (+ 31094us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:27.869099 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:27.869160 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:27.869262 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":119343,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":524,"raft.queue_time_us":77582,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":15895,"thread_start_us":1305828,"threads_started":2,"wal-append.queue_time_us":966742}
I20260709 13:13:28.262449  8061 rpcz_store.cc:276] 0709 13:13:26.982710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.835895 (+853185us) service_pool.cc:224] Handling call
0709 13:13:28.261964 (+426069us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.032672 10269 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:28.250735 10245 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.299207  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:28.299477  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:28.267900  8208 rpcz_store.cc:276] 0709 13:13:26.738721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.934442 (+1195721us) service_pool.cc:224] Handling call
0709 13:13:28.267415 (+332973us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.294548  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:28.304461  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:28.148013 10012 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:28.295233  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 56695us
I20260709 13:13:28.300217  8062 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:28.268654  8058 rpcz_store.cc:276] 0709 13:13:26.985175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.925242 (+940067us) service_pool.cc:224] Handling call
0709 13:13:28.267911 (+342669us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.256641  8063 raft_consensus.cc:2393] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 3: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:28.105907  8724 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:28.105577 10173 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:28.269197  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.291864  8207 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:28.149755 10315 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:28.145491 10249 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.151477 10188 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:28.108652 10187 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:28.149848  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:28.304327  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.265940 10286 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:28.305831  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:28.305051  8056 raft_consensus.cc:2376] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5. Re-sending same reply.
I20260709 13:13:28.294664  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:28.148566 10308 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:28.238070 10260 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:28.031319 10283 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:28.032434 10330 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:13:28.266420 10192 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:28.316365  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:28.281682  9853 raft_consensus.cc:3055] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:28.256393 10325 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:28.315361  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.036801 10174 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:28.265569 10039 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.260433  8209 rpcz_store.cc:276] 0709 13:13:26.793731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.716762 (+923031us) service_pool.cc:224] Handling call
0709 13:13:27.900158 (+183396us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.319219  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:28.262321  8135 rpcz_store.cc:276] 0709 13:13:27.044002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.046506 (+  2504us) service_pool.cc:224] Handling call
0709 13:13:27.046892 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:28.261236 (+1214344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:28.261564 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.319607  8057 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:28.293717  9913 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:28.294276  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 200) took 1610 ms. Trace:
I20260709 13:13:28.314764  8104 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:28.307083  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:28.323129  8102 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:28.298978  8722 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:28.325762  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:28.293196  8131 rpcz_store.cc:276] 0709 13:13:27.186592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.735813 (+549221us) service_pool.cc:224] Handling call
0709 13:13:28.266198 (+530385us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.258647 10319 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:28.327530  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:28.330091  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 597878us
I20260709 13:13:28.009294  8721 rpcz_store.cc:276] 0709 13:13:25.692798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.725175 (+ 32377us) service_pool.cc:224] Handling call
0709 13:13:25.741624 (+ 16449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:25.787002 (+ 45378us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:25.941823 (+154821us) log.cc:844] Serialized 14 byte log entry
0709 13:13:26.671639 (+729816us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:26.675421 (+  3782us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:26.676966 (+  1545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:26.698551 (+ 21585us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:26.736850 (+ 38299us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":567926,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":25463,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":6083,"thread_start_us":828564,"threads_started":3,"wal-append.queue_time_us":476685,"wal-append.run_cpu_time_us":1222,"wal-append.run_wall_time_us":1124533}
I20260709 13:13:28.333407  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1107) took 1249 ms. Trace:
I20260709 13:13:28.322638  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:28.320245 10349 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:28.339829 10010 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:28.322943 10096 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:13:28.320377 10246 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:13:28.299173 10263 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:28.281976 10081 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:28.316447 10245 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:28.284121 10027 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:28.330708  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.598937  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:28.603410  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 280376us
W20260709 13:13:28.610333  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.436890  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1097) took 1637 ms. Trace:
I20260709 13:13:28.647801  8062 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:28.648696  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:28.413584  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.603790  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:28.649434  8062 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:28.605151  8133 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:28.603848  8059 rpcz_store.cc:276] 0709 13:13:27.083559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.640865 (+557306us) service_pool.cc:224] Handling call
0709 13:13:28.333260 (+692395us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b54001e91d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:28.333360 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":768900864}
I20260709 13:13:28.650565  8058 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:28.651510  8130 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 1.
I20260709 13:13:28.604841  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:28.604244  8206 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:28.550875  8063 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:13:28.653532  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:28.654191  8063 raft_consensus.cc:606] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:28.646322  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 94030.3us
I20260709 13:13:28.654606  8063 raft_consensus.cc:614] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:28.607514  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:28.260999  8725 rpcz_store.cc:276] 0709 13:13:26.776242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.781213 (+  4971us) service_pool.cc:224] Handling call
0709 13:13:27.462120 (+680907us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:27.898170 (+436050us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4340096}
I20260709 13:13:28.653592  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55942 (request call id 280) took 1019 ms. Trace:
I20260709 13:13:28.656842  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:28.657725  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.657853  8137 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:28.658749  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.659284  8061 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:28.655248  8211 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:28.659304  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:28.660375  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1216) took 1921 ms. Trace:
I20260709 13:13:28.660737  8211 rpcz_store.cc:276] 0709 13:13:26.738984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.587903 (+1848919us) service_pool.cc:224] Handling call
0709 13:13:28.660316 (+ 72413us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.656551  8207 rpcz_store.cc:276] 0709 13:13:27.633837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.604630 (+970793us) service_pool.cc:224] Handling call
0709 13:13:28.653540 (+ 48910us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.655287  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:28.663349  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:28.669140  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:28.587585 10337 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:13:28.675921  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:28.676365  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:28.676954  8211 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
I20260709 13:13:28.677834  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:28.678040  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:28.678546  8211 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:28.672395  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 136746us
I20260709 13:13:28.605178  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:28.680168  8026 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:28.680982  8026 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:28.648054  8060 rpcz_store.cc:276] 0709 13:13:26.799520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.807745 (+  8225us) service_pool.cc:224] Handling call
0709 13:13:26.855584 (+ 47839us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:27.266594 (+411010us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:27.266639 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:27.277728 (+ 11089us) log.cc:844] Serialized 26 byte log entry
0709 13:13:27.409543 (+131815us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:28.066295 (+656752us) spinlock_profiling.cc:243] Waited 7.28 ms on lock 0x7b1800477520. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf
0709 13:13:28.221415 (+155120us) log.cc:844] Serialized 14 byte log entry
0709 13:13:28.434648 (+213233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:28.436490 (+  1842us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b5400169254. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:28.436560 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:28.436616 (+    56us) raft_consensus.cc:1685] finished
0709 13:13:28.436636 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:28.436821 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":212931,"prepare.queue_time_us":226056,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"raft.queue_time_us":216387,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":6401,"spinlock_wait_cycles":958651264,"thread_start_us":176152,"threads_started":2,"wal-append.queue_time_us":404743}
I20260709 13:13:28.673880  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:28.674882  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:28.675362  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.682704  8209 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:28.684718  8205 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:28.684911  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:28.682744  8730 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:28.605417 10260 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:28.604777 10257 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:28.688692  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:28.697388  8206 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:28.697639  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 50444.2us
I20260709 13:13:28.697824  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2252) took 1010 ms. Trace:
I20260709 13:13:28.697975  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:28.698167  8206 rpcz_store.cc:276] 0709 13:13:27.687170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.653737 (+966567us) service_pool.cc:224] Handling call
0709 13:13:28.697776 (+ 44039us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.327216 10207 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:28.690968  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:28.692058  8062 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
W20260709 13:13:28.432695  8203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1211) took 1695 ms (client timeout 1538 ms). Trace:
W20260709 13:13:28.704491  8203 rpcz_store.cc:269] 0709 13:13:26.737576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.460551 (+722975us) service_pool.cc:224] Handling call
0709 13:13:28.432501 (+971950us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b5400153bd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:28.432624 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94786944}
I20260709 13:13:28.705013  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.705678  8059 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:28.706609  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.706990 10258 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:28.669298 10299 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:13:28.707386 10258 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:28.707556  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.707811 10258 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:28.708643  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.693925 10304 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:28.677287 10315 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:13:28.341269  8724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 391) took 1669 ms (client timeout 1955 ms). Trace:
I20260709 13:13:28.709271 10315 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:28.709326  8724 rpcz_store.cc:269] 0709 13:13:26.671847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.672999 (+  1152us) service_pool.cc:224] Handling call
0709 13:13:28.341098 (+1668099us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400694e54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:28.341212 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894720384}
I20260709 13:13:28.693004  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.616693 10039 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:28.711459 10315 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:28.711933 10027 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:28.702253 10283 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:13:28.713128  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:28.713726  8063 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:28.689424  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.652328  9853 raft_consensus.cc:740] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:28.320207 10012 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:28.715852  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:28.716473  8061 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:28.717329 10012 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:28.648247 10349 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:28.718112 10349 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:28.718297  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:28.719420  8029 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:28.719614 10189 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:28.699573  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 384861us
I20260709 13:13:28.655795 10246 consensus_queue.cc:1243] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:28.715818 10258 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:28.723225  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1226) took 1891 ms. Trace:
I20260709 13:13:28.723438 10188 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:28.723598  8205 rpcz_store.cc:276] 0709 13:13:26.831409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.304180 (+1472771us) service_pool.cc:224] Handling call
0709 13:13:28.723060 (+418880us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b540006f9d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:28.723155 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":872730496}
I20260709 13:13:28.723767  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:28.724741 10188 raft_consensus.cc:3055] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:28.725080 10188 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:28.725703 10304 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:28.725793 10188 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:28.726285 10304 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:28.727154 10188 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:28.728391 10304 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:28.728936 10304 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:28.729513 10304 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:28.730127 10246 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:28.699867  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:28.689360 10269 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:28.700697  8175 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:28.434504 10170 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:28.701253 10323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.057s	sys 0.052s
I20260709 13:13:28.701792  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:28.433343 10320 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:28.704018  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.868285  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.868892  8062 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:28.869879  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.870707  8062 raft_consensus.cc:2393] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:28.656430 10263 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:28.689517  9876 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:28.730530 10246 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.713406 10353 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:13:28.657369 10308 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:28.691313  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:28.698577  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:28.734648  8061 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:28.928337  8207 raft_consensus.cc:2410] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:28.928582  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.929415  8061 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:28.731078  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1232) took 1664 ms. Trace:
W20260709 13:13:28.930438  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:28.760890  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 60233.9us
I20260709 13:13:28.396687  9734 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:28.776252  8209 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:28.931876  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2249) took 1263 ms. Trace:
I20260709 13:13:28.932183  8209 rpcz_store.cc:276] 0709 13:13:27.668627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.673495 (+1004868us) service_pool.cc:224] Handling call
0709 13:13:28.931824 (+258329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.932777  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:28.342965 10249 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:28.647040  8055 rpcz_store.cc:276] 0709 13:13:26.673612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.675323 (+  1711us) service_pool.cc:224] Handling call
0709 13:13:28.284206 (+1608883us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.699519  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:28.859009 10258 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:28.858664  9853 consensus_queue.cc:260] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:28.859596 10081 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:28.862955  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:28.707556 10087 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:28.865526 10349 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:28.866000 10217 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:28.669634  8204 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:28.866833  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:28.880255  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:28.881515 10039 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:28.734184  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.044128  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:29.044461  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:28.898159  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:28.954592 10207 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:28.902125  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:28.904618  8722 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:28.908743  8211 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
W20260709 13:13:28.866994 10323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.057s	sys 0.052s
I20260709 13:13:28.930389  8212 rpcz_store.cc:276] 0709 13:13:27.066659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.699615 (+1632956us) service_pool.cc:224] Handling call
0709 13:13:28.731019 (+ 31404us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:28.747541  8063 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:28.933842  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:28.938989  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:28.707387 10319 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:28.866490 10170 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:28.867668 10269 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:28.945225  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 253979us
I20260709 13:13:29.037771 10218 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:28.750119 10257 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:28.699472 10010 consensus_queue.cc:1059] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.212s
I20260709 13:13:28.870954 10291 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:28.960429  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1228) took 2004 ms. Trace:
I20260709 13:13:29.021482  8132 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:29.024205  8059 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:28.886811  9876 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:28.953452  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:28.918627 10246 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:28.938831 10027 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:28.901839  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:29.040012  8208 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:29.031491 10260 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:29.048697  8204 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2248) took 1380 ms (client timeout 1678 ms). Trace:
I20260709 13:13:29.051375 10325 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:29.021441 10315 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:29.037809  8206 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:29.086386  8207 rpcz_store.cc:276] 0709 13:13:26.956544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.698281 (+1741737us) service_pool.cc:224] Handling call
0709 13:13:28.960354 (+262073us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.045289  8130 raft_consensus.cc:2376] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5. Re-sending same reply.
I20260709 13:13:29.087237  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.087656  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.087860  8207 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:29.050940  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2253) took 1363 ms. Trace:
I20260709 13:13:29.088294  8211 rpcz_store.cc:276] 0709 13:13:27.687447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.677645 (+990198us) service_pool.cc:224] Handling call
0709 13:13:29.050866 (+373221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.088719  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.089090  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.089257  8207 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:29.054073  8061 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:29.057473  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 207) took 2342 ms. Trace:
I20260709 13:13:29.090003  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.090252  8062 rpcz_store.cc:276] 0709 13:13:26.715248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.056537 (+2341289us) service_pool.cc:224] Handling call
0709 13:13:29.056882 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.057076 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.057113 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.057167 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:29.057209 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.057289 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.057409 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.090901  8207 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:29.058485  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.091766  8062 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:29.092139  8212 raft_consensus.cc:2393] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:29.092340  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2101) took 2108 ms. Trace:
I20260709 13:13:29.092726  8062 rpcz_store.cc:276] 0709 13:13:26.984179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.091214 (+2107035us) service_pool.cc:224] Handling call
0709 13:13:29.091493 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.092104 (+   611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.092294 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.035596  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:29.060276  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 126516us
I20260709 13:13:29.095080  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:29.061523  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:29.097056  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:28.867746 10247 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.063269  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:29.050993 10207 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:29.076113 10320 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:29.060515  8055 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ade1b48685164d64a62772d186ac959f"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:13:29.041235 10039 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:29.083940  9876 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.098555  8730 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:29.105966  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.106732  8206 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:29.108256  8061 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:29.098920  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.108924  8212 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:29.109160  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:29.109675  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.076809 10002 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:29.096099  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:29.051223  9853 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:29.063601 10269 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:29.065724  8724 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:13:29.103842  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.103361  9913 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:29.047091  8029 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:29.105463 10320 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.120186  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:29.099830  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.094764 10010 consensus_queue.cc:1243] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:13:29.112004  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2104) took 2127 ms. Trace:
I20260709 13:13:29.098412 10027 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.126637 10027 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:29.080410 10349 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:29.074306 10170 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:29.113044  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:29.102069 10299 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:29.094242  8129 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 1.
I20260709 13:13:29.129521  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.130088  8129 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:29.123908 10325 raft_consensus.cc:2764] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:13:29.131057  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.132123  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.132692 10269 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.133204  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.133785  8056 raft_consensus.cc:2368] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4.
I20260709 13:13:29.133922  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1284) took 1348 ms. Trace:
I20260709 13:13:29.118119  8130 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:29.134749  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1298) took 1197 ms. Trace:
I20260709 13:13:29.101763 10207 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.135049  8130 rpcz_store.cc:276] 0709 13:13:27.937629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.103617 (+1165988us) service_pool.cc:224] Handling call
0709 13:13:29.134696 (+ 31079us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.136049  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.135246 10269 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:29.118130  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.139464  8206 raft_consensus.cc:2410] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:13:29.117412 10319 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:29.119302  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 72278.8us
I20260709 13:13:29.142401  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:29.143987  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:29.086109 10291 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:13:29.085933  8204 rpcz_store.cc:269] 0709 13:13:27.668354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.295145 (+626791us) service_pool.cc:224] Handling call
0709 13:13:29.043058 (+747913us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.110416  8212 raft_consensus.cc:2410] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:29.119079  9913 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:29.091764  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.111195  8055 raft_consensus.cc:606] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:29.123265  8061 rpcz_store.cc:276] 0709 13:13:26.984904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.110924 (+2126020us) service_pool.cc:224] Handling call
0709 13:13:29.111522 (+   598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.111682 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.111706 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.111788 (+    82us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:29.111823 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.111871 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.111958 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":702848}
I20260709 13:13:29.122025 10320 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.103317 10247 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:29.148229  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 422848us
I20260709 13:13:29.151281  8207 raft_consensus.cc:2410] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:13:29.128237  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:29.155946  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1219) took 2362 ms. Trace:
I20260709 13:13:29.169140  8212 rpcz_store.cc:276] 0709 13:13:26.793191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.155311 (+2362120us) service_pool.cc:224] Handling call
0709 13:13:29.155563 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.155731 (+   168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.155766 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.155794 (+    28us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:29.155812 (+    18us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.155844 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.155909 (+    65us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.170877  8212 raft_consensus.cc:3055] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:29.171180  8212 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:29.172453  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2106) took 2129 ms. Trace:
I20260709 13:13:29.172868  8061 rpcz_store.cc:276] 0709 13:13:27.042831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.151110 (+2108279us) service_pool.cc:224] Handling call
0709 13:13:29.172017 (+ 20907us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.172165 (+   148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.172190 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.172227 (+    37us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:29.172266 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.172314 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.172406 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.154279  8059 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:29.174161  8061 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:13:29.174750  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2107) took 2008 ms. Trace:
I20260709 13:13:29.175028  8061 rpcz_store.cc:276] 0709 13:13:27.166511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.173696 (+2007185us) service_pool.cc:224] Handling call
0709 13:13:29.173972 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:29.174530 (+   558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.174704 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.165833  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:29.142071 10319 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:29.174183  8063 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:29.177381  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:29.150022 10353 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.182984  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:29.172084  8212 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.154893  9913 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:29.210842  8212 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:29.211694 10096 consensus_queue.cc:1059] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:29.213456 10096 raft_consensus.cc:3055] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:29.213793 10096 raft_consensus.cc:740] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:29.188036  8211 raft_consensus.cc:2410] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:13:29.210875 10260 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:29.216887 10192 consensus_meta.cc:287] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.547s	user 0.005s	sys 0.001s
I20260709 13:13:29.217275 10192 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.218298  8130 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:29.150123 10325 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:29.220777 10325 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:29.221170 10325 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:29.154968  8055 raft_consensus.cc:614] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:29.151386 10291 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.204753  9734 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:29.205111 10027 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:29.325254  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.205493 10087 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:29.207245  8104 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:29.343120  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 135958us
I20260709 13:13:29.156309 10299 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:29.228981  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.231408  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.134281  8129 rpcz_store.cc:276] 0709 13:13:27.785584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.132963 (+1347379us) service_pool.cc:224] Handling call
0709 13:13:29.133873 (+   910us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:29.250376  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.167416 10010 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:29.314183  8212 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:13:29.321135  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:29.323295 10355 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.112s	user 0.003s	sys 0.000s
I20260709 13:13:29.332515  9913 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.339591 10192 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:29.185479 10319 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:29.179625 10039 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:29.186945 10304 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:29.250460 10353 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:29.347718  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:29.370026  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.370395  8206 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:29.370792  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1212) took 2633 ms. Trace:
I20260709 13:13:29.371313 10353 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:29.371773  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1217) took 2579 ms. Trace:
I20260709 13:13:29.372181  8206 rpcz_store.cc:276] 0709 13:13:26.792591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.140096 (+2347505us) service_pool.cc:224] Handling call
0709 13:13:29.155498 (+ 15402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.370775 (+215277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.371545 (+   770us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b5400110154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:29.371708 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493485952}
I20260709 13:13:29.369726  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:29.373637  8210 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:29.352280  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.348326  8104 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:29.348776  8212 pending_rounds.cc:85] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 2
I20260709 13:13:29.375270  8212 pending_rounds.cc:107] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:13:29.375522  8212 raft_consensus.cc:2889] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:29.375584  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:29.224889 10249 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:29.369076  8137 raft_consensus.cc:2410] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:13:29.374475  8138 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:29.374706  8102 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:29.376643  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:29.150395 10257 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.400431  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1225) took 2605 ms. Trace:
I20260709 13:13:29.400954  8212 rpcz_store.cc:276] 0709 13:13:26.794828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.170537 (+2375709us) service_pool.cc:224] Handling call
0709 13:13:29.170772 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.400113 (+229341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.400373 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.402267  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:29.379750  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:29.318830  9734 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:29.407630  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:29.404498  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 369574us
I20260709 13:13:29.361282 10241 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:29.410420  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.410867 10249 consensus_queue.cc:1059] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:29.411801 10249 raft_consensus.cc:3055] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:29.412050  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 284) took 1744 ms. Trace:
I20260709 13:13:29.412144 10249 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:29.412453  8206 rpcz_store.cc:276] 0709 13:13:27.667252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.411350 (+1744098us) service_pool.cc:224] Handling call
0709 13:13:29.411610 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.411753 (+   143us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.411776 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.411812 (+    36us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:29.411845 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.411895 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.412002 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.412954 10249 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.413163  9876 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.413908 10249 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:29.365797  9655 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:29.415292  9655 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:29.415678  9655 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.423102 10174 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:29.423553 10174 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:29.423996  9913 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:29.424361 10174 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.425443 10174 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:29.427030  8722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 540) took 1692 ms (client timeout 1865 ms). Trace:
W20260709 13:13:29.427488  8722 rpcz_store.cc:269] 0709 13:13:27.734415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.773030 (+ 38615us) service_pool.cc:224] Handling call
0709 13:13:29.426821 (+1653791us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54004d2e54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:29.426965 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848334080}
I20260709 13:13:29.341722 10096 consensus_queue.cc:260] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.430507 10096 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:29.430740  9734 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:29.431226  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:29.432092  8727 raft_consensus.cc:2410] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:13:29.432261  9980 raft_consensus.cc:3055] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:29.432643  9980 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:29.363816 10320 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.433450  9980 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.342428 10027 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:29.434587 10027 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.435541 10320 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:29.435765  8054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1100) took 2511 ms (client timeout 2155 ms). Trace:
W20260709 13:13:29.436169  8054 rpcz_store.cc:269] 0709 13:13:26.924433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.059321 (+134888us) service_pool.cc:224] Handling call
0709 13:13:29.435589 (+2376268us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540043ce54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:29.435690 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245902208}
I20260709 13:13:29.347476 10299 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:29.437870 10299 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:29.438702  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 393) took 1744 ms. Trace:
I20260709 13:13:29.439090  8726 rpcz_store.cc:276] 0709 13:13:27.694876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.703743 (+  8867us) service_pool.cc:224] Handling call
0709 13:13:27.731417 (+ 27674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:28.319843 (+588426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:28.322566 (+  2723us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:28.327093 (+  4527us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:28.329802 (+  2709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.438461 (+1108659us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b540060f754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:29.438513 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.438642 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1315805440}
W20260709 13:13:29.441478  8132 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1299) took 1400 ms (client timeout 1865 ms). Trace:
W20260709 13:13:29.441884  8132 rpcz_store.cc:269] 0709 13:13:28.040732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.292274 (+251542us) service_pool.cc:224] Handling call
0709 13:13:29.441327 (+1149053us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54001315d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:29.441422 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":861234432}
I20260709 13:13:29.442948  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.361119  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:29.445099  8129 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:29.446058 10291 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.473850 10291 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:29.446511  8138 raft_consensus.cc:2393] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:13:29.470543  8057 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1104) took 2388 ms (client timeout 1614 ms). Trace:
I20260709 13:13:29.474601  8723 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: []
W20260709 13:13:29.474889  8057 rpcz_store.cc:269] 0709 13:13:27.082441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:27.626657 (+544216us) service_pool.cc:224] Handling call
0709 13:13:29.470348 (+1843691us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b5400052154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:29.470464 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1111219072}
I20260709 13:13:29.475947  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.447930 10323 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:29.476553  8057 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 2.
I20260709 13:13:29.476730 10323 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.477152 10323 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.477495  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.448271 10325 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.463979  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1213) took 2725 ms. Trace:
I20260709 13:13:29.480724 10325 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:29.480086  8205 rpcz_store.cc:276] 0709 13:13:26.738123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.724493 (+1986370us) service_pool.cc:224] Handling call
0709 13:13:28.724767 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:28.725078 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:28.725106 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:28.725542 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 13:13:29.146112 (+420570us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:29.146146 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.146205 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:29.463709 (+317504us) raft_consensus.cc:1685] finished
0709 13:13:29.463734 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.463905 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35464,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":4514,"thread_start_us":420248,"threads_started":1,"wal-append.queue_time_us":735362}
I20260709 13:13:29.481726  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:29.468995 10218 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:13:29.484699  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:29.450047  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:29.431319  8133 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1287) took 1645 ms (client timeout 1968 ms). Trace:
W20260709 13:13:29.488469  8133 rpcz_store.cc:269] 0709 13:13:27.786326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.282368 (+496042us) service_pool.cc:224] Handling call
0709 13:13:29.431162 (+1148794us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b54003c8cd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:29.431258 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":694631168}
I20260709 13:13:29.489681  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:29.490300  8133 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:13:29.358367 10245 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:29.371186  8203 rpcz_store.cc:276] 0709 13:13:26.737852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.705385 (+1967533us) service_pool.cc:224] Handling call
0709 13:13:28.705695 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:28.706043 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:28.706077 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:28.706528 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:13:29.050083 (+343555us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:29.050161 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.056465 (+  6304us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:29.370590 (+314125us) raft_consensus.cc:1685] finished
0709 13:13:29.370609 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.370733 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6544,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":72,"thread_start_us":335686,"threads_started":1,"wal-append.queue_time_us":473091}
I20260709 13:13:29.446889 10039 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:29.491685  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:29.409101  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 283) took 1763 ms. Trace:
I20260709 13:13:29.492370  8203 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:29.492311  8212 rpcz_store.cc:276] 0709 13:13:27.666868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.404293 (+1737425us) service_pool.cc:224] Handling call
0709 13:13:29.408508 (+  4215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.408715 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.408761 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.408813 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:29.408863 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.408929 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.409042 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.493594  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.360620  8204 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:13:29.437538  8054 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
W20260709 13:13:29.463573  8208 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55942 (request call id 279) took 1830 ms (client timeout 1911 ms). Trace:
I20260709 13:13:29.495079  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2116) took 1462 ms. Trace:
I20260709 13:13:29.399979 10358 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:13:29.494944  8208 rpcz_store.cc:269] 0709 13:13:27.633597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.306858 (+673261us) service_pool.cc:224] Handling call
0709 13:13:29.463339 (+1156481us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b54002be654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:29.463506 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":990597888}
I20260709 13:13:29.357970  8055 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:13:29.433475 10247 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:29.360401 10207 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.496335  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:29.496784  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:29.497370  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.498627  8208 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:29.467784 10010 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:29.494863  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1222) took 2700 ms. Trace:
I20260709 13:13:29.502856 10010 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:29.503213  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.503137  8204 rpcz_store.cc:276] 0709 13:13:26.794010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.169955 (+2375945us) service_pool.cc:224] Handling call
0709 13:13:29.319480 (+149525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.494531 (+175051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.494796 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.503877  8057 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:29.504621  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.504807  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 286) took 1837 ms. Trace:
I20260709 13:13:29.505242  9655 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.505570  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.498662  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2112) took 1631 ms. Trace:
I20260709 13:13:29.506592  8055 rpcz_store.cc:276] 0709 13:13:27.867263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.357326 (+1490063us) service_pool.cc:224] Handling call
0709 13:13:29.357636 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.498426 (+140790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.498612 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.507361  9876 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:29.507673  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.507939  9876 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:29.508304  9876 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:29.457720  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.508653  9876 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.508913  8130 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:29.509362  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1289) took 1607 ms. Trace:
I20260709 13:13:29.509644  8130 rpcz_store.cc:276] 0709 13:13:27.901884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.457483 (+1555599us) service_pool.cc:224] Handling call
0709 13:13:29.509309 (+ 51826us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.510541  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.511198  8130 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:29.512151  8130 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:29.512672  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1276) took 1729 ms. Trace:
I20260709 13:13:29.497877 10207 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:29.513016  8130 rpcz_store.cc:276] 0709 13:13:27.783480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.511775 (+1728295us) service_pool.cc:224] Handling call
0709 13:13:29.512007 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.512458 (+   451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.512618 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.466825  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.514871  8131 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:29.515350  8210 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:29.515558  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1286) took 1729 ms. Trace:
W20260709 13:13:29.515908  8210 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2257) took 1646 ms (client timeout 1528 ms). Trace:
I20260709 13:13:29.515973  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:29.516371  8210 rpcz_store.cc:269] 0709 13:13:27.869277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.304579 (+435302us) service_pool.cc:224] Handling call
0709 13:13:29.369555 (+1064976us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b5800168398. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:29.515846 (+146291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10167040}
I20260709 13:13:29.497852  9980 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:29.495513  8054 rpcz_store.cc:276] 0709 13:13:28.032513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.436993 (+1404480us) service_pool.cc:224] Handling call
0709 13:13:29.437299 (+   306us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:29.494853 (+ 57554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.495034 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.519191  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:29.519680  8134 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 250) took 3687 ms (client timeout 1911 ms). Trace:
I20260709 13:13:29.520143  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:29.520079  8134 rpcz_store.cc:269] 0709 13:13:25.832250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:25.833111 (+   861us) service_pool.cc:224] Handling call
0709 13:13:29.519527 (+3686416us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b54004097d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:29.519617 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5333502720}
I20260709 13:13:29.501997  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.521632  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1288) took 1620 ms. Trace:
I20260709 13:13:29.521733  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.522235  8205 raft_consensus.cc:2410] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:13:29.522356  8054 raft_consensus.cc:2376] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4. Re-sending same reply.
I20260709 13:13:29.465232  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:29.523115  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.523694  8211 raft_consensus.cc:2368] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5.
I20260709 13:13:29.524061  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.524570  8054 raft_consensus.cc:2376] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6. Re-sending same reply.
I20260709 13:13:29.524595  8211 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:29.516590  8056 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:29.525333  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:29.525900  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.525338  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2254) took 1802 ms. Trace:
I20260709 13:13:29.526831  8211 rpcz_store.cc:276] 0709 13:13:27.722523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.524197 (+1801674us) service_pool.cc:224] Handling call
0709 13:13:29.524445 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.525144 (+   699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.525294 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.527022 10246 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.491798 10039 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:29.527674  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2100) took 2543 ms. Trace:
I20260709 13:13:29.527940 10039 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:29.528059  8063 rpcz_store.cc:276] 0709 13:13:26.983938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.061696 (+2077758us) service_pool.cc:224] Handling call
0709 13:13:29.174050 (+112354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.181198 (+  7148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.527506 (+346308us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400219650. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:29.527622 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257642752}
I20260709 13:13:29.528558  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 288) took 1624 ms. Trace:
I20260709 13:13:29.529381  8132 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:29.528936  8211 rpcz_store.cc:276] 0709 13:13:27.904218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.527888 (+1623670us) service_pool.cc:224] Handling call
0709 13:13:29.528131 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.528279 (+   148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.528302 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.528339 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:29.528372 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.528423 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.528510 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.530405 10246 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:29.522024  8134 rpcz_store.cc:276] 0709 13:13:27.901550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.520955 (+1619405us) service_pool.cc:224] Handling call
0709 13:13:29.521193 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.521345 (+   152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.521367 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.521401 (+    34us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:29.521429 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.521482 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.521582 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.531888 10263 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:29.532222  8134 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:29.533506  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1292) took 1598 ms. Trace:
I20260709 13:13:29.535410  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1282) took 1750 ms. Trace:
I20260709 13:13:29.535764  8137 rpcz_store.cc:276] 0709 13:13:27.785077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.044212 (+1259135us) service_pool.cc:224] Handling call
0709 13:13:29.535220 (+491008us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b540036ef54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:29.535358 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":744510336}
I20260709 13:13:29.533805  8134 rpcz_store.cc:276] 0709 13:13:27.935616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.531831 (+1596215us) service_pool.cc:224] Handling call
0709 13:13:29.532068 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.533300 (+  1232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.533458 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.537077  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1293) took 1601 ms. Trace:
I20260709 13:13:29.537657  8129 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:29.486088  8029 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:29.539217  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 53226.3us
I20260709 13:13:29.539364  8129 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:29.540009  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1295) took 1603 ms. Trace:
I20260709 13:13:29.540062  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:29.540753 10308 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:29.541086  9853 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:29.541699  9853 raft_consensus.cc:697] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:29.542112  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.542390  8131 rpcz_store.cc:276] 0709 13:13:27.786073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.514161 (+1728088us) service_pool.cc:224] Handling call
0709 13:13:29.514499 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.515300 (+   801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.515499 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.542505  9853 consensus_queue.cc:237] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.544226  8131 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:13:29.544857  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1301) took 1389 ms. Trace:
I20260709 13:13:29.545177  8131 rpcz_store.cc:276] 0709 13:13:28.155590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.543499 (+1387909us) service_pool.cc:224] Handling call
0709 13:13:29.543862 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.544615 (+   753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.544804 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.453794 10027 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.455837  8026 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:29.546285  8131 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [2.2-3.3]
W20260709 13:13:29.600337  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 472144us
I20260709 13:13:29.609402  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2111) took 1874 ms. Trace:
I20260709 13:13:29.537400  8137 rpcz_store.cc:276] 0709 13:13:27.935991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.536428 (+1600437us) service_pool.cc:224] Handling call
0709 13:13:29.536656 (+   228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.536806 (+   150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.536830 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.536865 (+    35us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:29.536892 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.536937 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.537032 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.514148  8130 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:29.547446 10358 raft_consensus.cc:697] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:29.611207  8206 raft_consensus.cc:1240] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:29.611414  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1280) took 1826 ms. Trace:
I20260709 13:13:29.611742  8130 rpcz_store.cc:276] 0709 13:13:27.784558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.513743 (+1729185us) service_pool.cc:224] Handling call
0709 13:13:29.514003 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.611107 (+ 97104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.611353 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:29.546586  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 171928us
I20260709 13:13:29.611840 10358 consensus_queue.cc:237] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.612835  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 285) took 1945 ms. Trace:
I20260709 13:13:29.613170  8206 rpcz_store.cc:276] 0709 13:13:27.667542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.413323 (+1745781us) service_pool.cc:224] Handling call
0709 13:13:29.413564 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.611622 (+198058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.612651 (+  1029us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b540014a354. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:29.612788 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":454181760}
W20260709 13:13:29.570751  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 249669us
I20260709 13:13:29.614502 10027 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:29.563527  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1220) took 2770 ms. Trace:
I20260709 13:13:29.529206  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.547900  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.556874  8135 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:13:29.574119 10319 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.617867  8212 raft_consensus.cc:2393] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:29.457723  9734 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.618158  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 961) took 1360 ms. Trace:
I20260709 13:13:29.549425  9655 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:29.618607  8135 rpcz_store.cc:276] 0709 13:13:28.257990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.556266 (+1298276us) service_pool.cc:224] Handling call
0709 13:13:29.556607 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:29.617868 (+ 61261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.618106 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.619122 10319 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:29.469689 10299 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.620083  8135 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:29.601106 10087 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:29.620716  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 967) took 1268 ms. Trace:
I20260709 13:13:29.620903  9734 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:29.558708 10304 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:29.547051  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.647971  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:29.648514  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 287) took 1980 ms. Trace:
I20260709 13:13:29.649353  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.649209  8210 rpcz_store.cc:276] 0709 13:13:27.668080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.517244 (+1849164us) service_pool.cc:224] Handling call
0709 13:13:29.517507 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.517690 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:29.636090 (+118400us) log.cc:844] Serialized 14 byte log entry
0709 13:13:29.648116 (+ 12026us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.648177 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:29.648259 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.648350 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.648467 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":108730,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":865,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":73,"thread_start_us":129820,"threads_started":2,"wal-append.queue_time_us":11073}
I20260709 13:13:29.547473  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 951) took 2380 ms. Trace:
I20260709 13:13:29.621095 10087 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.568926 10010 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.652513  8138 rpcz_store.cc:276] 0709 13:13:27.167103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.475281 (+2308178us) service_pool.cc:224] Handling call
0709 13:13:29.476063 (+   782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.547130 (+ 71067us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:29.547168 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.547207 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:29.547240 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.547319 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.547421 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70695,"raft.queue_time_us":70846,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92}
I20260709 13:13:29.576740  8203 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:29.654168 10010 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:29.559520  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1304) took 1352 ms. Trace:
I20260709 13:13:29.654876  8132 rpcz_store.cc:276] 0709 13:13:28.207117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.442649 (+1235532us) service_pool.cc:224] Handling call
0709 13:13:29.559298 (+116649us) spinlock_profiling.cc:243] Waited 85.7 ms on lock 0x7b54001f09d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:29.559451 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":197133184}
I20260709 13:13:29.656509  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 258) took 1223 ms. Trace:
I20260709 13:13:29.622921  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:29.595877  8209 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.013s	sys 0.020s
W20260709 13:13:29.657231  8209 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.013s	sys 0.022s
I20260709 13:13:29.658630  8132 rpcz_store.cc:276] 0709 13:13:28.432657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.655693 (+1223036us) service_pool.cc:224] Handling call
0709 13:13:29.655972 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.656139 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.656163 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.656200 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:29.656259 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.656331 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.656450 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:29.572502  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.623745 10325 raft_consensus.cc:2764] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:13:29.573707  8133 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
W20260709 13:13:29.661978  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.662650  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1214) took 2924 ms. Trace:
I20260709 13:13:29.662246  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 259) took 1229 ms. Trace:
I20260709 13:13:29.540386  8129 rpcz_store.cc:276] 0709 13:13:27.936655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.538718 (+1602063us) service_pool.cc:224] Handling call
0709 13:13:29.539031 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.539770 (+   739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.539959 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.664167  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:29.663316  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 255) took 1272 ms. Trace:
W20260709 13:13:29.599956  7956 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34134 (request call id 263) took 14785 ms (client timeout 15000 ms). Trace:
I20260709 13:13:29.663950  8133 rpcz_store.cc:276] 0709 13:13:28.432964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.661529 (+1228565us) service_pool.cc:224] Handling call
0709 13:13:29.661806 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.661958 (+   152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.661984 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.662024 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:29.662055 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.662111 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.662198 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.526293  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.666353  8054 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:29.667410  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:29.668062  8054 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.667490  8138 rpcz_store.cc:276] 0709 13:13:28.390852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.654588 (+1263736us) service_pool.cc:224] Handling call
0709 13:13:29.654879 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.655081 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:29.655332 (+   251us) log.cc:844] Serialized 14 byte log entry
0709 13:13:29.663022 (+  7690us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.663065 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:29.663103 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.663160 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.663268 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":824,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":1605,"raft.run_cpu_time_us":70,"raft.run_wall_time_us":70,"thread_start_us":7417,"threads_started":1,"wal-append.queue_time_us":6542}
I20260709 13:13:29.573958  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2115) took 1541 ms. Trace:
I20260709 13:13:29.576768  9876 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.670578  8060 rpcz_store.cc:276] 0709 13:13:28.032199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.429239 (+1397040us) service_pool.cc:224] Handling call
0709 13:13:29.429586 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:29.429956 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.429984 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:29.430492 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:13:29.573650 (+143158us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:29.573686 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.573749 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:29.573791 (+    42us) raft_consensus.cc:1685] finished
0709 13:13:29.573807 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.573911 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1888,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"thread_start_us":142062,"threads_started":1,"wal-append.queue_time_us":136905}
I20260709 13:13:29.671919  9876 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:29.585881  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2102) took 2601 ms. Trace:
I20260709 13:13:29.672508  8062 rpcz_store.cc:276] 0709 13:13:26.984428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.093571 (+2109143us) service_pool.cc:224] Handling call
0709 13:13:29.475237 (+381666us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:29.475624 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.475652 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:29.476157 (+   505us) log.cc:844] Serialized 52 byte log entry
0709 13:13:29.573955 (+ 97798us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:29.573994 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.574065 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:29.584879 (+ 10814us) raft_consensus.cc:1685] finished
0709 13:13:29.584905 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.585595 (+   690us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b540043ce50. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:29.585814 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":375,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":877057408,"thread_start_us":97431,"threads_started":1,"wal-append.queue_time_us":94721}
I20260709 13:13:29.674043  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:29.675031  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:29.675470  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:29.675993  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:29.676102  8723 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:29.676949  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.677596  8062 raft_consensus.cc:2393] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:29.678205  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.679093  8062 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:29.678973 10010 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:29.680040  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.680361  8054 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:29.610101  8058 rpcz_store.cc:276] 0709 13:13:27.735051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.343030 (+1607979us) service_pool.cc:224] Handling call
0709 13:13:29.401715 (+ 58685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.401978 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:29.572941 (+170963us) log.cc:844] Serialized 14 byte log entry
0709 13:13:29.609099 (+ 36158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.609144 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:29.609184 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.609244 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.609352 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":168193,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":69,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":177949,"threads_started":2,"wal-append.queue_time_us":34544}
I20260709 13:13:29.630921 10039 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.680709  8062 raft_consensus.cc:2376] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5. Re-sending same reply.
I20260709 13:13:29.681548  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:29.682114  8058 raft_consensus.cc:2376] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5. Re-sending same reply.
I20260709 13:13:29.682603 10039 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:29.631868  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 71035.1us
I20260709 13:13:29.632617 10269 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:29.683444  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2121) took 1649 ms. Trace:
I20260709 13:13:29.633227  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2110) took 1905 ms. Trace:
I20260709 13:13:29.683820  8058 rpcz_store.cc:276] 0709 13:13:28.033974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.682690 (+1648716us) service_pool.cc:224] Handling call
0709 13:13:29.682959 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.683125 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.683150 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.683212 (+    62us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:29.683251 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.683301 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.683395 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.506256  8057 raft_consensus.cc:2376] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6. Re-sending same reply.
I20260709 13:13:29.615828  8207 rpcz_store.cc:276] 0709 13:13:26.793465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.167479 (+2374014us) service_pool.cc:224] Handling call
0709 13:13:29.167793 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:29.168096 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.168121 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:29.168539 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 13:13:29.563086 (+394547us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:29.563140 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.563252 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:29.563306 (+    54us) raft_consensus.cc:1685] finished
0709 13:13:29.563323 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.563464 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36845,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":74,"thread_start_us":250041,"threads_started":1,"wal-append.queue_time_us":278542}
I20260709 13:13:29.635056  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2109) took 1924 ms. Trace:
I20260709 13:13:29.686107  8061 rpcz_store.cc:276] 0709 13:13:27.710308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.175686 (+1465378us) service_pool.cc:224] Handling call
0709 13:13:29.175979 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.183498 (+  7519us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:29.600555 (+417057us) log.cc:844] Serialized 14 byte log entry
0709 13:13:29.634316 (+ 33761us) log.cc:844] Serialized 14 byte log entry
0709 13:13:29.634719 (+   403us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.634784 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:29.634840 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.634910 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.635004 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1328,"prepare.queue_time_us":387653,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":169,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":148,"spinlock_wait_cycles":34176,"thread_start_us":428727,"threads_started":2,"wal-append.queue_time_us":30291}
I20260709 13:13:29.610227  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:29.596073 10323 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.613157  8130 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:13:29.615566  8696 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:29.701112  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 966) took 1370 ms. Trace:
W20260709 13:13:29.701295  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 85782.6us
I20260709 13:13:29.701557  8130 rpcz_store.cc:276] 0709 13:13:28.330867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.612686 (+1281819us) service_pool.cc:224] Handling call
0709 13:13:29.612960 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:29.700861 (+ 87901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.701049 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.629452  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2251) took 1942 ms. Trace:
I20260709 13:13:29.702483  8205 rpcz_store.cc:276] 0709 13:13:27.686843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.522887 (+1836044us) service_pool.cc:224] Handling call
0709 13:13:29.523155 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:29.552189 (+ 29034us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.552231 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:29.552734 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:13:29.627505 (+ 74771us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:29.627544 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.627604 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:29.629248 (+  1644us) raft_consensus.cc:1685] finished
0709 13:13:29.629272 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.629403 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28626,"raft.queue_time_us":29782,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":74477,"threads_started":1,"wal-append.queue_time_us":74756}
I20260709 13:13:29.651048  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:29.651662  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 290) took 1723 ms. Trace:
I20260709 13:13:29.651763 10304 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:13:29.621981  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 166231us
I20260709 13:13:29.714720  8026 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:29.579380  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 202766us
I20260709 13:13:29.621064  8135 rpcz_store.cc:276] 0709 13:13:28.352829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.619651 (+1266822us) service_pool.cc:224] Handling call
0709 13:13:29.619902 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.620522 (+   620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.620674 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:29.718686  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 338880us
I20260709 13:13:29.719856  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:29.720597  8207 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:29.576939  8211 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:29.722950  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.723790  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:29.723984  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.724553  8730 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:29.725584  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:29.663115  8209 rpcz_store.cc:276] 0709 13:13:26.738411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.932740 (+2194329us) service_pool.cc:224] Handling call
0709 13:13:28.933022 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:28.933319 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:28.933346 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:28.933759 (+   413us) log.cc:844] Serialized 26 byte log entry
0709 13:13:29.657570 (+723811us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:29.657606 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.662400 (+  4794us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:29.662461 (+    61us) raft_consensus.cc:1685] finished
0709 13:13:29.662481 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.662598 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1107,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":82959,"thread_start_us":619545,"threads_started":1,"wal-append.queue_time_us":436642}
I20260709 13:13:29.624325 10299 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:29.600967  7959 catalog_manager.cc:5697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 3, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 3 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:29.665163 10358 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:29.672111  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.728781  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.628803  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 960) took 1371 ms. Trace:
I20260709 13:13:29.729615  8207 raft_consensus.cc:2410] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:13:29.729816  8131 rpcz_store.cc:276] 0709 13:13:28.257711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.545795 (+1288084us) service_pool.cc:224] Handling call
0709 13:13:29.546080 (+   285us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:29.628558 (+ 82478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.628756 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.546798 10081 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:29.616753  8063 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:29.505169  8204 rpcz_store.cc:276] 0709 13:13:27.667818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.504058 (+1836240us) service_pool.cc:224] Handling call
0709 13:13:29.504347 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.504506 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.504530 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.504568 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:29.504603 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.504659 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.504761 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:29.680867  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.732556  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.729983  7959 catalog_manager.cc:5697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:29.680704 10319 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:29.688908  8057 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:29.734237  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2134) took 1345 ms. Trace:
I20260709 13:13:29.733847  7959 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 2 to 3, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 3 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:13:29.665676  7956 rpcz_store.cc:269] 0709 13:13:14.814324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:14.814711 (+   387us) service_pool.cc:224] Handling call
0709 13:13:29.599859 (+14785148us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:23.015382 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:23.199384 (+184002us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:23.199451 (+    67us) write_op.cc:435] Acquired schema lock
0709 13:13:23.199471 (+    20us) tablet.cc:662] Decoding operations
0709 13:13:23.200148 (+   677us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:23.200233 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:23.200262 (+    29us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:13:23.200803 (+   541us) tablet.cc:701] Row locks acquired
0709 13:13:23.200861 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:13:23.201010 (+   149us) write_op.cc:270] Start()
0709 13:13:23.201141 (+   131us) write_op.cc:276] Timestamp: P: 1783602803200949 usec, L: 0
0709 13:13:23.201176 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:13:23.236701 (+ 35525us) log.cc:844] Serialized 1684 byte log entry
0709 13:13:23.768902 (+532201us) op_driver.cc:464] REPLICATION: finished
0709 13:13:24.205524 (+436622us) write_op.cc:301] APPLY: starting
0709 13:13:24.226489 (+ 20965us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:24.261290 (+ 34801us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:24.334076 (+ 72786us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:24.381343 (+ 47267us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:24.381600 (+   257us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:13:24.381645 (+    45us) write_op.cc:312] APPLY: finished
0709 13:13:24.446777 (+ 65132us) log.cc:844] Serialized 41 byte log entry
0709 13:13:24.547261 (+100484us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:24.561883 (+ 14622us) write_op.cc:454] Released schema lock
0709 13:13:24.585221 (+ 23338us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:24.628562 (+ 43341us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:24.631293 (+  2731us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":47913,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":281859,"apply.run_cpu_time_us":6298,"apply.run_wall_time_us":507029,"num_ops":4,"prepare.queue_time_us":346067,"prepare.run_cpu_time_us":16880,"prepare.run_wall_time_us":467218,"raft.queue_time_us":174412,"raft.run_cpu_time_us":13829,"raft.run_wall_time_us":177490,"replication_time_us":567543,"thread_start_us":425531,"threads_started":4,"wal-append.queue_time_us":161793,"wal-append.run_cpu_time_us":20958,"wal-append.run_wall_time_us":2174609}]]}
I20260709 13:13:29.695297  8054 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:29.748521  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2136) took 1358 ms. Trace:
I20260709 13:13:29.748881  8054 rpcz_store.cc:276] 0709 13:13:28.389777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.694807 (+1305030us) service_pool.cc:224] Handling call
0709 13:13:29.695107 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.748274 (+ 53167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.748460 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.683917  8059 rpcz_store.cc:276] 0709 13:13:27.727769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.175702 (+1447933us) service_pool.cc:224] Handling call
0709 13:13:29.175979 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.176149 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:29.598440 (+422291us) log.cc:844] Serialized 14 byte log entry
0709 13:13:29.632298 (+ 33858us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.632361 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:29.632420 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.633030 (+   610us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.633165 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":392367,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":72,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":66,"spinlock_wait_cycles":17664,"thread_start_us":446606,"threads_started":2,"wal-append.queue_time_us":31611}
I20260709 13:13:29.700086  8061 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:29.750818  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2124) took 1647 ms. Trace:
I20260709 13:13:29.751296  8061 rpcz_store.cc:276] 0709 13:13:28.103664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.687696 (+1584032us) service_pool.cc:224] Handling call
0709 13:13:29.699884 (+ 12188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.750527 (+ 50643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.750755 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.705120  8208 rpcz_store.cc:276] 0709 13:13:27.928713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.543813 (+1615100us) service_pool.cc:224] Handling call
0709 13:13:29.544052 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.544242 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:29.633204 (+ 88962us) log.cc:844] Serialized 14 byte log entry
0709 13:13:29.651339 (+ 18135us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.651388 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:29.651439 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.651495 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.651602 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2112,"prepare.queue_time_us":82366,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"raft.queue_time_us":2902,"raft.run_cpu_time_us":59,"raft.run_wall_time_us":59,"thread_start_us":104351,"threads_started":2,"wal-append.queue_time_us":14548}
W20260709 13:13:29.712725  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 102565us
I20260709 13:13:29.753315  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:29.754284  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:29.749950  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:29.755204  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:29.713182  8101 leader_election.cc:400] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4.
I20260709 13:13:29.756038  8101 leader_election.cc:403] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:13:29.756870  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 43801.7us
I20260709 13:13:29.757727  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:29.610272  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:29.734566  8057 rpcz_store.cc:276] 0709 13:13:28.389276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.688415 (+1299139us) service_pool.cc:224] Handling call
0709 13:13:29.688641 (+   226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.734012 (+ 45371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.734187 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.689432  8029 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:29.758893  8056 raft_consensus.cc:2376] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6. Re-sending same reply.
W20260709 13:13:29.759232  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69870.7us
I20260709 13:13:29.735791  8058 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:29.729542  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.759970  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2122) took 1725 ms. Trace:
I20260709 13:13:29.732048  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.719990  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 963) took 1404 ms. Trace:
I20260709 13:13:29.728230 10027 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.735960 10319 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:29.738896  8210 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:29.738972  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:29.739271  8730 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:13:29.738972  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:29.735363  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:29.768482  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:29.736078 10081 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:29.741271  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.741503  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.712343  9980 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:13:29.742730 10370 consensus_queue.cc:1059] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:29.744760  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.777398 10241 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:29.745328 10269 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:29.745745  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:29.712430 10039 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:29.746760  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:29.719055 10170 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:29.739831 10299 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:29.729496  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 289) took 1809 ms. Trace:
I20260709 13:13:29.857761  8211 rpcz_store.cc:276] 0709 13:13:27.920192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.530084 (+1609892us) service_pool.cc:224] Handling call
0709 13:13:29.530319 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.722534 (+192215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.729440 (+  6906us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.858961  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:29.860131  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:29.860944  8211 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 6: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:29.757051  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.779196 10258 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:13:29.728308 10217 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:29.761725  8029 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:29.836238 10241 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:29.760305  8058 rpcz_store.cc:276] 0709 13:13:28.034286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.685029 (+1650743us) service_pool.cc:224] Handling call
0709 13:13:29.735626 (+ 50597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.759771 (+ 24145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.759925 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.763819  8206 raft_consensus.cc:2410] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:29.870528  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:29.751302  7959 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 2 to 3, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 3 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:29.769016  8207 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.866495 10170 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.874413  9734 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:29.874749 10170 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:29.875795 10170 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:29.876212  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:29.876775  8058 raft_consensus.cc:2376] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4. Re-sending same reply.
I20260709 13:13:29.876984  8054 raft_consensus.cc:2376] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6. Re-sending same reply.
I20260709 13:13:29.877429 10323 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:29.877784  8058 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:29.878646  8054 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:29.849253 10319 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.879698 10319 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.765950  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.880995  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.881668  8130 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:29.882375  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:29.860186  8175 connection.cc:256] RPC call timeout handler was delayed by -1.9919351169999118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32820000000000005s for a response
I20260709 13:13:29.883379  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:29.741572  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.778532  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:29.777154  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 54929.6us
I20260709 13:13:29.779493  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.812960  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:29.771338  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:29.824620  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.769325 10286 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4.
W20260709 13:13:29.861891  8104 connection.cc:256] RPC call timeout handler was delayed by -1.4827659310001309s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:13:29.884052  8130 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:29.812217  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:29.901942  8210 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:29.769184 10012 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:29.827396  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1294) took 1891 ms. Trace:
I20260709 13:13:29.902647 10012 raft_consensus.cc:697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 6 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:29.902581  8134 rpcz_store.cc:276] 0709 13:13:27.936313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.537178 (+1600865us) service_pool.cc:224] Handling call
0709 13:13:29.537421 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.537607 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:29.714610 (+177003us) log.cc:844] Serialized 14 byte log entry
0709 13:13:29.827085 (+112475us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:29.827130 (+    45us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:29.827171 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.827231 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.827342 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6078,"prepare.queue_time_us":59825,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":83,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":4757,"thread_start_us":288906,"threads_started":2,"wal-append.queue_time_us":150492}
I20260709 13:13:29.903589 10012 consensus_queue.cc:237] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.904578  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:29.906523  8724 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:13:29.899371  8055 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:29.908721  8131 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:29.909009  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.909590  8133 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:29.869848  9853 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.910497  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.910904  9853 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.911577  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:29.911827  8207 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:29.912563  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:29.777396 10081 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:29.778299 10308 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:29.918543 10308 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:29.918702  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 147443us
I20260709 13:13:29.918960 10308 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:29.919152 10039 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:29.919782  8101 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:29.919927 10039 raft_consensus.cc:3055] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:29.920308 10039 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:29.915109  7959 catalog_manager.cc:5697] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 4, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 4 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:29.921065 10039 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.717475 10304 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:29.831719  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:29.927943  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:29.832372  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 117725us
I20260709 13:13:29.929467  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:29.930487  8026 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:29.931947  8026 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:29.782465 10207 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:13:29.834689  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.839470  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:29.850669  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.944051  8057 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:29.773017 10027 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.946578 10394 raft_consensus.cc:3055] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:29.947110 10394 raft_consensus.cc:740] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:29.947978 10207 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:29.947886 10394 consensus_queue.cc:260] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.949076 10394 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.948915 10207 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:29.950083  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:29.856479  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:29.951664  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.951565  8061 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:29.833436 10174 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:29.864043  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:29.737854  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:29.835539 10269 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:29.760622  8137 rpcz_store.cc:276] 0709 13:13:28.315949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.610470 (+1294521us) service_pool.cc:224] Handling call
0709 13:13:29.610746 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:29.611281 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.611306 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:29.611747 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 13:13:29.719698 (+107951us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:29.719736 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.719799 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:29.719835 (+    36us) raft_consensus.cc:1685] finished
0709 13:13:29.719846 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:29.719943 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":106126,"threads_started":1,"wal-append.queue_time_us":98414}
I20260709 13:13:29.962520  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.963518  8137 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:29.871179  8063 raft_consensus.cc:2376] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6. Re-sending same reply.
I20260709 13:13:29.922181  7959 catalog_manager.cc:5697] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 4, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 4 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:29.836272  9876 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:29.869217 10217 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:29.894719  8135 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.854743  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:29.845957  8203 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:29.892642  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:29.971940  8132 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
W20260709 13:13:29.884007  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6297960529998363s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
W20260709 13:13:29.973779  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6072960529998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
W20260709 13:13:29.974107  8175 connection.cc:256] RPC call timeout handler was delayed by -1.4335960529999738s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18320000000000003s for a response
W20260709 13:13:29.974503  8175 connection.cc:256] RPC call timeout handler was delayed by -1.3516960529998414s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19230000000000003s for a response
W20260709 13:13:29.974779  8175 connection.cc:256] RPC call timeout handler was delayed by -1.3444960529998298s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
W20260709 13:13:29.974987  8175 connection.cc:256] RPC call timeout handler was delayed by -1.3309960529998079s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1946s for a response
W20260709 13:13:29.975235  8175 connection.cc:256] RPC call timeout handler was delayed by -1.3228960530000222s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
W20260709 13:13:29.975390  8175 connection.cc:256] RPC call timeout handler was delayed by -1.316596053000012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1962s for a response
W20260709 13:13:29.975533  8175 connection.cc:256] RPC call timeout handler was delayed by -1.2652960529999291s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20190000000000002s for a response
W20260709 13:13:29.975670  8175 connection.cc:256] RPC call timeout handler was delayed by -1.2328960529998767s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20550000000000002s for a response
W20260709 13:13:29.975813  8175 connection.cc:256] RPC call timeout handler was delayed by -1.1482960529999673s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2149s for a response
W20260709 13:13:29.975951  8175 connection.cc:256] RPC call timeout handler was delayed by -1.1428960529999586s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2155s for a response
W20260709 13:13:29.976086  8175 connection.cc:256] RPC call timeout handler was delayed by -1.0753960529998494s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.223s for a response
I20260709 13:13:29.763059 10325 raft_consensus.cc:2764] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:13:29.894968  8730 raft_consensus.cc:2376] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5. Re-sending same reply.
I20260709 13:13:29.897486  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:29.898898 10382 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:13:29.962998 10039 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.814239  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:29.900718  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1113) took 1156 ms. Trace:
I20260709 13:13:29.982636  8058 rpcz_store.cc:276] 0709 13:13:28.721559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.877375 (+1155816us) service_pool.cc:224] Handling call
0709 13:13:29.877641 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.878146 (+   505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.878307 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:29.896468  8204 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:29.912858  9853 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:29.889379 10247 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.918095 10286 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:29.831712  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:29.923987  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:29.924853  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:29.926036  8101 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:29.926573  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 60297.5us
I20260709 13:13:29.992926  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:29.927410 10257 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:29.946679 10027 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:29.855406  8136 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 257) took 1423 ms (client timeout 1683 ms). Trace:
W20260709 13:13:29.999317  8136 rpcz_store.cc:269] 0709 13:13:28.432294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:28.651239 (+218945us) service_pool.cc:224] Handling call
0709 13:13:29.848269 (+1197030us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540036e554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:29.855352 (+  7083us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2507252096}
I20260709 13:13:29.953403  8026 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:29.967374 10002 consensus_queue.cc:1059] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:13:29.957324  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:30.002692  8207 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:29.958184  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:29.964699  7959 catalog_manager.cc:5697] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:30.005438  7959 catalog_manager.cc:5697] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 2, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:29.966331  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:30.008188  8137 raft_consensus.cc:2410] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:13:29.894701  8104 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:29.951359 10299 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:30.009656 10299 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:29.962258 10170 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:30.010556 10170 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:30.011050 10170 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:29.985486  8135 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:29.999146 10247 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:29.973178 10217 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:29.989202  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:30.015884  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:30.016212  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:30.016355 10247 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:29.995972  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:30.017280 10174 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:30.017740 10174 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:30.017863  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:30.021765  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:13:29.997301  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 71292.7us
I20260709 13:13:30.023943  8101 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.030429 10361 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:30.031196 10361 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:29.983526  8726 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:30.032439  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2119) took 1999 ms. Trace:
I20260709 13:13:30.032919  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:30.032793  8062 rpcz_store.cc:276] 0709 13:13:28.033394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.682050 (+1648656us) service_pool.cc:224] Handling call
0709 13:13:29.682445 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:29.682830 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:29.682868 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:29.683431 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:13:29.968895 (+285464us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:29.968953 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:29.969040 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:30.032252 (+ 63212us) raft_consensus.cc:1685] finished
0709 13:13:30.032271 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:30.032388 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":468,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":285037,"threads_started":1,"wal-append.queue_time_us":346935}
I20260709 13:13:30.034621  8210 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:30.034718  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:29.998505  8212 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:48498
I20260709 13:13:29.883958 10241 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:29.972947  9876 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:30.000590  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 47265.3us
I20260709 13:13:29.958851 10397 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:29.995088  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:30.041271  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 46274.4us
I20260709 13:13:30.041756  8693 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.042819  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:30.043089 10100 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:30.043749 10100 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:30.044616 10100 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:30.047379 10397 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:30.047838 10397 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:30.001602 10258 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:29.997680 10323 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:30.038991  8207 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:30.066985  8101 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:30.000432  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 238766us
I20260709 13:13:30.068277  8029 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:29.986209  8722 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
W20260709 13:13:30.039757  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 145131us
I20260709 13:13:30.042886 10087 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:30.038143 10027 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:30.007063  7959 catalog_manager.cc:5697] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:30.035772  8212 raft_consensus.cc:606] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:30.094913  8212 raft_consensus.cc:614] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:30.036193 10247 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:30.098070  7959 catalog_manager.cc:5697] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 3, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 3 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:30.100867  7959 catalog_manager.cc:5697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:30.038375  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:30.058982 10370 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:30.105656  7959 catalog_manager.cc:5697] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 3, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 3 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
W20260709 13:13:30.104987  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 67091.7us
I20260709 13:13:30.107596  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.033612 10217 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:30.068372 10361 raft_consensus.cc:2764] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:13:30.092665 10382 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:13:30.114827  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 47926.1us
I20260709 13:13:30.069491  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:30.124222  7959 catalog_manager.cc:5697] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 2 to 4, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 4 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:30.119956  8027 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:13:30.113287  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 62063.6us
I20260709 13:13:30.092516 10257 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:30.157295  8101 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.122674  8104 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.126883 10290 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:13:30.159492 10081 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:30.094795 10325 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:13:30.066720 10371 consensus_queue.cc:1059] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:13:30.051576 10403 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 13:13:30.163797  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:30.164733 10241 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:30.174631  8206 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:30.201246  8212 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:13:30.201573  8212 pending_rounds.cc:85] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 6
I20260709 13:13:30.201769  8212 pending_rounds.cc:107] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:13:30.201969  8212 raft_consensus.cc:2889] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:30.116921 10087 raft_consensus.cc:697] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:30.114394 10012 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:30.120038 10260 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:30.175994 10369 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:30.176487 10406 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:30.178157 10308 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:30.191779 10257 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:30.194587 10356 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:30.191717  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69154.5us
I20260709 13:13:30.234134  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.173439 10304 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:30.194675  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 74806.6us
W20260709 13:13:30.195760  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 127553us
I20260709 13:13:30.197651  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:30.236114  8104 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:30.237624  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 79646.8us
I20260709 13:13:30.224573 10400 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:30.097121 10027 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:30.193776 10002 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:30.196797 10358 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:30.192561  8101 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.195787  8726 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:30.238322  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:30.243571  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 51076.2us
I20260709 13:13:30.243871  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:30.227202 10369 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:30.246243  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:30.233814 10257 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:30.261309  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:30.253288 10325 consensus_queue.cc:1059] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:30.240033  8104 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.249082  9655 raft_consensus.cc:2764] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:13:30.251983 10218 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:30.252597 10217 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:30.252800 10405 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:30.241091 10283 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:30.238201 10260 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:30.255029 10308 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:30.255478 10012 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:30.255434 10269 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:30.246640 10027 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:30.257051 10304 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:30.246042 10188 consensus_queue.cc:1059] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:13:30.258486 10400 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:30.249125  8723 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
W20260709 13:13:30.270051  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 162571us
I20260709 13:13:30.279310  8104 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.228720 10397 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:30.241243 10319 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:30.279510  8101 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:30.236114 10039 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:13:30.289173 10290 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:30.255564 10241 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:30.280612 10291 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:30.280499 10218 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:30.283272 10258 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:30.299964  8176 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:30.306962  8693 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.287730 10405 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:13:30.309517  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 72592.6us
W20260709 13:13:30.311075  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 114852us
W20260709 13:13:30.314644  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:30.262693 10369 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:30.317109  8101 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.317801  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:30.330963  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 68651.4us
I20260709 13:13:30.329686 10188 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:30.308348  9876 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:30.308495 10415 raft_consensus.cc:3055] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:30.289943 10207 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:13:30.335680  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:30.338856  8026 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.331473 10218 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:30.293592 10370 raft_consensus.cc:697] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:30.328263 10291 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:30.328754  9980 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:30.348248  8104 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:30.360322  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:30.301281 10174 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:30.355203  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:30.364652  8061 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:30.300819 10087 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:30.357856  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.340212 10405 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:30.359001  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:30.341537  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:30.347127 10258 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:30.360037  8059 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:30.375448  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:30.349491 10415 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:30.352134  9655 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:30.344767 10319 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:30.344628 10012 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.346925  9734 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:30.367012  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:30.341260 10290 raft_consensus.cc:697] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:30.373345  8061 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
W20260709 13:13:30.406242  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58064.8us
I20260709 13:13:30.373805  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:30.408289  8137 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:30.374442  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:30.409300  8134 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:30.376024  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:30.369195 10081 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:30.359491  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.360307 10291 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:30.398622 10100 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:30.361958 10323 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:30.347302 10403 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:13:30.403566  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.396912 10319 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:30.404397  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:30.354736 10096 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:13:30.356665 10260 consensus_queue.cc:1059] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:13:30.367164 10039 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:30.381297 10308 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:30.413818  8104 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:30.426542  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:30.430732  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:30.349393 10356 consensus_queue.cc:1059] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:13:30.422380  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:13:30.406533 10002 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:30.406641  9734 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:30.409507 10418 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:13:30.444566  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 105842us
W20260709 13:13:30.467125  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 53398.8us
I20260709 13:13:30.364717 10414 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.080s
I20260709 13:13:30.467684  8026 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:30.429368 10100 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:30.406960 10012 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:30.438020 10403 consensus_queue.cc:1243] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:13:30.456840  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:30.429528 10323 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:30.439594 10087 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:30.441205 10319 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:30.360184 10400 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:30.440017  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:30.467046  9734 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:30.466403 10410 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:30.576602 10410 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:30.466025 10416 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:30.469797  9655 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:30.476212 10100 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:30.516235 10290 consensus_queue.cc:237] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:30.515827  9876 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:13:30.477176 10370 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:30.526701 10412 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:13:30.468274 10415 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:30.528443 10419 log.cc:927] Time spent T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:13:30.529183 10218 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:30.532907 10170 consensus_meta.cc:287] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.521s	user 0.007s	sys 0.003s
I20260709 13:13:30.533031 10319 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:30.536152 10394 consensus_meta.cc:287] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.587s	user 0.005s	sys 0.005s
I20260709 13:13:30.526046 10358 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:30.540650 10405 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:30.542184 10291 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:30.549177 10247 raft_consensus.cc:3055] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:30.516474 10397 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:30.481762 10304 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:30.559849 10283 raft_consensus.cc:3055] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:30.562708 10409 raft_consensus.cc:3055] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:30.476364 10269 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:30.485463 10418 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:30.480154 10323 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:30.567902 10081 raft_consensus.cc:3055] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:30.568521 10027 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:30.570554 10012 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:30.485970  8104 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:30.575637 10174 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:30.578158 10410 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:30.590912  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 116058us
I20260709 13:13:30.682709  8131 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:13:30.672081 10247 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:13:30.703539  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 234777us
I20260709 13:13:30.673282 10415 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:30.750546  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 264640us
I20260709 13:13:30.771744  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:30.704373  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 344939us
I20260709 13:13:30.681023 10283 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:30.680585 10409 raft_consensus.cc:740] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:30.660013 10170 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:30.690703 10081 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:30.774307  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:30.770115  8176 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:30.795639  8133 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:30.787724  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:30.788033  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:30.788420  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:30.800454  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:30.730934  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:13:30.814953  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 44955.4us
I20260709 13:13:30.702939 10304 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:30.709280 10434 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:30.734499 10269 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:30.937747  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:30.938617  8057 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:30.772634 10260 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:30.808602 10299 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:30.941275 10174 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:30.728667 10188 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:30.995802  8137 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:30.794653  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:30.968532  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 253583us
W20260709 13:13:30.979923 10427 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:13:30.979846 10269 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:30.980424 10218 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:31.062955  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 126219us
W20260709 13:13:30.984342 10369 consensus_meta.cc:287] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.626s	user 0.009s	sys 0.000s
I20260709 13:13:30.984841 10010 raft_consensus.cc:3055] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:30.985406  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:30.986668  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:30.987560  8132 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:30.988108 10100 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:30.985412  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:30.991881 10027 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:30.992341 10304 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:30.976296 10260 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:30.993528 10039 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:31.115531  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:30.994438 10291 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:31.119925  8138 raft_consensus.cc:2368] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5.
I20260709 13:13:30.994771 10406 raft_consensus.cc:3055] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:30.962253 10418 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:30.995817 10081 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:30.794556  9876 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.169s
I20260709 13:13:30.996650 10405 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:30.776319 10436 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.946408 10241 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:31.043946  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 249363us
I20260709 13:13:31.134878  8029 leader_election.cc:400] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5.
I20260709 13:13:31.135167  8029 leader_election.cc:403] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:13:30.982465 10416 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:30.966408 10420 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:31.065433  8207 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:31.138368  8207 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:13:31.066850  8208 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:31.067258  8211 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:31.068254  8209 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:31.070096  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:30.976176 10299 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:30.998469 10174 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.120633 10027 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:31.064863 10408 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:30.976620 10096 consensus_queue.cc:1059] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.137s
W20260709 13:13:31.124060  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 349869us
I20260709 13:13:31.099282 10010 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:30.978324 10434 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:31.125281 10257 consensus_meta.cc:287] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.844s	user 0.006s	sys 0.004s
I20260709 13:13:31.118966 10260 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:30.995802 10426 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:30.976923 10283 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.151454  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:31.123004 10406 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:31.151436 10319 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:31.110718 10369 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.173695  8729 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:31.153620 10174 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:31.163803  8204 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:31.197062  8729 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:31.197909  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:31.165277 10405 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5.
I20260709 13:13:31.011826 10409 consensus_queue.cc:260] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.170694  8203 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:31.166930 10081 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:31.170010  9980 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:31.197860  8102 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:31.187148 10417 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:31.151286 10241 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.163648 10420 raft_consensus.cc:697] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:31.194402 10370 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:31.179296 10426 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:31.168398 10027 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.182073  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:31.243932  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 61959.6us
I20260709 13:13:31.244585  8029 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:31.252218 10432 log.cc:927] Time spent T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:13:31.200475  8725 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:31.256038  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:31.256103  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:31.257285  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:31.257388  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:31.234426 10096 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:31.258250  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.182446 10257 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.263603  8205 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:31.258265  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:31.154155 10188 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.259271  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:31.238405 10283 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:31.151557 10416 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.265331  8209 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:31.259867 10445 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.239953 10170 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:31.239845  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:31.254841  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:31.261643  8208 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:31.240916 10417 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:31.281492  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:31.253146 10450 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:31.253904 10370 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.244067 10426 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:31.288810  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:31.297055  8207 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:13:31.143883 10436 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.259024 10405 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:31.292196  8136 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:31.234354 10299 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.285847  8205 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:31.301779  8209 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:31.289991 10444 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:31.237737 10010 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.289829 10241 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:31.311757  8206 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:31.313051  8207 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:31.294692 10417 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:31.277052  9980 raft_consensus.cc:697] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:31.291723  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:31.275595 10096 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.346691  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:31.347239  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:31.347966  8730 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 2: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:31.319172  8724 raft_consensus.cc:2393] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 6: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:31.311976 10269 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:31.315418 10444 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:31.314929 10325 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:31.319381  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:31.328298 10010 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:31.361869  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:31.366770  8730 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:31.291319 10445 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.367192  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:31.370509  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 131479us
I20260709 13:13:31.372095  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:31.374147  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 388789us
I20260709 13:13:31.376302  8102 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:31.380515  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:31.364292 10444 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:31.369876  8204 raft_consensus.cc:2393] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:31.391892  8211 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:31.398293 10358 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:31.378064 10241 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:31.402817 10241 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:31.403731 10241 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:31.404498 10439 log.cc:927] Time spent T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.278s	user 0.000s	sys 0.000s
I20260709 13:13:31.313407 10420 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.368708 10325 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.432209 10457 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:31.432722 10457 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.434778 10457 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:31.436307  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:31.412155  8721 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:31.377489 10458 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
W20260709 13:13:31.440110  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.443264  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:31.425254 10002 consensus_meta.cc:287] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.957s	user 0.007s	sys 0.003s
I20260709 13:13:31.419260 10027 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:31.422394 10434 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:31.453413  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 77571.6us
I20260709 13:13:31.453565 10369 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:31.460562  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 82739.5us
I20260709 13:13:31.426497 10269 consensus_queue.cc:1059] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:13:31.446100 10257 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:31.448999 10002 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:31.468007 10096 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:31.468905  8730 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:31.421808  9980 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:31.472105 10446 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:13:31.474476 10454 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:13:31.478504  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:31.443840 10405 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:31.461447  9876 consensus_queue.cc:1059] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:13:31.497828 10465 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
W20260709 13:13:31.483682  9734 consensus_meta.cc:287] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.907s	user 0.010s	sys 0.001s
I20260709 13:13:31.508143  9734 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.510533  8726 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:31.511051  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:31.514638  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:31.515177  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.475251 10416 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:31.480892 10462 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:31.512825 10247 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.510329  9734 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:31.517757 10291 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.517706 10010 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.558881  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 262) took 3125 ms. Trace:
I20260709 13:13:31.527693 10461 raft_consensus.cc:3055] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:31.560379 10461 raft_consensus.cc:740] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:31.559273  8129 rpcz_store.cc:276] 0709 13:13:28.433785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.664887 (+1231102us) service_pool.cc:224] Handling call
0709 13:13:29.665218 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:29.746512 (+ 81294us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:29.811145 (+ 64633us) log.cc:844] Serialized 14 byte log entry
0709 13:13:30.105322 (+294177us) log.cc:844] Serialized 14 byte log entry
0709 13:13:30.182490 (+ 77168us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:30.182544 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:30.182592 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:30.287748 (+105156us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b180042e4a0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:13:30.945352 (+657604us) spinlock_profiling.cc:243] Waited 5.57 ms on lock 0x7b180042e4a0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ef72a6 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419
0709 13:13:31.558643 (+613291us) spinlock_profiling.cc:243] Waited 81 ms on lock 0x7b540035d9d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:31.558697 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:31.558825 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82817,"prepare.queue_time_us":89572,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":4073,"raft.queue_time_us":112448,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":4205,"spinlock_wait_cycles":212376448,"thread_start_us":152104,"threads_started":1,"wal-append.queue_time_us":335927,"wal-append.run_cpu_time_us":1501,"wal-append.run_wall_time_us":1063571}
I20260709 13:13:31.522964 10319 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:31.527456 10471 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.530974 10469 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.554445 10416 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.532641 10436 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:31.562204  9876 raft_consensus.cc:3055] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:31.588126  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:31.588784  8061 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:31.561174 10461 consensus_queue.cc:260] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.599941  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:31.587958  9876 raft_consensus.cc:740] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:31.580492 10405 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.594638  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:31.595243  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:31.553432 10217 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:13:31.601907  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:31.602077  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:31.614224  8026 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:31.564586 10291 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.613379  8059 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:31.613441  8063 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:31.566797 10010 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.615317  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:31.529743 10462 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.617487  8062 raft_consensus.cc:2393] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:13:31.627750 10468 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:13:31.618372  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:31.629217  8060 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:31.579479 10319 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.626111  8026 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:31.627628  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:31.630545  8055 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:31.632216 10319 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:31.633301  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:31.633857  8060 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:13:31.634109  8055 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:31.634407  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:31.635188  8724 raft_consensus.cc:2393] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:31.635311  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:31.616235 10370 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:31.636071 10323 consensus_meta.cc:287] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.906s	user 0.009s	sys 0.000s
I20260709 13:13:31.636340 10370 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:31.636507 10323 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.637210  8055 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:31.636966 10370 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.637563  8721 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:31.638559  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:31.639139  8055 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:31.600063 10445 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:31.580974 10471 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.612414 10325 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:31.646499 10479 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:13:31.647662 10415 consensus_meta.cc:287] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.872s	user 0.007s	sys 0.001s
I20260709 13:13:31.647956 10414 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:31.589032 10416 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.647904 10461 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:31.584178 10469 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.650564 10170 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:13:31.654619  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:31.641985 10470 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:31.669391  8058 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:31.648326 10174 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.671572 10010 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:31.650732 10394 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:31.671726 10416 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:31.673285 10444 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.654249 10323 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:31.687597  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:31.688730  8058 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:31.689555  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:31.687623  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:31.736522  8061 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:31.736595  8060 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:31.698535 10174 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:31.704738 10394 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.707901  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:31.708724 10291 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:31.633890 10260 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
W20260709 13:13:31.726496  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:31.649365  9876 consensus_queue.cc:260] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.791595 10471 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:31.793100 10450 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:31.876083  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:31.814312  8061 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:31.916930  8062 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:31.918361  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:31.835633  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 166376us
W20260709 13:13:31.796947  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:31.920317  8175 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.526s (SENT) [suppressed 42 similar messages]
I20260709 13:13:31.827958 10469 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:31.927981  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:31.896071 10474 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.240s	user 0.000s	sys 0.000s
W20260709 13:13:31.765908 10442 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.529s	user 0.000s	sys 0.004s
W20260709 13:13:31.863596 10283 consensus_meta.cc:287] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.577s	user 0.009s	sys 0.000s
I20260709 13:13:31.930866  8730 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
W20260709 13:13:31.928683  8175 leader_election.cc:336] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.652s (SENT)
I20260709 13:13:31.932156  8134 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:13:31.936137  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:31.937598  8175 leader_election.cc:336] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.810s (SENT)
I20260709 13:13:31.933399  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:31.915997 10462 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:31.915352 10444 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:31.838814 10323 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:31.938817  8061 raft_consensus.cc:2393] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:13:31.861474  8725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 406) took 2133 ms (client timeout 2231 ms). Trace:
W20260709 13:13:31.939723  8175 leader_election.cc:336] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.865s (SENT)
I20260709 13:13:31.877408 10370 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:31.940876  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:31.856251 10010 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.934087  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:31.946116 10394 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:31.936904  9876 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:31.950971  8062 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:31.943636 10027 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:31.955838 10258 raft_consensus.cc:3055] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:31.956211 10258 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:31.957886  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:31.685076 10426 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.931442 10471 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:31.931277  8055 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:31.791906 10417 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:31.960829  8723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 413) took 1189 ms. Trace:
W20260709 13:13:31.940634  8725 rpcz_store.cc:269] 0709 13:13:29.716352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:29.728094 (+ 11742us) service_pool.cc:224] Handling call
0709 13:13:30.042672 (+314578us) spinlock_profiling.cc:243] Waited 7.46 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:31.828795 (+1786123us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b5400640354. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:31.848225 (+ 19430us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1462493184}
I20260709 13:13:31.963815 10434 raft_consensus.cc:3055] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:31.940495 10291 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:31.974496  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:31.975319  8061 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:31.976163  8062 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:31.957221 10010 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.944336 10323 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.980916  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:31.971930 10434 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:31.984592 10426 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:31.990202  8102 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:31.993041  8058 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:31.970165 10471 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:31.994764 10286 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:31.995635  8726 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:13:32.017920  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:32.107020  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.107426  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.109323  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.180452  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:32.184700  8062 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:32.185016  8058 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:32.111296  8060 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:32.000590  9655 consensus_queue.cc:1059] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:13:32.116045 10087 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.116432  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:32.117838  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:31.970984  8723 rpcz_store.cc:276] 0709 13:13:30.771138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.775077 (+  3939us) service_pool.cc:224] Handling call
0709 13:13:31.960135 (+1185058us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:13:31.960318 (+   183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:31.960489 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:31.960523 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:31.960570 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:31.960601 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.960658 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:31.960766 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270824832}
I20260709 13:13:32.118811 10436 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:31.983122 10258 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.194499  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.042609 10434 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.119768 10283 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.195609 10436 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.196066 10283 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.197676 10462 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.118973 10420 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.194640 10258 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:32.198982 10420 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.122067  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.198592 10462 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.203581  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.205202  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.205730 10462 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.134984  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.135468  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 988) took 1762 ms. Trace:
I20260709 13:13:32.131168  8056 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:32.140395 10417 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.142400 10010 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:32.143116 10444 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.207508 10286 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:32.208042 10444 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.208666 10286 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.210646 10444 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.211323 10010 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:32.206676  8131 rpcz_store.cc:276] 0709 13:13:30.372654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.374832 (+  2178us) service_pool.cc:224] Handling call
0709 13:13:30.375095 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:30.765631 (+390536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:32.134696 (+1369065us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54001aa754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:32.135211 (+   515us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":706780928}
I20260709 13:13:32.211858  8176 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:32.212138 10010 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.212533  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.213339  8131 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:32.214975 10291 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:32.215775 10291 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:32.216284 10426 consensus_queue.cc:1059] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:32.143709  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.144200  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.145476 10397 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.146196  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.146166  8059 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:32.148357  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 991) took 1774 ms. Trace:
I20260709 13:13:32.148993 10471 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:32.150087 10484 log.cc:927] Time spent T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Append to log took a long time: real 0.179s	user 0.004s	sys 0.001s
I20260709 13:13:32.149948  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:32.157678  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:32.159030  9853 raft_consensus.cc:3055] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:32.171379 10450 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:32.174634  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.175621  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.183152  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.183353  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.186441  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.190735  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.193099  9655 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:32.234331  9655 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:32.235036  8055 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:32.235749  9655 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.237824  9655 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:31.995090 10260 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.196552  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.137401 10323 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.194615  8730 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:32.222589  8136 rpcz_store.cc:276] 0709 13:13:30.373461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:30.373565 (+   104us) service_pool.cc:224] Handling call
0709 13:13:30.397421 (+ 23856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:31.308765 (+911344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:32.147844 (+839079us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b54003c96d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:32.148175 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2057419008}
I20260709 13:13:32.231537 10397 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:32.232527  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.232604  8058 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:32.233791  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.230177  8132 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:32.232446  8138 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:32.207319  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:32.245306  8131 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:32.208758 10087 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.249032  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.249220  8135 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:32.249853  8063 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:32.249791 10087 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.257016  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:32.257509  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.229724  9853 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:32.258505  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.246556 10420 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:32.231820  8722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 430) took 1051 ms. Trace:
I20260709 13:13:32.257957  8060 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
W20260709 13:13:32.264026  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.254242  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.260624  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.248627 10397 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.243865 10319 raft_consensus.cc:3055] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:32.271214  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.272957 10356 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:32.217415 10291 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.273772 10356 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.274704 10436 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:32.275134  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.275319 10436 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:32.276067  8054 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:32.276186 10356 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:32.265791 10426 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:32.269577 10319 raft_consensus.cc:740] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:32.276981 10426 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.277760  8056 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:32.278738  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.274333  9853 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.278366 10319 consensus_queue.cc:260] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.276993  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.282372  8061 raft_consensus.cc:2376] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5. Re-sending same reply.
I20260709 13:13:32.269053 10010 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:32.285108  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.286706  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1321) took 1301 ms. Trace:
W20260709 13:13:32.287127 10440 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 1.167s	user 0.003s	sys 0.006s
W20260709 13:13:32.294230 10473 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.652s	user 0.012s	sys 0.018s
I20260709 13:13:32.294620  9853 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:32.291131  8137 rpcz_store.cc:276] 0709 13:13:30.984702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.018809 (+ 34107us) service_pool.cc:224] Handling call
0709 13:13:31.032190 (+ 13381us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:31.061207 (+ 29017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:31.061250 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:31.114089 (+ 52839us) log.cc:844] Serialized 52 byte log entry
0709 13:13:31.292332 (+178243us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:31.305634 (+ 13302us) log.cc:844] Serialized 14 byte log entry
0709 13:13:31.305852 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:31.305944 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:31.596061 (+290117us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b4800002dd8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:13:32.192638 (+596577us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b180022d680. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:13:32.285926 (+ 93288us) raft_consensus.cc:1685] finished
0709 13:13:32.285961 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:32.286290 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30003,"prepare.queue_time_us":14956,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":1508,"raft.queue_time_us":50097,"raft.run_cpu_time_us":2550,"raft.run_wall_time_us":541612,"spinlock_wait_cycles":40812544,"thread_start_us":149929,"threads_started":1,"wal-append.queue_time_us":229850}
I20260709 13:13:32.294144  8026 leader_election.cc:400] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5.
I20260709 13:13:32.297444  8026 leader_election.cc:403] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:13:32.293628 10450 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.266072  8722 rpcz_store.cc:276] 0709 13:13:31.180360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.183074 (+  2714us) service_pool.cc:224] Handling call
0709 13:13:32.156901 (+973827us) spinlock_profiling.cc:243] Waited 6.16 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:32.231696 (+ 74795us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14177664}
I20260709 13:13:32.302635  8056 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:32.303157  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:32.303789  8063 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:32.305351  8063 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:32.306262  8056 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:32.306651  8054 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:32.308029 10319 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:32.308866  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:32.309396 10408 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:32.309170 10263 consensus_queue.cc:1059] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:32.310372  8726 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:32.279316 10426 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:32.310566 10263 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:32.296415 10420 consensus_queue.cc:1059] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:13:32.317683 10263 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.311507 10081 consensus_queue.cc:1059] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:32.312330 10260 consensus_queue.cc:1059] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:32.312608 10283 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.313071 10444 raft_consensus.cc:3055] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:32.313159  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.314193 10409 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:32.318125 10434 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:32.329121 10260 raft_consensus.cc:3055] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:32.329614 10260 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:32.321575 10406 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.330766  8060 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:32.330852 10260 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.331741 10406 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:32.332656 10260 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:32.335309 10081 raft_consensus.cc:3055] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:32.320559  8061 raft_consensus.cc:2393] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:32.335774 10081 raft_consensus.cc:740] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:32.337484 10444 raft_consensus.cc:740] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:32.337503  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.319978  8058 raft_consensus.cc:2376] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6. Re-sending same reply.
I20260709 13:13:32.337751 10081 consensus_queue.cc:260] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.338724 10444 consensus_queue.cc:260] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.340137 10444 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:32.328576 10299 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:32.317763 10426 raft_consensus.cc:3055] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:32.341461 10426 raft_consensus.cc:740] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:32.327991 10436 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.343449 10436 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.345053 10100 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.344749 10263 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.342608  8131 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:32.348188 10286 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:32.349141 10441 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 1.106s	user 0.000s	sys 0.002s
W20260709 13:13:32.352111 10449 log.cc:927] Time spent T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 1.114s	user 0.007s	sys 0.000s
W20260709 13:13:32.310985  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 66814.9us
I20260709 13:13:32.360440  8137 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:13:32.362162  8102 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:32.363476 10422 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:32.364065 10422 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:32.365263 10422 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:32.366454 10400 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.045s	user 0.038s	sys 0.060s
W20260709 13:13:32.366923 10400 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.142s	user 0.038s	sys 0.060s
W20260709 13:13:32.367802  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 61111us
I20260709 13:13:32.368096  8728 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:32.369169  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1316) took 1385 ms. Trace:
I20260709 13:13:32.369545  8134 rpcz_store.cc:276] 0709 13:13:30.983270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.010509 (+ 27239us) service_pool.cc:224] Handling call
0709 13:13:31.011148 (+   639us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:31.936291 (+925143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:32.368289 (+431998us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b5400042ed4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:32.368816 (+   527us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2117365632}
I20260709 13:13:32.369839  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 556) took 1173 ms. Trace:
W20260709 13:13:32.371433  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.370790  8728 rpcz_store.cc:276] 0709 13:13:31.196236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:31.196553 (+   317us) service_pool.cc:224] Handling call
0709 13:13:31.234448 (+ 37895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:32.368756 (+1134308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:32.369595 (+   839us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54006aead4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:32.369773 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2605970688}
I20260709 13:13:32.377131  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:32.379727  8054 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:32.390653  8059 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:32.392923  8134 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:13:32.393345  8061 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:32.394388 10263 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.342638 10426 consensus_queue.cc:260] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.371235 10247 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:32.402992  8057 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 2.
I20260709 13:13:32.406286  8055 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:32.413764  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.416875  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.402817 10081 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:32.420212  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.420472  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.418169  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.424897  8060 raft_consensus.cc:2410] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:13:32.425048  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.422008  8059 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:32.430270  8060 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:13:32.420266 10010 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.439247  8057 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
W20260709 13:13:32.435515  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 204646us
I20260709 13:13:32.441275 10217 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.441608  8063 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 6: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:32.441977 10217 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.442866 10174 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:32.441298  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:32.445361  8134 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:32.442363 10416 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.444265  8056 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:32.439962  8055 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:32.448329  8059 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:32.432046 10400 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.451229  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.452772  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:32.451824  8060 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:32.452481  8176 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:32.458843 10450 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.460251  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:32.460078 10400 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.462247 10027 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:32.463203 10027 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:32.464517 10027 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:32.464907 10027 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:32.466856 10291 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:32.467478 10291 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.469911 10291 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:32.471618  8061 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:32.476017  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.477089  8212 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:32.477389  8055 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:32.478820 10426 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:32.482883  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.483410 10459 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.484230  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.484157 10459 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.485055 10010 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.485491  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:32.486068  8133 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:32.486680  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.487167 10010 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:32.487236  8132 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:32.488135  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.488760  8132 raft_consensus.cc:2393] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:32.489485  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.490159  8132 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:32.490675  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.491768  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.492291  8136 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:32.493599  8130 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:13:32.494545  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.495332  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.495548  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:32.496599  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 46681.2us
W20260709 13:13:32.506369  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 57176.6us
I20260709 13:13:32.506812  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.507968  8059 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:32.507794  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 45457.1us
I20260709 13:13:32.514582  8056 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:13:32.516608  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.455597 10408 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5.
I20260709 13:13:32.466007 10027 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.631011  8137 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:32.627754  8130 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
W20260709 13:13:32.652320  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 54333.4us
W20260709 13:13:32.557819  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.527251  8060 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:32.543865  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:32.474579 10416 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.544304  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.594249  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:32.626547  8062 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
W20260709 13:13:32.546909  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 97617.1us
I20260709 13:13:32.664177  8130 pending_rounds.cc:85] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 4
W20260709 13:13:32.658521  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.522658  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:32.579916  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:32.591303 10487 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:13:32.672336  8058 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
W20260709 13:13:32.634140  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 49535.4us
I20260709 13:13:32.589841  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:32.674747  8138 raft_consensus.cc:2393] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:32.675737  8056 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:32.622189 10100 raft_consensus.cc:3055] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:32.669124  8130 pending_rounds.cc:107] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:13:32.679266  8130 raft_consensus.cc:2889] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:32.679692  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 85620.2us
I20260709 13:13:32.583268  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:32.621495 10459 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.667451 10420 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:32.620225  9853 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.594592 10436 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.625586 10462 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:32.622315 10081 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.686201  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.697227  8137 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:32.588555 10286 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.554699 10174 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.139s
I20260709 13:13:32.588373 10422 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:32.554843 10010 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.146s
I20260709 13:13:32.626142 10217 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:32.698406  8131 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:32.698321  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:32.680882 10100 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:32.699082 10027 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:32.699131  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:32.701853  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.683010 10471 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:13:32.704141 10258 consensus_meta.cc:287] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.505s	user 0.012s	sys 0.000s
I20260709 13:13:32.708112  8056 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:32.708859  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.754009  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.579787 10356 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.109s
W20260709 13:13:32.757292  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.757040  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.776271  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.514607  9980 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.169s
W20260709 13:13:32.760301  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.579042 10283 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.117s
I20260709 13:13:32.772217  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.771018  8212 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:32.785699  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:32.696732  9853 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.757601 10420 raft_consensus.cc:697] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:32.791589  8726 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:32.801115  8132 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:32.775285 10471 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.806526  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.810715  8058 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 6: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:32.808856  8060 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:32.814092  9655 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:32.814967  9655 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:32.814069  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.813123  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.809571  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:32.817762  8726 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:32.765630 10081 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:32.818377  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 167592us
I20260709 13:13:32.764010 10462 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.816680  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:32.820415  8060 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:32.817443  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:32.811082  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:32.823400  8056 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:32.824280  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:32.825088  8132 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:32.835944  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:32.836527  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.807504 10471 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:32.804807  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:32.820475 10217 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:32.840392  8203 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 13:13:32.847888  8203 pending_rounds.cc:85] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 7
I20260709 13:13:32.809912  9980 consensus_queue.cc:1243] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:32.848090  9655 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.850872  8203 pending_rounds.cc:107] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 13:13:32.854589  8212 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:32.855187 10263 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:32.856207  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:32.857649  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:32.854876 10369 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.858738 10369 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.854650  8203 raft_consensus.cc:2889] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:32.860477 10499 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:32.866573  8061 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:32.866477  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:32.867421  8208 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:32.868048 10443 consensus_queue.cc:1059] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
W20260709 13:13:32.873064 10299 consensus_meta.cc:287] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.532s	user 0.008s	sys 0.004s
I20260709 13:13:32.875221 10263 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.875876 10263 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.877972 10263 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.826488  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:32.851548  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 47556.3us
I20260709 13:13:32.879495  8135 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:32.888810  8061 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:32.824101 10400 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:32.891021 10494 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 13:13:32.891608 10434 consensus_meta.cc:287] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.562s	user 0.015s	sys 0.006s
W20260709 13:13:32.892776  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.893515 10408 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:32.894148 10408 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.894506 10420 consensus_queue.cc:237] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.895556 10416 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:32.896377 10408 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.897497  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:32.898514  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 61817.4us
I20260709 13:13:32.898880 10100 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.900370 10100 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:32.900894 10260 consensus_meta.cc:287] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.568s	user 0.012s	sys 0.002s
I20260709 13:13:32.901041  9853 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.902123  8138 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:13:32.902534  8138 pending_rounds.cc:85] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 6
I20260709 13:13:32.902774  8138 pending_rounds.cc:107] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:13:32.903005  8138 raft_consensus.cc:2889] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:32.903775  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:32.903990  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:32.904160  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:32.904934  8204 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:32.905175  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:32.905915  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:32.909569 10217 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.799976 10422 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:32.906191  8203 raft_consensus.cc:2410] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:13:32.910676 10459 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:32.910741  8062 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:32.798712 10286 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:32.908921 10462 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:32.905311  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 67741.7us
I20260709 13:13:32.911643  8210 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:32.922143  8133 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:32.922855  8054 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:32.923302  8129 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:32.854763  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:32.937953  8134 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:32.924938  8205 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: []
W20260709 13:13:32.923492  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 161327us
I20260709 13:13:32.925268  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:32.946512  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.946421 10409 consensus_queue.cc:1059] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:32.947724  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.948300  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:32.948397  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:32.948405  8055 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:32.949776  8061 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:32.949955  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:32.946833  8058 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:13:32.948545  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:32.953665  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 52917.4us
I20260709 13:13:32.953913  8056 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
W20260709 13:13:32.955375  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:32.961913  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 445797us
I20260709 13:13:32.964959  9876 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:32.965428  9876 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:32.965544 10422 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:32.965837  9876 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:32.967929  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:32.968551  8205 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:32.968734  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:32.969610  8203 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:32.970033  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:32.971602  8204 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:32.972640  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:32.973273 10291 raft_consensus.cc:3055] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:32.973737 10291 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:32.983950  8132 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:33.238261  8060 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:33.065776  8135 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:33.307235  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1332) took 1129 ms. Trace:
I20260709 13:13:33.090142  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:33.124675  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.040236  8131 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
W20260709 13:13:33.349323  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 362788us
I20260709 13:13:32.975461  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:33.360351  8206 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:32.982811  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:33.185511  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:33.351999  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:33.062326  8133 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:33.328395  8212 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:33.355979  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:32.974743  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 69049.2us
I20260709 13:13:33.359337  8129 rpcz_store.cc:276] 0709 13:13:32.177567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:32.177982 (+   415us) service_pool.cc:224] Handling call
0709 13:13:33.306960 (+1128978us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54001bb054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:33.307118 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1363612032}
I20260709 13:13:33.355304  8137 raft_consensus.cc:3055] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:33.364944  8137 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:32.984452  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:33.355487  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 430268us
W20260709 13:13:32.989306  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.339062  8101 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:13:33.350035  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 266358us
W20260709 13:13:33.340363  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 460661us
I20260709 13:13:33.377485  8058 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
W20260709 13:13:33.378399  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:33.383353  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 105627us
I20260709 13:13:33.365882  8137 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:33.383517  9655 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:33.384271  9655 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.384387  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:33.384780  9655 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:33.386906  8207 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:33.384555  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:32.973897  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.274678 10299 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.058388 10319 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:33.349452 10436 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.023s	sys 0.018s
W20260709 13:13:32.984704 10444 consensus_meta.cc:287] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.644s	user 0.010s	sys 0.001s
I20260709 13:13:33.085016 10286 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.980333 10435 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:33.447093  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.363026 10081 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:33.088495 10471 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:33.119688 10426 consensus_queue.cc:1059] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.275s
I20260709 13:13:33.444568  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.392926 10369 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:32.975139 10247 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.425s
I20260709 13:13:33.411674  8205 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
W20260709 13:13:33.444000 10443 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.012s	sys 0.025s
W20260709 13:13:33.451056  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 527120us
W20260709 13:13:33.430953  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:33.454262  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 59757.7us
I20260709 13:13:33.456245  8208 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:33.460659  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:33.388834 10434 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:33.461419  8026 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:13:33.436069 10436 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.024s	sys 0.019s
I20260709 13:13:33.465972  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.483371  8210 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:33.466858  8137 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:33.474586  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 499270us
W20260709 13:13:33.476328 10443 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.013s	sys 0.026s
I20260709 13:13:33.476689  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.491979  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.493630  8205 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:33.494333  8209 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:33.508217  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.509486  8206 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:33.512730  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.514817  8205 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:33.503335 10435 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:33.518714 10435 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:33.519097 10435 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:33.473404 10081 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:13:33.522689  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:33.525847  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 51589us
I20260709 13:13:33.466634  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.437497 10299 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:33.436793 10319 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:33.530459 10516 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.435339 10505 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 13:13:33.453184  8204 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:33.447352 10447 consensus_queue.cc:1059] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:13:33.529734  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.538489  8101 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:33.546644  8212 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:33.526578 10217 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:33.550740  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:33.540014  9876 consensus_meta.cc:287] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.574s	user 0.005s	sys 0.005s
I20260709 13:13:33.527868 10369 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.494078 10420 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:13:33.535358 10434 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.567201  8210 raft_consensus.cc:2410] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:13:33.573494  8056 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
W20260709 13:13:33.559988  9980 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:13:33.566383 10516 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:33.582901  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 51571.9us
I20260709 13:13:33.556708 10459 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:33.584445  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 190601us
I20260709 13:13:33.556005 10447 consensus_queue.cc:1243] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:13:33.601869  8209 raft_consensus.cc:2410] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:13:33.571105 10081 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:13:33.641026  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.649178  9655 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:33.649477  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:33.651688  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:33.658243  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.659343  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.659714  8207 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
W20260709 13:13:33.667259  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 58612.4us
I20260709 13:13:33.669543  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:33.606148  8203 raft_consensus.cc:2410] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:33.670822  9655 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:33.671859  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 74226.2us
I20260709 13:13:33.608847 10081 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.572235  9876 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:33.669476  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:33.551539 10291 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:33.596994 10434 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:33.666914 10506 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.663165 10459 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:33.685249  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.675189 10510 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.690800 10510 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:33.691751 10434 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:33.692297 10434 consensus_queue.cc:1059] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:33.692880 10510 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:33.693645 10217 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:33.694813  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.695132  8208 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 6: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:33.693536  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:33.608403 10369 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:33.698881  8210 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:33.695745  8056 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:33.699138  8104 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:33.702036  8054 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:33.677275 10081 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:33.703604  8175 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:33.700280  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:13:33.703956  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 181399us
I20260709 13:13:33.710381  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.715837  8104 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:33.717610  8207 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:33.726929  9655 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:33.727447  9655 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.727845  9655 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:33.738821  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:33.743299  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 101048us
I20260709 13:13:33.705403 10471 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:33.748821  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 74645.2us
I20260709 13:13:33.751055  8175 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:33.746807  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:33.693554 10506 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:33.762408  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 81274.3us
W20260709 13:13:33.762828  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 86526.8us
I20260709 13:13:33.743235 10420 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:33.745180 10499 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:33.712019 10459 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:33.765955  8104 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:33.776407  8104 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:33.753911  8722 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:33.776726  9655 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:33.754138 10299 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:33.753718 10511 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:33.789049 10459 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:33.777505 10471 raft_consensus.cc:3055] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:33.790508 10471 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:33.791514 10471 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:33.781338 10436 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:33.793080 10436 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:33.746191 10515 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:13:33.794050 10436 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:33.794950 10299 consensus_queue.cc:1059] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:33.795789 10299 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:33.758875 10319 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:33.780373 10283 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:33.777798 10426 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:33.804674 10426 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.805142 10426 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:33.806964  8693 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:33.807308 10400 raft_consensus.cc:3055] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:33.807885 10515 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:33.781776  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 57320.9us
I20260709 13:13:33.809085  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:33.781983 10513 log.cc:927] Time spent T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:13:33.809401 10531 consensus_queue.cc:1059] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:33.810532  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:33.811095 10409 raft_consensus.cc:3055] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:33.811189 10514 consensus_queue.cc:1059] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:33.811547 10409 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:33.812011 10514 consensus_queue.cc:1243] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:33.812219  9876 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:33.759068 10521 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:33.812250 10283 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:13:33.788189 10509 log.cc:927] Time spent T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:13:33.812866  9876 raft_consensus.cc:3055] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:33.813071 10283 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:33.813386  9876 raft_consensus.cc:740] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:33.813616 10283 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:33.812507 10514 raft_consensus.cc:3055] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:33.815011 10514 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:33.816022 10514 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:33.807874 10400 raft_consensus.cc:740] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:33.817149  8696 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:33.812549  8207 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:33.818130 10400 consensus_queue.cc:260] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:33.819319 10518 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:33.819703 10434 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:33.783465  8211 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:33.820736 10027 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:33.821154 10027 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:33.821849  8029 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:33.819154 10510 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:13:33.784178 10504 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:13:33.812959 10521 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:33.823145  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:33.824287 10356 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:33.824683 10356 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:33.824805  8029 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:33.824940 10521 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:33.810493 10447 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:33.825611 10447 consensus_queue.cc:1243] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:33.825604 10356 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:33.839481 10356 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:33.826578 10434 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:33.823149  8101 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:33.827093 10521 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:33.812485 10409 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:33.828403 10435 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:33.831663  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 224) took 1362 ms. Trace:
I20260709 13:13:33.831663  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2207) took 1010 ms. Trace:
I20260709 13:13:33.851792  9655 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:33.832844  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:33.833035 10369 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:33.851930  8054 rpcz_store.cc:276] 0709 13:13:32.821143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.185168 (+364025us) service_pool.cc:224] Handling call
0709 13:13:33.831494 (+646326us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b5400333e54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:33.831602 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783730432}
I20260709 13:13:33.817164 10514 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:33.833666  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 95177.7us
I20260709 13:13:33.834544  9876 consensus_queue.cc:260] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:33.781610  8104 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:33.871204  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 89663.2us
I20260709 13:13:33.871349  9655 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:33.871903  8104 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:33.871974  9655 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.872540  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:33.822150 10027 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:33.838583 10471 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:33.820955 10518 consensus_queue.cc:1059] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:13:33.840670  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 149817us
I20260709 13:13:33.875989  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:33.851141  8063 rpcz_store.cc:276] 0709 13:13:32.469445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:32.490440 (+ 20995us) service_pool.cc:224] Handling call
0709 13:13:32.490776 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:32.671508 (+180732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:32.671538 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:32.695961 (+ 24423us) log.cc:844] Serialized 26 byte log entry
0709 13:13:32.954836 (+258875us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:33.352625 (+397789us) log.cc:844] Serialized 14 byte log entry
0709 13:13:33.353195 (+   570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:33.353315 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:33.831451 (+478136us) raft_consensus.cc:1685] finished
0709 13:13:33.831471 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:33.831586 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":238942,"prepare.queue_time_us":397182,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":179228,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":10201,"thread_start_us":146554,"threads_started":1,"wal-append.queue_time_us":885065}
I20260709 13:13:33.878947 10519 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:33.879361 10519 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:33.880162  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:33.880242 10519 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:33.880949  8724 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:33.881657  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:33.882368  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:33.883615  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:33.884851 10319 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:33.885253 10319 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.885636 10319 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:33.885931  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:33.886831  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:33.894405  8721 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:33.887359 10529 consensus_queue.cc:1059] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:33.896347 10286 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:33.879377  9876 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:33.896710 10319 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:33.886005 10516 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:33.891019 10027 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:33.891122 10435 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:33.891469 10462 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:33.891721 10510 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:33.891772 10526 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:33.892475 10521 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:33.874092 10499 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:33.888306  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:33.872453  9655 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:33.901556  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:33.905839  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 126092us
W20260709 13:13:33.956008  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 137226us
I20260709 13:13:33.936555  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:33.964461  8026 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:13:33.912920  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 80159.4us
I20260709 13:13:33.877192 10434 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.085s
W20260709 13:13:33.984174  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 88196.7us
I20260709 13:13:33.909801 10462 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:33.985126  8104 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:33.899119 10444 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:33.935801  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:33.990255  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 78114.6us
I20260709 13:13:33.912060 10010 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:33.961650 10499 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:33.996450  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 107158us
I20260709 13:13:33.963233 10526 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:34.001344  9655 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:33.899963 10531 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
W20260709 13:13:34.000295  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:33.963023 10510 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:33.991003  8027 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:13:34.003768  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 67316.4us
I20260709 13:13:33.997046  8104 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:34.027354  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.044845  9655 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:33.991127 10462 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:33.992187 10444 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.004567  8696 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:33.996836 10515 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:34.058329  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 73115.6us
W20260709 13:13:34.059985  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.060631  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:34.061282  8207 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:33.990043 10521 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:34.062640 10525 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:34.063596 10525 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.064327  8723 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:34.064440 10521 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:33.907130 10524 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.123s
I20260709 13:13:34.064088 10525 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:34.065551  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 68751.7us
W20260709 13:13:34.066046  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 62561.1us
W20260709 13:13:34.069686  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 76837us
I20260709 13:13:34.070748  9655 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:13:34.071846  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 74829.4us
I20260709 13:13:34.072690  8104 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:34.066892  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:33.995940 10319 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:33.998721 10420 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:34.065023 10521 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:34.047701 10436 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:34.001855 10499 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:33.997495 10506 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:34.005219 10435 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:34.092465  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:34.096060  8696 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:33.996062 10434 consensus_queue.cc:1243] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:13:34.065486 10444 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:33.990919 10426 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:34.130688  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58094.5us
I20260709 13:13:34.134543  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:34.105420 10010 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:34.105837  8721 raft_consensus.cc:2393] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
W20260709 13:13:34.131371  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 70736us
I20260709 13:13:34.101405 10436 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.101101 10521 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.146620  9655 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:34.106298 10435 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.106933 10420 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 6 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:34.157166  8211 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:13:34.140079 10516 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:34.143972 10434 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.145236 10506 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:13:34.151072  8175 connection.cc:256] RPC call timeout handler was delayed by -1.1621842389999983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
I20260709 13:13:34.155414 10435 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:34.186633  8175 connection.cc:256] RPC call timeout handler was delayed by -1.1405842389999634s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2082s for a response
I20260709 13:13:34.149821 10521 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:34.150027 10436 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:34.184367  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:34.158406 10511 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:34.188326  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 52695.5us
I20260709 13:13:34.193295  8104 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:34.189465  8175 connection.cc:256] RPC call timeout handler was delayed by -1.1099842390001413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2116s for a response
I20260709 13:13:34.133177 10491 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.269s
I20260709 13:13:34.157301 10459 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.185467  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:34.189054  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 123357us
I20260709 13:13:34.157083 10319 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.199826  9655 raft_consensus.cc:697] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:34.179404 10516 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.132378 10538 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 13:13:34.187693 10506 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.209319  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:34.207609 10516 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:34.192808 10434 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:34.219066  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.217885  9655 consensus_queue.cc:237] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.208724 10506 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:34.224612  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 54943.5us
I20260709 13:13:34.211872 10369 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:34.225250  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:34.179860 10510 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:34.215718 10459 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:34.226734 10540 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:13:34.227945 10536 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:13:34.218624 10511 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:13:34.213773 10524 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:34.238812 10539 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:13:34.241417  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1337) took 1605 ms. Trace:
I20260709 13:13:34.242386 10537 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:34.241940 10420 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:34.242779 10541 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:34.206923 10319 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:34.226761 10533 log.cc:927] Time spent T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
W20260709 13:13:34.247223  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 77320.4us
W20260709 13:13:34.275383  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 284471us
I20260709 13:13:34.277833  8693 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:34.258289  8130 rpcz_store.cc:276] 0709 13:13:32.635635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:32.774270 (+138635us) service_pool.cc:224] Handling call
0709 13:13:32.791378 (+ 17108us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:32.897219 (+105841us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:33.339639 (+442420us) log.cc:844] Serialized 14 byte log entry
0709 13:13:33.833905 (+494266us) spinlock_profiling.cc:243] Waited 8.04 ms on lock 0x7b180037c480. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ea3c0a 00007f6dc15e3219 00007f6db7f2ba40 00007f6db7f264e4 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf 00007f6dbb030038
0709 13:13:33.834092 (+   187us) log.cc:844] Serialized 14 byte log entry
0709 13:13:33.834268 (+   176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:33.834311 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:33.847738 (+ 13427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:34.241217 (+393479us) spinlock_profiling.cc:243] Waited 61.2 ms on lock 0x7b540008cd54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:34.241260 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:34.241366 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":294230,"prepare.queue_time_us":952070,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":3096,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":12622,"spinlock_wait_cycles":159173120,"thread_start_us":618269,"threads_started":2,"wal-append.queue_time_us":330063}
W20260709 13:13:34.280503  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 55313us
W20260709 13:13:34.257689 10081 consensus_meta.cc:287] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.546s	user 0.006s	sys 0.003s
I20260709 13:13:34.189431 10526 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.271322 10286 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:34.285918  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 76638.8us
I20260709 13:13:34.291896  8101 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:34.293766  8101 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:34.278178 10525 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:34.301692  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:34.274623 10547 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:13:34.289968 10515 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:34.314874 10551 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:34.291705 10426 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:34.290915  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:34.328047 10551 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:34.328087 10525 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.328634 10551 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.329020 10551 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:34.328791 10525 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:34.332141 10511 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:34.332608 10511 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.332796 10525 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:34.332945 10511 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:34.337713 10532 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
W20260709 13:13:34.338440  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.339733 10545 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:34.280226 10524 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:34.350615  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 72857.7us
W20260709 13:13:34.313053 10528 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:13:34.286564 10081 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:34.353504  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:34.355008 10537 raft_consensus.cc:697] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:34.290961 10283 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:34.355682  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 62016.7us
I20260709 13:13:34.357491  8693 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:34.358533  8101 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:34.271625 10540 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:34.359321  9655 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:34.361233  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:34.366463  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 64806.7us
I20260709 13:13:34.367375  8029 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:34.360827  8175 leader_election.cc:336] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.501s (SENT)
I20260709 13:13:34.350836 10541 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:34.370551 10535 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:34.370733  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:34.371357  8056 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:34.371977  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:34.372269  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:34.372572  8057 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:34.372669  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:34.372875  8056 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:34.373374  8059 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:13:34.375295 10546 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:13:34.375728  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:34.376268  8055 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:34.376695 10286 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:34.375162  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:34.378217  8175 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:34.382889  8061 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:13:34.384140  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 183087us
I20260709 13:13:34.389845  8057 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:34.377836  8721 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:34.360946 10510 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:34.410368  9655 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:34.418128  8055 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:34.382103  9980 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:34.428995 10558 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:13:34.432636  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 216007us
I20260709 13:13:34.405336 10510 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:34.473042  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 115856us
W20260709 13:13:34.476917  8175 leader_election.cc:336] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.510s (SENT)
I20260709 13:13:34.477551  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:34.491257  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 127199us
I20260709 13:13:34.472667 10510 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:34.411700 10506 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.488862 10538 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.441901 10462 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:34.511536  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 77708.3us
I20260709 13:13:34.514529  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:34.520722  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:34.438825  9980 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.528369  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:34.494467 10319 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:34.529412  8060 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:34.498786 10426 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:34.494699 10459 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:34.530961  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 54045.4us
I20260709 13:13:34.528160 10471 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.534485  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:34.535555  9655 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.536191  9655 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.535538  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:34.537714  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1272) took 1634 ms. Trace:
I20260709 13:13:34.541414  8063 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:34.541873  8055 raft_consensus.cc:2376] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5. Re-sending same reply.
I20260709 13:13:34.542244  9655 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:34.528846  9876 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.542980  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:34.542470  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:34.508332 10286 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:13:34.550711  9655 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.548883  8060 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:34.553133  9655 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:34.554548  8175 leader_election.cc:336] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.526s (SENT)
I20260709 13:13:34.543239 10369 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:34.563591 10557 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.564180 10557 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.566002 10462 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:34.563778 10548 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:13:34.494230 10537 consensus_queue.cc:237] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:34.564582 10556 log.cc:927] Time spent T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:13:34.539763  8206 rpcz_store.cc:276] 0709 13:13:32.903713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.510523 (+606810us) service_pool.cc:224] Handling call
0709 13:13:33.511042 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:33.511632 (+   590us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:33.671739 (+160107us) log.cc:844] Serialized 14 byte log entry
0709 13:13:33.777153 (+105414us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:33.810229 (+ 33076us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b48002301d8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:13:33.839999 (+ 29770us) log.cc:844] Serialized 26 byte log entry
0709 13:13:33.840152 (+   153us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:33.840188 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:33.840260 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:34.533792 (+693532us) raft_consensus.cc:1685] finished
0709 13:13:34.537541 (+  3749us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:34.537662 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":39081,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":46,"raft.queue_time_us":4158,"raft.run_cpu_time_us":2037,"raft.run_wall_time_us":32644,"spinlock_wait_cycles":64657664,"thread_start_us":251606,"threads_started":2,"wal-append.queue_time_us":277062}
I20260709 13:13:34.552610 10319 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.548137 10552 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:34.593670  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:34.594127  9655 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:34.594843  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:34.597118  8135 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:34.527159 10538 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.558570 10283 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:34.608587  8729 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:34.558939 10553 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:34.564618  9980 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:34.552641  9876 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.612941  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:34.621637  8136 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:34.622219  8131 raft_consensus.cc:2410] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:34.613693  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:34.610437  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:34.571296 10506 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:34.562475 10369 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:34.616597  8134 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:13:34.594465 10356 consensus_meta.cc:287] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.755s	user 0.005s	sys 0.006s
W20260709 13:13:34.617563  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 66309.9us
I20260709 13:13:34.548518 10471 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.619046  8130 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:34.562023  9734 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.188s
I20260709 13:13:34.622967  8137 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:34.553591 10521 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:34.566279 10557 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:34.630465  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.668479  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:34.630877  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:34.634897 10524 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:34.599215 10511 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.600819 10319 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:34.639259  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:34.678380  8175 leader_election.cc:336] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.567s (SENT)
I20260709 13:13:34.640089  8205 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:34.649065  8204 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:34.642063  8207 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:13:34.686911  8207 pending_rounds.cc:85] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 2
I20260709 13:13:34.687150  8207 pending_rounds.cc:107] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:13:34.687377  8207 raft_consensus.cc:2889] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:34.643324  8135 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:34.620798  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:34.596760 10552 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:34.630124 10515 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:34.692023  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2313) took 1164 ms. Trace:
W20260709 13:13:34.647193  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.668210  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:34.645747  8102 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:34.693162  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 52202.8us
I20260709 13:13:34.623982  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:34.669189  8208 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
W20260709 13:13:34.648296 10514 consensus_meta.cc:287] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.773s	user 0.006s	sys 0.006s
W20260709 13:13:34.648370 10499 consensus_meta.cc:287] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.513s	user 0.006s	sys 0.004s
I20260709 13:13:34.644110 10369 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:34.634995 10434 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:34.641155  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 264164us
I20260709 13:13:34.678496  8206 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
W20260709 13:13:34.703136  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.679529  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:34.680739  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:34.688557  8063 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:34.672998 10553 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.690780 10574 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:34.694635  8132 raft_consensus.cc:2410] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:13:34.704078  8205 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:34.692411  8212 rpcz_store.cc:276] 0709 13:13:33.523879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.558534 (+ 34655us) service_pool.cc:224] Handling call
0709 13:13:33.574620 (+ 16086us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:33.640652 (+ 66032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:33.640684 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:33.641272 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 13:13:34.152016 (+510744us) spinlock_profiling.cc:243] Waited 8.88 ms on lock 0x7b1800421ae0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:13:34.183426 (+ 31410us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:34.183482 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:34.183574 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:34.679859 (+496285us) raft_consensus.cc:1685] finished
0709 13:13:34.679876 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:34.688078 (+  8202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7324,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":11391,"spinlock_wait_cycles":20434304,"thread_start_us":30700,"threads_started":1,"wal-append.queue_time_us":561787}
W20260709 13:13:34.704988 10554 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.200s	user 0.001s	sys 0.001s
I20260709 13:13:34.705320 10283 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:34.700402  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:34.707314  8726 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:34.705343  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:34.708290  8212 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:34.708798  8029 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:34.709925  8135 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:13:34.714246  8029 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:34.692396 10552 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:34.709946 10550 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:13:34.717757  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:34.729362  8208 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
W20260709 13:13:34.730674  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 232023us
W20260709 13:13:34.734989  8175 leader_election.cc:336] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.593s (SENT)
I20260709 13:13:34.733363  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:34.713011 10566 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.688290 10551 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.736490  8063 raft_consensus.cc:2376] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7. Re-sending same reply.
I20260709 13:13:34.763653  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:34.736059 10516 consensus_meta.cc:287] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.000s
I20260709 13:13:34.717329 10434 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:34.761579  8132 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:13:34.715802 10553 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:34.778357  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:34.704485 10499 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:34.774199 10569 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:13:34.774367  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 45872.1us
I20260709 13:13:34.774544  8722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 427) took 1241 ms. Trace:
W20260709 13:13:34.775020 10568 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:13:34.774971 10541 raft_consensus.cc:2764] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:13:34.775848 10538 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:34.815546  8208 raft_consensus.cc:2393] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:34.776681 10471 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:34.777530 10567 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 13:13:34.778350 10570 log.cc:927] Time spent T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:13:34.773480  8132 pending_rounds.cc:85] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 5
W20260709 13:13:34.784529  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 70351.1us
I20260709 13:13:34.804755  8057 raft_consensus.cc:2376] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5. Re-sending same reply.
I20260709 13:13:34.819888  8132 pending_rounds.cc:107] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:13:34.784346 10434 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:34.730489 10561 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.252s
I20260709 13:13:34.718277 10459 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:13:34.813313  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:34.814865  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:34.806764  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 290604us
I20260709 13:13:34.773514 10571 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:34.781749  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:34.844995  8134 raft_consensus.cc:2393] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:34.820969  8129 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:13:34.820200  8132 raft_consensus.cc:2889] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:34.807644 10578 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:34.821882  8175 leader_election.cc:336] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.663s (SENT)
I20260709 13:13:34.785521 10557 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.780771 10516 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:34.842828  8726 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:34.843580  8137 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:34.854425 10557 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.855479  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:34.855903 10516 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:34.856173  8137 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:34.816843  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:34.854017  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:34.858315  8063 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:34.826692  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:34.812176  8722 rpcz_store.cc:276] 0709 13:13:33.533372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.535251 (+  1879us) service_pool.cc:224] Handling call
0709 13:13:33.584156 (+ 48905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:33.780077 (+195921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:34.774293 (+994216us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b5400640354. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:34.774471 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":289109248}
I20260709 13:13:34.813534  9876 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:34.852290  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:34.820904 10444 consensus_meta.cc:287] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.671s	user 0.010s	sys 0.000s
I20260709 13:13:34.856915  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:34.853821 10578 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.856877 10557 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:34.858623  8175 leader_election.cc:336] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.709s (SENT)
I20260709 13:13:34.859195 10217 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:34.825831 10436 consensus_meta.cc:287] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.620s	user 0.007s	sys 0.000s
W20260709 13:13:34.827049 10572 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:13:34.860953  8057 raft_consensus.cc:2376] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7. Re-sending same reply.
W20260709 13:13:34.826952 10435 consensus_meta.cc:287] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.623s	user 0.008s	sys 0.001s
I20260709 13:13:34.828888  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:34.828794 10499 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:34.865036 10538 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.865471 10436 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:34.862665  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:34.877487  8134 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:34.825362 10319 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:34.873706  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:34.822858 10511 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:34.830112 10434 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:34.859756 10541 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.884068  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:34.887458  8134 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:34.888417  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:34.885376  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:34.886842  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:34.890606  8210 raft_consensus.cc:2393] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:34.891180  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:34.877285 10283 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.895570  8138 raft_consensus.cc:2393] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:34.895578  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:34.899827  8135 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:34.891847  8137 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 1.
I20260709 13:13:34.785581 10566 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:34.882653 10420 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:34.902524  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 43937.2us
I20260709 13:13:34.867056 10444 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.905305  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:35.015166  8137 raft_consensus.cc:2376] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5. Re-sending same reply.
I20260709 13:13:34.890982  9980 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.829440  8058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:47112
W20260709 13:13:35.020949  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 99682us
I20260709 13:13:35.021188  8058 raft_consensus.cc:606] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:35.021553  8058 raft_consensus.cc:614] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:34.878074 10435 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:34.891295 10578 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.052023  8131 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:35.027195 10356 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:34.894834 10541 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:34.906934 10426 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.122s
I20260709 13:13:35.028669  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:35.028918  8175 leader_election.cc:336] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.830s (SENT)
I20260709 13:13:35.067628  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:34.882709 10538 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.074301  8056 raft_consensus.cc:2368] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6.
I20260709 13:13:35.033742  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:35.035990 10444 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.036520  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.016091 10247 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:35.038130  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:34.879736 10510 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.060107  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:35.090077  8135 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:35.039125  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.095114  8730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 444) took 1099 ms. Trace:
I20260709 13:13:34.903692 10283 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.022385 10420 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.065693 10356 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.039085  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.102578  8058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:13:35.100628  8730 rpcz_store.cc:276] 0709 13:13:33.995706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.996126 (+   420us) service_pool.cc:224] Handling call
0709 13:13:35.033595 (+1037469us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:35.092285 (+ 58690us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4912000}
I20260709 13:13:35.052235  9980 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.107985  8058 raft_consensus.cc:606] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.108383  8058 raft_consensus.cc:614] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:35.108408  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:35.076000 10515 consensus_queue.cc:1059] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:13:35.112527  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:35.082149 10553 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.122509  8211 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:35.099633 10010 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.123184  8210 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:35.100418 10551 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:35.111446 10577 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 13:13:35.130179  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:35.134501  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.135214  8211 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:35.136135  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.136206 10283 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.136723  8211 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:35.137681  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.139101 10530 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:35.139598 10530 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:35.140615 10530 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.141844 10530 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.144140  9980 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.145582  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:35.128984  8203 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:35.151806  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:35.154307  8135 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
W20260709 13:13:35.154896 10582 log.cc:927] Time spent T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:13:35.121886 10589 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:35.149887  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:13:35.132453 10519 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:35.174400  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.152432  8137 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:35.183303 10436 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.187788  8134 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:35.192639  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:35.198748 10435 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.199855  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:35.200624  8137 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:35.201471  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.203625 10356 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.212769 10510 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.276162 10558 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:35.276710 10558 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.129776 10010 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.284857  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:35.288036  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 259134us
I20260709 13:13:35.292392 10435 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:35.282920 10541 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:35.262028 10587 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.133s	user 0.004s	sys 0.000s
I20260709 13:13:35.282560 10535 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.300933  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.306613  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.291483 10538 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.307595  8211 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.309077 10010 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:35.309587  8029 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.323432 10420 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:35.323922  8029 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.324199  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:35.313612 10521 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.325896 10521 consensus_queue.cc:1059] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:35.326514 10541 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:35.326764 10521 raft_consensus.cc:3055] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:35.326992 10541 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.327239 10521 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:35.327438 10541 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:35.328123 10521 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.329423 10521 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:35.331640  8029 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:35.333945  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:13:35.335112 10369 consensus_meta.cc:287] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.618s	user 0.007s	sys 0.001s
I20260709 13:13:35.335558 10369 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.337806 10369 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:35.321892 10535 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.339322 10525 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:35.339797 10525 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.340149 10525 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:35.340540  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.341194 10081 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.316010  8209 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:35.318476  8026 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.324954  8721 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:35.325229 10564 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:35.348878  8175 leader_election.cc:336] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.839s (SENT)
I20260709 13:13:35.359822  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:35.359892  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:35.341158  8210 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.361989  8211 raft_consensus.cc:2410] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:13:35.370361  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.369797 10010 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:35.340557 10535 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.378096  8101 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:35.378572  8209 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
W20260709 13:13:35.379702 10552 consensus_meta.cc:287] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.637s	user 0.007s	sys 0.002s
I20260709 13:13:35.384407 10291 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:35.387480  8175 leader_election.cc:336] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.913s (SENT)
I20260709 13:13:35.392210 10435 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:13:35.400553  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.405803  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.406663 10537 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:35.368193 10471 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:35.404927 10558 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:35.411154 10434 consensus_meta.cc:287] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.518s	user 0.003s	sys 0.006s
I20260709 13:13:35.412110  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.416363 10541 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.421130  8210 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:35.446218  8027 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.423177 10010 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.424829  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.426016 10574 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.427433 10521 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.427858 10579 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.429477  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:35.442466 10538 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:35.430868  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:35.455808  8058 raft_consensus.cc:2410] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:13:35.431529  8062 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:35.432422  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:35.433773 10540 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.434463  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:35.459051  8057 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.435177  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:35.436918  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.437610 10081 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.465907  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:35.443111  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:35.449997 10514 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
W20260709 13:13:35.467185  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 79739.1us
I20260709 13:13:35.458154 10471 raft_consensus.cc:697] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:35.467681  8057 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
W20260709 13:13:35.467929  8175 leader_election.cc:336] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.918s (SENT)
I20260709 13:13:35.468760  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:35.408483  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:35.469111  8175 leader_election.cc:336] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.943s (SENT)
I20260709 13:13:35.469575  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:35.469738  8724 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:35.470355  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 447) took 1138 ms. Trace:
I20260709 13:13:35.470393 10420 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:35.470633  8725 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:35.470820  8724 rpcz_store.cc:276] 0709 13:13:34.332261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:34.332425 (+   164us) service_pool.cc:224] Handling call
0709 13:13:35.408322 (+1075897us) spinlock_profiling.cc:243] Waited 9.28 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:35.470276 (+ 61954us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21347072}
I20260709 13:13:35.454195 10537 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.446113  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:35.447999  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.448700  8027 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:35.473631 10537 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:35.454723 10521 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.474792  8027 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:35.429754  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:35.476052  8054 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 6: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:35.476423 10521 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.477064  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:35.455762 10538 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:35.452606  8206 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:35.478850 10524 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.479410  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.479487 10524 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.456075 10579 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.480358  8206 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.449777 10535 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:35.449856 10534 consensus_queue.cc:1059] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:13:35.450084 10530 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.495048  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.495198 10530 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.496168  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.496560  8057 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.496747  8211 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:35.461493  8212 raft_consensus.cc:2393] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:35.451401 10434 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.497596 10521 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.498142 10521 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.499040 10434 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.499266 10530 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.490329 10525 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.458555 10010 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.501377 10521 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.463986  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:35.502827  8062 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:35.503677  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:35.504678  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:35.505175  8062 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:35.505514  8057 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:35.506666  8063 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 6: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:35.506893  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:35.454555 10574 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.507988  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:35.508513  8063 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:35.508695  8104 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.509522  8104 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.510281  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.510653 10010 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.511293  8104 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.452797  8203 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:13:35.465175  8055 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:35.512984 10462 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:35.513347 10462 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.513466  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:35.513697 10462 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.514142  8055 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:35.445751 10552 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.515209  8055 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:13:35.515854  8060 raft_consensus.cc:1240] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:35.516366  8055 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:35.516886 10552 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.465816 10081 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.517578 10545 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:35.518023 10545 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:35.438647  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 60622.9us
I20260709 13:13:35.518255  8060 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:13:35.519584 10081 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.468407 10471 consensus_queue.cc:237] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.520737  8029 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.521984  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.522926 10524 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.486172  8058 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:35.490834 10555 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:35.491515  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:35.524287 10555 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:35.524711  8061 raft_consensus.cc:2393] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:35.525226  8101 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.525260 10555 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.526467  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:35.527716 10547 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:35.528023  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:35.528427 10409 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:35.528474 10547 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:35.529371 10547 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:35.529992 10547 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:35.493317  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.548599  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.548274 10547 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.549157  8204 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:35.550812 10535 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:35.551534 10535 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:35.552043 10535 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.552413 10535 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.552568  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.552846 10462 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.553285  8212 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:35.493896 10540 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:35.554623 10540 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:35.554879 10462 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.444515  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.477926 10538 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.555027 10540 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:35.556670 10566 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.511333  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:35.557874  7961 catalog_manager.cc:5697] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 2 to 5, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 5 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:35.559037  8056 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:35.559425  8206 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:35.554261  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.559649  7961 catalog_manager.cc:5697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:35.560556  8212 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.560708  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.512504  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.518738 10571 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:13:35.520792  8175 leader_election.cc:336] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.953s (SENT)
I20260709 13:13:35.477910  8054 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:35.522504 10552 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:35.518492 10565 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:35.529277 10551 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:35.492179 10435 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:35.531106 10545 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:35.472988  8210 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:35.531858  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:35.532649 10400 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:35.506008  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:35.572072  8062 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:35.534593  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.535995 10514 consensus_queue.cc:1243] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:13:35.536321  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:35.576545 10538 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.536368 10443 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:35.536988  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:35.578759  8724 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:35.579030 10519 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:35.537356  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 91311.6us
I20260709 13:13:35.579629 10519 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.580525  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:35.537993 10580 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:35.538070  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.583369  8176 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.538853 10534 raft_consensus.cc:3055] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:35.539621  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:35.511214 10541 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.538729 10319 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:13:35.482172 10579 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.541041 10555 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.538125  8721 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:35.542858  8063 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:35.546707 10574 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.555001  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.617477  8207 raft_consensus.cc:2410] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:13:35.561494  8206 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:35.562384  8209 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:35.562914  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:35.564525  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 52072.2us
I20260709 13:13:35.490805 10558 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.567798  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:35.507418  8057 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.536484  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.566287  7961 catalog_manager.cc:5697] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 3, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 3 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:35.563503 10571 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.573668  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.574306 10462 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:35.575331  8204 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:35.581756  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:35.582510  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:35.584391  8134 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:35.646800  8729 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:35.586647  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:35.587196  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 49224.2us
I20260709 13:13:35.647531  8061 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:35.648624  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.538708  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.649427  8205 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:35.592617 10524 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:35.650475  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.584352  8212 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:35.587275  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.583709  8101 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.568961 10566 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:35.590108  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:35.655026  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:35.569155 10526 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.031s	sys 0.024s
I20260709 13:13:35.568800 10516 raft_consensus.cc:3055] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:35.568347 10565 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.593415  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:35.657302  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.657521  8135 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:35.596773  8132 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:13:35.658131  8211 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.599331  8056 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:13:35.589092 10534 raft_consensus.cc:740] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:35.600338  8725 raft_consensus.cc:2410] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:13:35.602398 10319 raft_consensus.cc:3055] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:13:35.609006  7959 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34120 (request call id 246) took 12140 ms (client timeout 15000 ms). Trace:
I20260709 13:13:35.671281  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:35.673764  8057 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:35.646064 10571 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.622689  8104 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:35.659544 10526 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.032s	sys 0.024s
I20260709 13:13:35.586720 10580 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.631115  8210 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:35.591722 10552 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:35.646476  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:35.652138  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 68498.7us
I20260709 13:13:35.727376  8207 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:35.727675  8101 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.728030  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1289) took 1046 ms. Trace:
I20260709 13:13:35.728451  8207 rpcz_store.cc:276] 0709 13:13:34.681344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.646211 (+964867us) service_pool.cc:224] Handling call
0709 13:13:35.727959 (+ 81748us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:35.594604 10531 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:35.729404 10531 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:35.598101  8058 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:13:35.661245  8209 raft_consensus.cc:2410] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:13:35.734035  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1287) took 1053 ms. Trace:
I20260709 13:13:35.661123  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.652804 10426 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:35.667290  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:35.737396  8054 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:35.663244 10519 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.669829  8061 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:35.670817  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:35.738718  8062 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.671456  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.734467  8209 rpcz_store.cc:276] 0709 13:13:34.680771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.444260 (+763489us) service_pool.cc:224] Handling call
0709 13:13:35.733969 (+289709us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:35.671957  8206 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:35.609853 10538 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.662781  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:35.569378 10525 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:35.675480  8693 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.746469  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.747149  8205 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:35.694169 10530 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.748117  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.653331 10524 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.718811  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:35.717595  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:35.694182  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 71608.4us
I20260709 13:13:35.663745 10565 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.663410 10435 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.722873  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:35.730991 10580 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:35.646471  7961 catalog_manager.cc:5697] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 2, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:35.741117  8211 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:35.762089  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.763099  8211 raft_consensus.cc:2410] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:13:35.763995  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.764853  8211 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 3: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:13:35.736532  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 75493.2us
I20260709 13:13:35.766561  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.770042  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:35.770648  8210 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:35.740653 10434 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:35.663496 10462 raft_consensus.cc:697] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:35.659293 10516 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:35.746214  8727 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
W20260709 13:13:35.747087  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 71700.2us
I20260709 13:13:35.777194  8211 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:13:35.690057  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.597178 10283 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 13:13:35.750460  8136 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:35.750653  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.689406 10521 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.726483 10081 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:35.673591 10319 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:35.689661 10541 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:35.690428 10566 raft_consensus.cc:697] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:35.789206 10426 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:35.789745 10426 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.696156  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.653944  9980 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:35.738131  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:35.775000  8135 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
I20260709 13:13:35.776094  8054 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:35.767966  8029 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.798964  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.777807  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1290) took 1096 ms. Trace:
I20260709 13:13:35.778291  8212 raft_consensus.cc:2393] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:35.694478 10574 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:13:35.751629 10524 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:35.753479 10519 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:35.758528 10356 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:35.785380  8104 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.802022  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.790671  8208 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:13:35.793203  7961 catalog_manager.cc:5697] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 3 to 5, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 5 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:35.752846  8205 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:35.795275  8062 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:35.789633 10566 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.805016  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:35.777467  8203 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:35.825830  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 333) took 1123 ms. Trace:
I20260709 13:13:35.829188  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:35.829384 10525 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:35.829699  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:35.824594 10471 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:35.825131  7961 catalog_manager.cc:5697] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 2 to 4, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 4 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:35.839308  8204 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:35.840160 10516 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.799631  8211 rpcz_store.cc:276] 0709 13:13:34.681622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.776424 (+1094802us) service_pool.cc:224] Handling call
0709 13:13:35.776951 (+   527us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:35.777586 (+   635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.777760 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:35.841111 10356 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:35.805933 10519 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:35.787348 10521 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.799451 10409 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
W20260709 13:13:35.673805  7959 rpcz_store.cc:269] 0709 13:13:23.487332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:23.520139 (+ 32807us) service_pool.cc:224] Handling call
0709 13:13:35.608770 (+12088631us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:31.514024 (+     0us) spinlock_profiling.cc:243] Waited 7.47 ms on lock 0x7b5400172ba8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc13d29e1 00007f6dc13d164c 00007f6dbea60adc 00007f6dbea4c0a6 00007f6dbea4bd58 00007f6dbea509f1 00007f6dbea509a9 00007f6dbea50939 00007f6dbea50901 00007f6dbea4fb2d 000000000038f354
0709 13:13:31.528510 (+ 14486us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:31.619153 (+ 90643us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:31.626637 (+  7484us) write_op.cc:435] Acquired schema lock
0709 13:13:31.638389 (+ 11752us) tablet.cc:662] Decoding operations
0709 13:13:31.639874 (+  1485us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:31.639980 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:31.640019 (+    39us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:13:31.671515 (+ 31496us) tablet.cc:701] Row locks acquired
0709 13:13:31.671532 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:13:31.671678 (+   146us) write_op.cc:270] Start()
0709 13:13:31.671854 (+   176us) write_op.cc:276] Timestamp: P: 1783602811671629 usec, L: 0
0709 13:13:31.671877 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:13:31.672538 (+   661us) log.cc:844] Serialized 5186 byte log entry
0709 13:13:32.482466 (+809928us) op_driver.cc:464] REPLICATION: finished
0709 13:13:33.090307 (+607841us) write_op.cc:301] APPLY: starting
0709 13:13:33.186805 (+ 96498us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:33.187290 (+   485us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:33.187342 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:33.303947 (+116605us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:33.304374 (+   427us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:13:33.304452 (+    78us) write_op.cc:312] APPLY: finished
0709 13:13:33.305026 (+   574us) log.cc:844] Serialized 95 byte log entry
0709 13:13:33.305257 (+   231us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:33.305467 (+   210us) write_op.cc:454] Released schema lock
0709 13:13:33.305566 (+    99us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:33.306003 (+   437us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:33.306038 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":30959,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":605953,"apply.run_cpu_time_us":5392,"apply.run_wall_time_us":217119,"num_ops":13,"prepare.queue_time_us":119504,"prepare.run_cpu_time_us":55479,"prepare.run_wall_time_us":1421821,"raft.queue_time_us":195266,"raft.run_cpu_time_us":32483,"raft.run_wall_time_us":330993,"replication_time_us":780885,"spinlock_wait_cycles":17182464,"thread_start_us":558831,"threads_started":4,"wal-append.queue_time_us":263253,"wal-append.run_cpu_time_us":17727,"wal-append.run_wall_time_us":2377060}]]}
I20260709 13:13:35.854147 10525 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:13:35.843922  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 116332us
I20260709 13:13:35.829223  8029 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.773183 10530 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.829293  9980 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:35.845079  8210 rpcz_store.cc:276] 0709 13:13:34.702595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.779969 (+1077374us) service_pool.cc:224] Handling call
0709 13:13:35.784390 (+  4421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.825358 (+ 40968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:35.825401 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:35.825471 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:35.825531 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.825612 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:35.825760 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:35.830214  7945 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a003b4584d3747a381522d67299aaed2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 84d775d1767e4cc4aa7b9c627d8f0fb8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:35.774011 10435 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.795969 10081 raft_consensus.cc:697] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:35.802707 10579 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:35.804674 10434 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:35.775661 10534 consensus_queue.cc:260] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.873005  8726 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:13:35.867251 10516 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:35.876968 10462 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.878937 10426 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.874985  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:35.864921 10356 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:35.845800 10544 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:35.883883  8208 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:35.841524 10444 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:35.885059  7961 catalog_manager.cc:5697] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 3 to 4, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 4 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:35.866032 10519 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:35.872545  8725 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:35.890942 10540 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.868891  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2327) took 1017 ms. Trace:
I20260709 13:13:35.876283 10525 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:35.909129  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:35.911100  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 81948.6us
I20260709 13:13:35.918859 10471 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:13:35.914098  7945 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 119630us
I20260709 13:13:35.892143 10518 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:35.964880 10471 raft_consensus.cc:697] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:35.964939 10518 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.928135 10444 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.966190 10471 consensus_queue.cc:237] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.966804 10444 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:35.967901  8136 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:13:35.913015  9980 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.968693  8696 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.969161  9980 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:35.926911 10558 consensus_queue.cc:1059] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:13:35.911283  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:35.971684 10574 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:35.970085  8696 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.934037 10356 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:35.936506  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 415713us
I20260709 13:13:35.937850 10524 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.943444  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:35.978681  8135 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:35.946049 10530 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.947402 10521 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.948067  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2328) took 1096 ms. Trace:
I20260709 13:13:35.950291 10525 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:35.964111 10540 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:35.972061 10574 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:35.973331  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 62129.8us
I20260709 13:13:35.933017  8029 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:35.989913  8175 leader_election.cc:336] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 2.034s (SENT)
I20260709 13:13:35.990509  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:35.992612  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 59704.5us
I20260709 13:13:35.973489  8696 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.974105  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:35.995257  8101 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.907577 10535 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:35.930194 10434 raft_consensus.cc:697] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:35.980291  7961 catalog_manager.cc:5697] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 3, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 3 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:35.999608  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:35.979014 10409 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:35.980623 10557 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:35.993613  8029 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:36.001137  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:36.002879  8101 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:13:35.992633  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 190710us
I20260709 13:13:36.005465  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:36.007010  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:36.009334  8101 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:36.010812  8101 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:36.023653  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.025409  8730 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:36.027000  8029 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:35.988538 10534 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:36.028370  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:36.047899  8729 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:36.048694  8209 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:36.050323  8027 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:36.000101 10574 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:36.052321  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:36.053005  8207 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:36.000025 10521 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:36.022858 10551 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:35.931473  8204 rpcz_store.cc:276] 0709 13:13:34.851786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.838679 (+986893us) service_pool.cc:224] Handling call
0709 13:13:35.838997 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.856804 (+ 17807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.868822 (+ 12018us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:36.031798  7961 catalog_manager.cc:5697] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:36.004878 10400 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:36.004457 10562 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:36.058804  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:36.020287 10541 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:36.026428 10526 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:36.023854 10544 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:36.013074 10530 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:36.069320  8693 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:36.029454 10369 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:36.031441 10010 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:13:36.045964 10566 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:36.046006 10540 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:36.084764  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 94900.6us
W20260709 13:13:36.069314 10604 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:13:36.010131 10606 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:13:36.028246 10537 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:13:36.068084 10521 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:36.079909 10547 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:36.074839 10541 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:36.058542 10557 consensus_queue.cc:1059] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:13:36.079700 10526 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:36.098929  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 48625.6us
I20260709 13:13:36.053865 10545 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.079458 10552 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:36.110915  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 100179us
I20260709 13:13:36.080430  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:36.114717  8175 leader_election.cc:336] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 2.047s (SENT)
I20260709 13:13:36.136560  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:36.078306 10609 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:36.083611 10530 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:36.083834 10369 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:35.985026  8203 rpcz_store.cc:276] 0709 13:13:34.852014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.866312 (+1014298us) service_pool.cc:224] Handling call
0709 13:13:35.908667 (+ 42355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.919999 (+ 11332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:35.921225 (+  1226us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:35.926595 (+  5370us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:35.929128 (+  2533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.947842 (+ 18714us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:35.947997 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:36.060536 10436 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:13:36.083874 10400 raft_consensus.cc:697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:36.094322 10409 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:36.077119 10426 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:13:36.099823 10462 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.182664 10462 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.184327 10580 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:36.184796 10462 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:36.186616  8728 raft_consensus.cc:2393] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:36.092499 10551 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:36.187583 10544 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:36.187893 10551 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.188740 10524 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:36.189162 10531 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:36.138726  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.190789  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:36.191242  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:36.191622  8723 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:36.192313  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:36.193595  8134 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:13:36.194001 10535 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:36.140579  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.195534 10525 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:36.096311 10610 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:36.113852 10521 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:36.108736 10566 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:36.142315  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.067279 10081 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.158731  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:36.158859  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:36.159602  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:36.160256 10552 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:36.143960  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.314508  8137 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:36.143826  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.130812 10541 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:36.133270 10434 consensus_queue.cc:237] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.168784 10545 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:36.168931 10615 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:13:36.108383 10540 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.329337  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:36.329908  8136 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:36.169890 10600 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.399s	user 0.001s	sys 0.001s
I20260709 13:13:36.142479 10530 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:36.170154 10614 log.cc:927] Time spent T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:13:36.142076 10571 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.175542 10623 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:36.178761  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 64081us
I20260709 13:13:36.178843 10400 consensus_queue.cc:237] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:36.179914 10409 raft_consensus.cc:697] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:36.144471 10547 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:36.140321 10609 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:36.141858 10622 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:36.234853  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2341) took 1265 ms. Trace:
I20260709 13:13:36.144902  8722 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:36.256052  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 334) took 1553 ms. Trace:
W20260709 13:13:36.285328  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 84167.1us
I20260709 13:13:36.144608 10369 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:36.256397  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:36.135708 10526 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:36.290812  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:36.295688 10531 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:36.168977 10435 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:36.254979 10580 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:36.304230  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 298820us
I20260709 13:13:36.303694  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2336) took 1441 ms. Trace:
W20260709 13:13:36.374370 10619 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:13:36.375491  8104 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:36.374624  8210 rpcz_store.cc:276] 0709 13:13:34.862557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.913342 (+1050785us) service_pool.cc:224] Handling call
0709 13:13:35.913586 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.913766 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:36.078112 (+164346us) log.cc:844] Serialized 14 byte log entry
0709 13:13:36.118697 (+ 40585us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:36.119074 (+   377us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:36.119539 (+   465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.302770 (+183231us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:36.303567 (+   797us) inbound_call.cc:177] Queueing success response
0709 13:13:36.312585 (+  9018us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b48002301d8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
Metrics: {"prepare.queue_time_us":260643,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":63133,"raft.run_cpu_time_us":2521,"raft.run_wall_time_us":132963,"spinlock_wait_cycles":33466752,"thread_start_us":163635,"threads_started":2,"wal-append.queue_time_us":102902}
I20260709 13:13:36.354071 10623 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:36.376681  8210 raft_consensus.cc:1240] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:36.377456 10635 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:36.377949 10623 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:36.305648 10607 raft_consensus.cc:3055] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:36.387116 10607 raft_consensus.cc:740] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:36.388018 10607 consensus_queue.cc:260] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.388221 10547 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.389219 10607 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:36.316286  8061 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:36.356424 10609 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:36.317582  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:36.317835  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:36.394829  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:36.396739  8060 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:36.318398  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:36.397408  8175 connection.cc:256] RPC call timeout handler was delayed by -1.8756400600000234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2167s for a response
I20260709 13:13:36.290545  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:36.298831 10610 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:36.330024  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:36.402948  8054 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:36.332310  8058 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:36.278950  9980 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:36.338186  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.286368 10565 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.328406 10552 raft_consensus.cc:697] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:36.356436  8208 rpcz_store.cc:276] 0709 13:13:34.899813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.924764 (+1024951us) service_pool.cc:224] Handling call
0709 13:13:35.974808 (+ 50044us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.974941 (+   133us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:36.097427 (+122486us) log.cc:844] Serialized 14 byte log entry
0709 13:13:36.164565 (+ 67138us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:36.164634 (+    69us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:36.164695 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.164777 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:36.164909 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1596,"prepare.queue_time_us":199714,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":23,"raft.queue_time_us":25342,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":60,"thread_start_us":158811,"threads_started":2,"wal-append.queue_time_us":192553}
I20260709 13:13:36.360703  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:36.361366  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:36.405443  8063 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:36.359440  8132 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:36.364755  8138 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 3: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:36.360164 10622 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:36.373114 10574 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:36.362309 10555 consensus_meta.cc:287] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.731s	user 0.011s	sys 0.000s
I20260709 13:13:36.381035  8210 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:13:36.419293  8210 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:36.421463 10574 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:36.422690 10635 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.423173 10635 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:36.386587 10426 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:36.317315  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:36.361732  8205 rpcz_store.cc:276] 0709 13:13:34.702908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.803934 (+1101026us) service_pool.cc:224] Handling call
0709 13:13:35.804245 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:35.804435 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:35.986524 (+182089us) log.cc:844] Serialized 14 byte log entry
0709 13:13:36.255731 (+269207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:36.255793 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:36.255834 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.255894 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:36.255998 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2217,"prepare.queue_time_us":309942,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":3751,"raft.queue_time_us":63419,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":21396,"thread_start_us":426941,"threads_started":3,"wal-append.queue_time_us":129862}
I20260709 13:13:36.397099  8061 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:36.397842  8055 raft_consensus.cc:2393] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:36.328040  8056 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
W20260709 13:13:36.398080  8175 connection.cc:256] RPC call timeout handler was delayed by -1.855428757000027s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2533s for a response
W20260709 13:13:36.437984  8175 connection.cc:256] RPC call timeout handler was delayed by -1.8180400599999302s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2231s for a response
W20260709 13:13:36.438365  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.408923  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:36.439147  8729 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:36.411482  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2326) took 1560 ms. Trace:
I20260709 13:13:36.413089 10622 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:36.433068  8063 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
W20260709 13:13:36.439543  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7084065510000528s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2489s for a response
W20260709 13:13:36.443375  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6607657350000409s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22490000000000002s for a response
W20260709 13:13:36.443689  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6551400599998942s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2412s for a response
W20260709 13:13:36.443974  8175 connection.cc:256] RPC call timeout handler was delayed by -1.5467287569999826s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2876s for a response
I20260709 13:13:36.439647  8212 rpcz_store.cc:276] 0709 13:13:34.851514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.807585 (+956071us) service_pool.cc:224] Handling call
0709 13:13:35.807870 (+   285us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:35.808183 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:35.808210 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:35.808882 (+   672us) log.cc:844] Serialized 52 byte log entry
0709 13:13:35.893563 (+ 84681us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:35.893601 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:35.893669 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:36.411213 (+517544us) raft_consensus.cc:1685] finished
0709 13:13:36.411248 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:36.411417 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":888,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":21271,"thread_start_us":54786,"threads_started":1,"wal-append.queue_time_us":160515}
I20260709 13:13:36.444705  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:36.446671  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:36.447681  8133 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:36.450018 10635 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:36.450733  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:36.456197 10571 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:36.446142 10409 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:36.462880  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:36.466288  8055 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:36.413920  8723 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:36.472821  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:36.462941  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.474171  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1295) took 1125 ms. Trace:
W20260709 13:13:36.442325 10516 consensus_meta.cc:287] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.002s
I20260709 13:13:36.475333  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:36.412618 10610 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:36.480688  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:36.396687 10444 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.481139  8060 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:36.483330  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:36.480989  8061 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:36.481503  8204 rpcz_store.cc:276] 0709 13:13:35.348809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.070859 (+722050us) service_pool.cc:224] Handling call
0709 13:13:36.085566 (+ 14707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:36.085833 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:36.295334 (+209501us) log.cc:844] Serialized 14 byte log entry
0709 13:13:36.473733 (+178399us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:36.473818 (+    85us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:36.473880 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.473967 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:36.474103 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2616,"prepare.queue_time_us":88951,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":2729,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":384498,"threads_started":2}
W20260709 13:13:36.482394  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 455430us
I20260709 13:13:36.487964  8727 raft_consensus.cc:2410] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:13:36.405191 10627 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:13:36.488855  8059 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:36.490072  8061 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:13:36.368995 10566 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:36.482091 10010 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:36.478601 10630 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:36.446645 10426 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.525754  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.484848 10524 consensus_queue.cc:1059] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:13:36.531613  8060 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:36.532673  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:36.533383  8060 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:36.485023 10544 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:13:36.523801 10434 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:36.543905  8136 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:36.545034  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.545867  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:36.546057  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.546922  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:36.547605  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.540994 10540 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:36.555915  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.556810  8138 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:36.562975  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.569073  8136 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
W20260709 13:13:36.571151  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:36.571849  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 91253.5us
I20260709 13:13:36.545397 10524 raft_consensus.cc:2764] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:13:36.591534  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:36.522178 10552 consensus_queue.cc:237] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.552855 10630 raft_consensus.cc:3055] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:36.601456  8138 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:36.553020 10471 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.602808  8175 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:36.543730 10434 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.603546  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:36.605931 10630 raft_consensus.cc:740] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:36.605731 10648 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:36.617784 10544 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:36.617461  9980 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:36.636672  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:36.609951 10471 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.643493  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:36.644327  8055 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:36.645749  8729 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:36.627995 10565 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:36.626586  8725 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 3.
I20260709 13:13:36.650134 10648 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:13:36.648072 10519 consensus_meta.cc:287] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.675s	user 0.009s	sys 0.002s
I20260709 13:13:36.648162 10444 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:36.661772  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:36.645001 10541 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.681648  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:36.682641 10625 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 13:13:36.683615  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:36.684180  8054 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:36.684996  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:36.685619  8054 raft_consensus.cc:2393] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:36.666513 10630 consensus_queue.cc:260] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.687139  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:36.687381  8722 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:36.687563 10630 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:36.687712  8057 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:36.688608  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:36.689893  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2335) took 1827 ms. Trace:
I20260709 13:13:36.689049 10369 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:36.654937 10526 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.690876  8726 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:13:36.800763  8057 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:13:36.703569  8723 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:36.686259  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:36.727854  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 452) took 1150 ms. Trace:
I20260709 13:13:36.806600  8728 rpcz_store.cc:276] 0709 13:13:35.577553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.589827 (+ 12274us) service_pool.cc:224] Handling call
0709 13:13:36.727596 (+1137769us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b5400653954. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:36.727782 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1222770048}
I20260709 13:13:36.805238  8026 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:36.808871  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:36.690299  8211 rpcz_store.cc:276] 0709 13:13:34.862314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.018026 (+1155712us) service_pool.cc:224] Handling call
0709 13:13:36.018332 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:36.018754 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:36.018802 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:36.019270 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 13:13:36.112437 (+ 93167us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:36.325245 (+212808us) log.cc:844] Serialized 14 byte log entry
0709 13:13:36.325431 (+   186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.325495 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:36.689661 (+364166us) raft_consensus.cc:1685] finished
0709 13:13:36.689683 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:36.689837 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2778,"prepare.queue_time_us":62515,"prepare.run_cpu_time_us":22,"prepare.run_wall_time_us":22,"raft.queue_time_us":4314,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":36990,"thread_start_us":248414,"threads_started":2,"wal-append.queue_time_us":357743}
W20260709 13:13:36.712067 10646 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
W20260709 13:13:36.712383 10639 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
W20260709 13:13:36.713598 10650 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:13:36.817416  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:36.830264  8055 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 3.
W20260709 13:13:36.716773 10647 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.165s	user 0.000s	sys 0.004s
I20260709 13:13:36.819789  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:36.661082 10519 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:36.714928 10444 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:13:36.713083 10643 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.168s	user 0.002s	sys 0.000s
W20260709 13:13:36.712774 10642 log.cc:927] Time spent T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:13:36.822844  8730 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:13:36.803251 10426 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:36.819952  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.812438 10540 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:36.836609  8132 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:36.838379  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.722465 10565 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.107s
I20260709 13:13:36.721421 10521 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.814420 10645 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:36.821738 10217 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:36.714542 10654 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.126s
I20260709 13:13:36.844756  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.833952  8721 raft_consensus.cc:2410] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:13:36.718046 10530 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:36.851155  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 205235us
I20260709 13:13:36.853991  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.828717 10518 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:36.721251 10622 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:36.861598  8135 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:13:36.856185  8132 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:36.861820  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:36.821952 10525 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:13:36.692533 10319 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:36.868328  8056 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
W20260709 13:13:36.870532  7945 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a003b4584d3747a381522d67299aaed2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 84d775d1767e4cc4aa7b9c627d8f0fb8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:36.869112  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:36.873927  8132 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:36.875257  8063 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:36.713976 10648 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.877544  8060 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:36.878780 10319 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:36.891101 10541 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:36.892612 10640 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.373s	user 0.000s	sys 0.001s
I20260709 13:13:36.864989 10645 raft_consensus.cc:697] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:36.893811 10645 consensus_queue.cc:237] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.880841 10606 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:36.883548  8027 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:36.883956 10530 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:36.885499 10622 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:36.887384  8059 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:36.887537 10610 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.872668  8724 raft_consensus.cc:1240] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:36.889245 10471 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:36.889564 10609 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:36.894258 10519 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:36.865216 10540 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:36.890048 10434 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:36.846064 10444 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:36.903714  8027 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:36.898319 10606 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:36.908998 10526 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:36.909768  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:36.910172  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:36.910948 10369 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:36.910787 10659 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:36.906132 10649 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:13:36.912051  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2347) took 1642 ms. Trace:
I20260709 13:13:36.913357  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:36.936060  8136 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:37.176082  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:36.985037  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.018255  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 304) took 1007 ms. Trace:
W20260709 13:13:37.032338  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:37.170594  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 260488us
I20260709 13:13:37.180897  8062 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:36.939040  8027 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:37.181627  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:36.915421 10541 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:37.014602  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:37.182232  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 243287us
I20260709 13:13:37.224587 10541 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.013216 10606 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.179519  8135 rpcz_store.cc:276] 0709 13:13:36.011268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.011487 (+   219us) service_pool.cc:224] Handling call
0709 13:13:36.046538 (+ 35051us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:36.862097 (+815559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.018005 (+155908us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b54001aac54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:37.018198 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1873789696}
I20260709 13:13:36.962986  8129 raft_consensus.cc:2393] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:37.045245  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:37.226001  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:36.937961  8059 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:36.981737 10471 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:36.962185  8055 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:13:37.194515  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.215730 10400 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.183548 10521 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:37.265457  8729 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:37.188385  8211 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:13:37.206058  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:37.228300  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 47822.1us
I20260709 13:13:37.225644  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:37.267028  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.234297 10443 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.267549  8211 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:37.267668  8728 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:37.033419  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:37.233723  8138 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:37.233120  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:37.158463 10634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.006s	sys 0.022s
W20260709 13:13:36.970006  8026 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.687s (SENT) [suppressed 26 similar messages]
W20260709 13:13:37.232116  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.180341  8209 rpcz_store.cc:276] 0709 13:13:35.269464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.069856 (+800392us) service_pool.cc:224] Handling call
0709 13:13:36.084250 (+ 14394us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:36.326777 (+242527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:36.326809 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:36.333794 (+  6985us) log.cc:844] Serialized 52 byte log entry
0709 13:13:36.791078 (+457284us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:36.821296 (+ 30218us) log.cc:844] Serialized 14 byte log entry
0709 13:13:36.821521 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.821625 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:36.911844 (+ 90219us) raft_consensus.cc:1685] finished
0709 13:13:36.911875 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:36.912001 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17247,"prepare.queue_time_us":30256,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":25017,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":19637,"thread_start_us":649182,"threads_started":2,"wal-append.queue_time_us":532458}
I20260709 13:13:37.235464  8723 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260709 13:13:37.237870  8205 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:37.234949  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:37.363852  8723 pending_rounds.cc:85] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: Aborting all ops after (but not including) 4
I20260709 13:13:37.364130  8723 pending_rounds.cc:107] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:13:37.364364  8723 raft_consensus.cc:2889] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:37.364267  8203 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 2.
I20260709 13:13:37.364617  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.236490  8062 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:37.240790  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.083178  8721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 584) took 1256 ms. Trace:
I20260709 13:13:37.259110  8054 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:37.203755 10319 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:37.018669  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1298) took 1654 ms. Trace:
I20260709 13:13:37.015463 10610 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:37.023118 10609 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:36.981917 10636 raft_consensus.cc:2764] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:13:37.261130  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.261663  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:37.370225  8059 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:13:37.019452  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.212046  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:37.263265  8057 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:13:37.183938 10526 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:37.300632  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:37.268518  8061 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 6: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:37.301482  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:37.234122 10511 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
W20260709 13:13:37.348569  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 81301.5us
W20260709 13:13:37.352240  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.367719  8206 rpcz_store.cc:276] 0709 13:13:35.364689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.090691 (+726002us) service_pool.cc:224] Handling call
0709 13:13:36.091080 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:36.296032 (+204952us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:36.321778 (+ 25746us) log.cc:844] Serialized 14 byte log entry
0709 13:13:36.524962 (+203184us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:36.525396 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 13:13:36.525508 (+   112us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:36.525537 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:36.525600 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:36.712042 (+186442us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b1800407aa0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:13:37.018388 (+306346us) raft_consensus.cc:1685] finished
0709 13:13:37.018498 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:37.018619 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92704,"prepare.queue_time_us":26654,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":52,"raft.queue_time_us":111874,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":1384,"spinlock_wait_cycles":8091008,"thread_start_us":367581,"threads_started":3,"wal-append.queue_time_us":206999}
I20260709 13:13:37.379981  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.380527  8206 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:13:37.347819  8210 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:37.268244  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:37.354440  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1367) took 1215 ms. Trace:
I20260709 13:13:37.265154 10400 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.389802  8133 rpcz_store.cc:276] 0709 13:13:36.138821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.140235 (+  1414us) service_pool.cc:224] Handling call
0709 13:13:37.354218 (+1213983us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54001f18d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:37.354386 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308005376}
I20260709 13:13:37.258752 10541 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:36.935631 10540 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.354029  9980 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.361029  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.276367 10547 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 13:13:37.365150  8205 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:37.366035  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:37.411806  8059 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:37.367007  8696 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:37.412714  8730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 445) took 1125 ms. Trace:
I20260709 13:13:37.413091  8730 rpcz_store.cc:276] 0709 13:13:36.287635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.288715 (+  1080us) service_pool.cc:224] Handling call
0709 13:13:36.289074 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:36.836120 (+547046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.412501 (+576381us) spinlock_profiling.cc:243] Waited 533 ms on lock 0x7b540060de54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:37.412663 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1226888448}
I20260709 13:13:37.416229  8206 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:37.355523 10471 raft_consensus.cc:697] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:37.387245  8054 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:37.389473  8057 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:13:37.366816 10634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.007s	sys 0.026s
W20260709 13:13:37.393672  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 181709us
I20260709 13:13:37.398001  8211 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:37.368739 10319 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:37.428848  8205 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:37.429574  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:37.430016  8205 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:37.430748  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:37.417619 10668 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:13:37.403875  8208 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:37.404574  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.378050 10519 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.367684  8721 rpcz_store.cc:276] 0709 13:13:35.826663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.828930 (+  2267us) service_pool.cc:224] Handling call
0709 13:13:37.082916 (+1253986us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b54005802d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:37.083100 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2149248384}
W20260709 13:13:37.412109  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 442114us
I20260709 13:13:37.417315  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:37.361877 10443 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.436280  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:37.437726 10443 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:37.419950  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:37.438448  8138 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:37.439893  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:37.441403 10630 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:37.442749  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:37.420499  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.443862  8211 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 6: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:37.444936  8211 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:37.411559 10369 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:37.446241 10369 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.448408 10369 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:37.450610  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:37.452554  8175 leader_election.cc:400] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6.
I20260709 13:13:37.452895  8175 leader_election.cc:403] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:37.453879  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:37.399875  8210 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:37.438352 10443 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:37.455636 10443 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.456043 10443 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:37.461074  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1361) took 1324 ms. Trace:
I20260709 13:13:37.461470  8129 rpcz_store.cc:276] 0709 13:13:36.137003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.137104 (+   101us) service_pool.cc:224] Handling call
0709 13:13:37.460887 (+1323783us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b54003c87d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:37.461021 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1763855744}
I20260709 13:13:37.464187  8134 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:13:37.465801 10400 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:37.469306  8101 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:37.414932 10636 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:37.453959  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 446) took 1075 ms. Trace:
I20260709 13:13:37.399428 10609 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:13:37.414772  9980 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.455636 10565 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.470674  8026 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:37.475394  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 94581.2us
I20260709 13:13:37.476186  8730 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:37.432024  8205 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:13:37.494462 10610 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:37.438236 10519 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.494959 10610 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.495537  8104 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:37.497062 10519 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:37.425450 10319 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:37.491218 10547 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:37.477233 10648 raft_consensus.cc:2764] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:13:37.477618 10649 raft_consensus.cc:2764] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:13:37.498675  8207 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:37.498543  8205 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:13:37.511943  8205 pending_rounds.cc:85] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 3
I20260709 13:13:37.512220  8205 pending_rounds.cc:107] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:13:37.512514  8205 raft_consensus.cc:2889] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:37.503280  8026 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:37.522976  8208 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:37.400415 10659 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:37.552371  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 185439us
I20260709 13:13:37.570206  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:37.571871  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2249) took 1253 ms. Trace:
I20260709 13:13:37.572351  8063 rpcz_store.cc:276] 0709 13:13:36.318037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.467486 (+149449us) service_pool.cc:224] Handling call
0709 13:13:36.467835 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:36.875598 (+407763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.571662 (+696064us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b5400052654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:37.571816 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":936558464}
I20260709 13:13:37.599241  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 589) took 1400 ms. Trace:
I20260709 13:13:37.601305  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2255) took 1281 ms. Trace:
W20260709 13:13:37.606153  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 110660us
I20260709 13:13:37.598598  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 582) took 1801 ms. Trace:
I20260709 13:13:37.505487 10511 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:37.483340 10636 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.497025 10610 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:37.609494  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 106318us
I20260709 13:13:37.608268  8727 rpcz_store.cc:276] 0709 13:13:35.797594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:35.797759 (+   165us) service_pool.cc:224] Handling call
0709 13:13:37.571394 (+1773635us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b540060ead4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:37.598529 (+ 27135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1461438848}
I20260709 13:13:37.616482  8026 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:37.599704  8726 rpcz_store.cc:276] 0709 13:13:36.198355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.198626 (+   271us) service_pool.cc:224] Handling call
0709 13:13:36.690378 (+491752us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:36.801729 (+111351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.599011 (+797282us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b5400650250. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:37.599174 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":964708096}
I20260709 13:13:37.601756  8056 rpcz_store.cc:276] 0709 13:13:36.319384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.490644 (+171260us) service_pool.cc:224] Handling call
0709 13:13:36.517611 (+ 26967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:36.870612 (+353001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.601050 (+730438us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b540028b4d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:37.601244 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":806187264}
I20260709 13:13:37.498324 10540 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:37.611554  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 157734us
I20260709 13:13:37.623162  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:37.540239 10630 raft_consensus.cc:2764] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:13:37.538825 10525 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:37.552920 10607 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:37.612663  8104 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:37.632807  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:37.637367  8026 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:37.575237 10659 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.614786 10435 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:37.477970  8724 rpcz_store.cc:276] 0709 13:13:36.378748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.390398 (+ 11650us) service_pool.cc:224] Handling call
0709 13:13:36.390735 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:36.917782 (+527047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.453734 (+535952us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b5400640354. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:37.453901 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1071220096}
W20260709 13:13:37.638624  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 169392us
I20260709 13:13:37.563125 10319 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.628305 10516 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:37.553066 10547 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.651496  8101 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:37.651692  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 43718us
I20260709 13:13:37.653061  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 305) took 1454 ms. Trace:
I20260709 13:13:37.542265 10443 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.598495 10369 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:13:37.660226  8134 rpcz_store.cc:276] 0709 13:13:36.198709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:36.278007 (+ 79298us) service_pool.cc:224] Handling call
0709 13:13:37.463961 (+1185954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:37.464564 (+   603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:37.652816 (+188252us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54003c87d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:37.652991 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2705749760}
I20260709 13:13:37.639263 10540 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:37.659994  8203 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:37.605281 10471 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.661732  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:37.543493 10565 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.660358  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:37.656628 10670 log.cc:927] Time spent T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:13:37.669866  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:37.668143  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:37.678119  8134 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:37.655915 10444 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:37.639048 10521 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:37.671334  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 99960.2us
W20260709 13:13:37.672384 10638 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.936s	user 0.002s	sys 0.010s
I20260709 13:13:37.650946 10525 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:37.672930  8204 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:37.659848  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:37.661131  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:37.670009  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:37.659350 10676 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:37.657387 10632 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6.
W20260709 13:13:37.669669 10673 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 13:13:37.665853  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 94847.9us
W20260709 13:13:37.671602 10669 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:13:37.656862 10672 log.cc:927] Time spent T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:13:37.660235 10674 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:37.656826 10516 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:37.674690 10400 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:13:37.657459 10679 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:37.647255 10607 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.672714 10511 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:37.697006 10636 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:37.696868 10516 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.697525 10636 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.697939 10516 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:37.697996 10636 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.699903 10606 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:37.700157 10636 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:37.702514 10319 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:37.702567 10426 raft_consensus.cc:3055] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:37.703280 10426 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:37.684911  8135 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:37.676618  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:37.704277  8130 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:37.685645  8101 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:37.684719  8137 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:37.673388 10540 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.669837 10657 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:37.657425 10667 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:13:37.687503  8176 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:37.691843  8693 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:37.659747 10666 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.687640 10519 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.743057 10519 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.705139  8203 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:37.688481 10444 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:37.686333 10681 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:37.688712 10680 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:37.711510  8101 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:37.690671 10676 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:13:37.736424  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 44640.8us
W20260709 13:13:37.736728  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 49308.9us
I20260709 13:13:37.741061 10443 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:37.735658  9980 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:37.692260 10683 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:37.697855 10607 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:37.695426 10679 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.710839 10540 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:37.736403 10609 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:37.749989 10444 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.710711 10511 raft_consensus.cc:697] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:37.773209 10686 consensus_queue.cc:1059] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:13:37.706817 10674 raft_consensus.cc:697] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:37.770104 10657 consensus_queue.cc:1059] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:13:37.767859 10682 consensus_queue.cc:1059] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:13:37.774999 10674 consensus_queue.cc:237] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.775976 10682 raft_consensus.cc:3055] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:37.776388 10682 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:37.777202 10682 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.777727 10659 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:37.751662 10319 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:37.765295 10443 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:37.763811 10426 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.779181 10426 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:37.780210  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 87709.8us
I20260709 13:13:37.773763 10511 consensus_queue.cc:237] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.782583 10547 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:37.767341  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:37.783306  8208 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 2.
I20260709 13:13:37.785118  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:37.761837  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 138780us
I20260709 13:13:37.769167  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:37.769935  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.772019  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.785743  8206 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:37.787613  8696 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:37.787753  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:37.789167  8211 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:37.760481 10677 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:37.789656  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 274454us
I20260709 13:13:37.706696 10687 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:13:37.796023  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.797621  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.804632  8206 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:37.766188 10684 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:37.802397  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:37.803047  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:37.805646  8209 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:37.804912  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:37.762989 10683 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:37.767168 10679 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:37.785399 10444 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:37.785841 10682 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:37.812845  8208 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:37.820075  8206 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:13:37.763123 10610 consensus_queue.cc:1059] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:13:37.822901  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:37.825820  8212 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:37.826756  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.825275  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:37.796422 10677 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.748956 10666 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.782164 10541 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.835544  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:37.804764 10688 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:37.839156  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 51463.9us
I20260709 13:13:37.840423  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:37.842088  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:37.842140  8175 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:37.810137 10683 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:37.844434  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 88821us
I20260709 13:13:37.760083 10680 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.845681  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:37.828220 10659 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:37.845036  8101 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:37.851856  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.852418  8207 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:37.851311  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:37.854646  8029 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:37.835374 10636 raft_consensus.cc:3055] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:37.829212 10319 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:37.856860  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.837726 10677 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:37.844476 10667 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:37.865991  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.866789  8063 raft_consensus.cc:1240] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:37.866855  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:37.867765  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:37.868111  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:37.877627  8057 raft_consensus.cc:2368] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
I20260709 13:13:37.877702  8054 raft_consensus.cc:2376] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7. Re-sending same reply.
I20260709 13:13:37.854616 10217 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:37.854377 10686 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:37.852231 10525 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:37.854188 10657 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:37.865322 10681 raft_consensus.cc:697] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:37.873528  8130 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:37.828982 10674 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:13:37.863962 10633 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:37.851915 10659 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:37.868743 10692 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.885576  8212 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:37.885934 10610 raft_consensus.cc:3055] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:37.886458  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:37.887380  8132 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:37.887890  8062 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:13:37.888382  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:37.861430 10636 raft_consensus.cc:740] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:37.858616  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:37.847239 10623 consensus_queue.cc:1059] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:13:37.921329 10623 raft_consensus.cc:3055] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:37.921684 10623 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:37.861121 10471 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:37.923853 10701 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.924340 10701 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.925480 10623 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.925864 10319 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.926663 10623 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:37.926798 10636 consensus_queue.cc:260] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.928016 10636 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:37.924278  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:37.908097 10565 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:37.932524 10516 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:37.932955 10516 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.933324 10516 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:37.934031 10674 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:37.934670 10674 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:37.935437 10674 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.936753 10674 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:37.843266 10676 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.938622 10701 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:37.939729 10666 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:37.940415  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:37.881022  8207 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:37.843276 10511 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:13:37.860504 10435 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:37.897908  8061 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 13:13:37.898406  8135 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:37.898566  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:37.882465 10686 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.848208 10688 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.906229  8132 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:13:37.907828  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:37.973799  8062 raft_consensus.cc:2393] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 6: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:37.891590 10657 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.915902  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:37.979597  8137 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:37.917742 10519 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:37.918685 10541 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:37.919740  8212 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:37.980304  8029 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:37.981375  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:37.982064  8212 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:13:37.984828  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:37.920256  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:37.920735  8058 raft_consensus.cc:2368] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6.
I20260709 13:13:37.904460 10547 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:37.947284 10701 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:37.986901 10633 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:37.986699 10701 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.987694 10633 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:13:37.948242  7945 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a003b4584d3747a381522d67299aaed2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 84d775d1767e4cc4aa7b9c627d8f0fb8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:13:37.968142 10693 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:13:37.989233 10701 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:37.968159  8207 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
W20260709 13:13:37.969143 10699 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:13:37.970784 10681 consensus_queue.cc:237] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.904095 10610 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:37.975014 10217 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:37.971560  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:37.990193  8207 pending_rounds.cc:85] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 4
I20260709 13:13:38.025218  8207 pending_rounds.cc:107] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:13:38.025473  8207 raft_consensus.cc:2889] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:37.993352  8721 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:37.994472 10659 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:37.914465 10692 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:37.998785 10521 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:37.905706 10689 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:38.052352 10689 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.000013 10680 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:37.948137 10435 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:37.999605 10516 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.001569 10444 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.004416 10683 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.005661  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:38.063159  8059 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:38.045881 10541 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:37.975270 10686 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:38.066217  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 308662us
I20260709 13:13:37.978242 10687 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.006395 10657 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:38.022056  8212 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
W20260709 13:13:38.022641  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 170271us
I20260709 13:13:37.985909  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:38.014068 10426 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.050753 10684 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:38.015511 10636 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.065840 10659 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:38.006255  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.073288  8724 raft_consensus.cc:2410] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:13:38.008589  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.081470 10666 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.045228 10633 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:38.081959 10666 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:38.082463  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 399660us
I20260709 13:13:38.082146 10633 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.083202  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:38.084302 10633 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:38.084635 10692 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:38.084692 10666 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:37.975710  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:38.085844  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 110214us
I20260709 13:13:38.086495  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.087423 10683 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:38.087517  8175 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:38.088029 10683 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:38.088596  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.088752 10680 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:38.089314 10680 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:38.089646  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.092626 10649 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:38.093621  8211 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:38.095203 10679 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.078902 10684 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.097293 10679 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:38.100117 10688 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:38.101483  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:38.102206  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.060112 10435 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:38.069415 10541 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:38.071384  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 85545us
I20260709 13:13:38.072363  8209 raft_consensus.cc:2393] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:38.072333  8212 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:38.024796 10217 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.061188 10702 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.018791 10547 raft_consensus.cc:697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:38.080400  8055 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:37.998170 10677 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.081636 10689 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:38.063273 10519 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:13:38.128288  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:38.117916  8101 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:38.119967  8029 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.120208  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:38.075295 10521 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:38.126462  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.127756  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.128150  8175 leader_election.cc:400] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 2 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 2. Current term is 3.
I20260709 13:13:38.133128  8205 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:38.088621 10683 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:38.130139  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 47025.3us
I20260709 13:13:38.135679  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:38.129184  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 62150us
I20260709 13:13:38.134238  8696 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:38.138881  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.102896 10636 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.150012  8726 raft_consensus.cc:2393] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 4: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:38.119599 10692 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:38.077396 10426 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.075042 10687 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.131135  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:38.141571  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:38.139791 10633 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:38.144868 10369 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:38.174687 10667 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:38.145974 10081 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:38.122028 10540 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:38.184376 10703 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:13:38.187011 10516 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:38.190531 10444 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:38.119230 10704 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:13:38.149221 10666 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:13:38.193810 10217 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:38.128525 10676 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:13:38.195318 10677 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:38.199398 10541 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:38.162844 10712 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:38.166661 10705 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:38.129769 10702 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.148600 10525 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:13:38.171679 10635 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:13:38.208966  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.192399 10426 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:38.165717 10565 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:13:38.156775 10511 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
W20260709 13:13:38.223922  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 89784.9us
I20260709 13:13:38.177595 10369 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:38.226352  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.167279 10692 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.225417  8724 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:38.176085 10683 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:13:38.286197 10683 raft_consensus.cc:3055] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:38.286319  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.286589 10683 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:38.287140  8132 raft_consensus.cc:2410] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:13:38.287400 10683 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.288718 10683 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.289584 10636 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:38.182511 10633 raft_consensus.cc:697] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:38.270850  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.273190  8205 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:13:38.291087 10633 consensus_queue.cc:237] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.292284  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:38.292958  8205 raft_consensus.cc:2393] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 4: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:38.275825  8207 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:38.277679  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:38.295926  8204 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:38.278367 10711 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:13:38.275405 10369 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:38.331935  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.279286 10702 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:38.340121  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 134551us
I20260709 13:13:38.280524  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:38.281471  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:38.282222 10692 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:38.201205 10547 consensus_queue.cc:237] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.347527  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:38.345710  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.263377 10541 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.077s
W20260709 13:13:38.339869  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 204245us
I20260709 13:13:38.358057  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.361349  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.340692 10217 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.341305 10632 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:38.347452 10636 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:38.372748  8204 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:13:38.374056  8696 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:38.370800  8104 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:38.374616  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.377692  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.378297  8057 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:38.379235  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.380851  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.381524  8062 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 6: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:38.330590 10688 consensus_queue.cc:1059] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:13:38.383369  8061 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:13:38.382218  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.388008  8061 pending_rounds.cc:85] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 3
I20260709 13:13:38.388123  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.388307 10516 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:38.388401  8061 pending_rounds.cc:107] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:13:38.388859  8061 raft_consensus.cc:2889] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:38.388929 10516 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:38.371407  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:38.389304  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:38.384509  8136 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:38.389845 10516 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.390380  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:38.390925  8059 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:38.368749 10426 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:38.392416  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.393021  8063 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:38.393764  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.394325  8056 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:38.394691  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.394547 10435 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.395710  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.396271  8054 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:38.396414  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.375063 10217 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.379745  8057 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:38.398043  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.385327  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.397059  8132 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:38.384626  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:38.396548 10435 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:38.367332 10702 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:38.404816  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 267004us
I20260709 13:13:38.385627  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.406062  8138 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:38.367154 10521 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:38.360344 10667 raft_consensus.cc:697] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:38.405807  8693 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:38.405823  8059 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:38.359709 10623 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:38.349792 10319 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:13:38.261960 10705 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.408815  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:38.415640  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:38.382540 10704 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:38.419625 10713 consensus_queue.cc:1059] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:38.382941 10436 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:38.387816 10692 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:38.421639  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:38.422978  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.429883 10667 consensus_queue.cc:237] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.421133 10444 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:13:38.433920  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.435595  8059 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:38.436247  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.437187  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:38.437793  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.444684 10444 raft_consensus.cc:3055] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:38.421819 10688 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:38.421312 10400 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:38.399490 10697 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
I20260709 13:13:38.413415 10674 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:13:38.412760 10676 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:13:38.424047 10715 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:38.485169  8026 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:38.440543 10636 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:13:38.418814 10623 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.441195 10702 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:38.442009 10689 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.493513  8726 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:13:38.413100 10521 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.413149 10635 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:13:38.383587 10632 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.492642  8026 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:38.516571  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:38.442711 10217 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:38.492533 10400 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:38.525230 10632 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:38.446738 10471 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 13:13:38.525614 10400 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:38.497047  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 88915.7us
I20260709 13:13:38.528861  8062 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:38.498517  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:38.499042 10552 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:38.504772 10686 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.487476 10716 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:38.505847 10644 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:38.507493 10623 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:38.450059 10687 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:38.445096 10444 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:38.448741 10713 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:38.458971 10682 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 13:13:38.521564 10727 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:38.522028 10728 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:38.522629 10521 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:38.509063 10715 raft_consensus.cc:697] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:13:38.529862  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 92144.1us
I20260709 13:13:38.529496  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:38.434966 10436 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.493664 10702 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:38.563512  8129 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:13:38.529357 10525 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:13:38.524503 10692 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:38.535761 10552 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.566545  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.569187  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:38.606715  8203 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:38.607491  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:38.607748  8057 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:38.608023  8203 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:38.608718  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:38.609238  8203 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:38.609902  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:38.569321  8208 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 13:13:38.610463  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 111502us
I20260709 13:13:38.572381  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:38.611615  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.572989  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:38.612581  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:38.543308 10716 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:13:38.578775  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 141677us
I20260709 13:13:38.564785 10705 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:38.613444  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:38.562621 10713 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.535053 10541 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:13:38.600215  8129 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:38.612752  8207 raft_consensus.cc:2410] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:38.573571  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:38.613090  8063 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:38.618450  8208 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:13:38.608718  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.618829  8208 pending_rounds.cc:85] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 2
I20260709 13:13:38.564756 10444 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.567814 10635 raft_consensus.cc:3055] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:13:38.621022  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 54557.4us
I20260709 13:13:38.621716  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:38.619086  8208 pending_rounds.cc:107] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:13:38.619032  8057 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:38.596033 10704 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:38.623142  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.626664  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:38.622610  8208 raft_consensus.cc:2889] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:38.595393 10552 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.625981  8054 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:13:38.628166  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:38.597934 10702 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.628746  8207 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:38.627256  8129 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:38.653033  8060 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:13:38.652928  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.655953  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.656690  8203 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:38.654613  8175 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:38.657533  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.658471  8056 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:38.614161 10521 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.614382 10728 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.658490  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:38.617316 10713 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:38.666242  8063 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
W20260709 13:13:38.669750  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.670820  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:38.614661 10716 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:38.653695 10692 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:38.621294 10635 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:38.676941  8207 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:38.652347 10630 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:38.678181  8060 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:13:38.680049  8129 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:13:38.616722 10623 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:13:38.662042 10683 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:38.660159 10659 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:38.712411  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.713217  8135 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:38.714224  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:38.714874  8135 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:38.686478  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.691751 10436 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:38.662004 10217 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:38.682757 10426 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:38.721575 10426 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.721971 10426 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:38.722561  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.723646  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.725032 10217 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:38.725474 10217 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.725777 10734 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.697086 10369 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.726261 10734 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.727857 10369 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:38.728010 10444 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.729084 10734 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:38.734095  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:38.735263  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:38.735966  8210 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 6: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:38.736653  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:38.737025  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.738997  8209 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:38.738765 10635 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.700728  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:13:38.706915 10729 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:13:38.739856 10635 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.659192 10737 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:38.682614 10696 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.740849 10700 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:38.740715 10696 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.741250 10700 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:38.741624 10700 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:38.720522  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:38.742750  8133 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:38.674223 10704 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:38.652536 10727 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:13:38.676452 10710 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:38.720808 10677 raft_consensus.cc:3055] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:38.742206 10737 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:13:38.758549  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 57914.5us
I20260709 13:13:38.761343  8209 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:13:38.761660  8135 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:13:38.804983  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:38.756227  8175 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:38.806037  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 49891.1us
I20260709 13:13:38.683638 10692 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.797752  8101 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:38.683279 10630 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:38.756148  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:38.824618  8133 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.675282 10521 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:38.711676 10683 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:38.810052  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:38.856812  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 59150.1us
W20260709 13:13:38.845129  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:38.856611 10547 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:38.859581 10547 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:38.859943 10547 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.677286 10728 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.801322 10677 raft_consensus.cc:740] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:38.866021  8104 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:38.870435  8101 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:38.870677 10677 consensus_queue.cc:260] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.871655 10677 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:38.872084 10511 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:38.872592 10684 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:38.873235 10684 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:38.873394 10511 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:38.868507  8133 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:38.871470 10681 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:38.809046 10686 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:38.824542 10692 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:38.808327 10710 raft_consensus.cc:697] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:13:38.941169  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 65877.7us
I20260709 13:13:38.869619 10609 consensus_queue.cc:1059] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:38.802019 10436 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:13:38.942184  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:38.881446  8054 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:13:38.809648  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:38.963711  8062 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:38.966019  8061 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:13:38.966084  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:38.969034  8726 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 3.
I20260709 13:13:38.967226  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.971165  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:38.973539  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1192) took 1096 ms. Trace:
I20260709 13:13:38.942641 10081 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:38.679069 10715 consensus_queue.cc:237] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:38.974020  8058 rpcz_store.cc:276] 0709 13:13:37.866563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:37.866664 (+   101us) service_pool.cc:224] Handling call
0709 13:13:38.963287 (+1096623us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b540027a954. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:38.963425 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71332608}
I20260709 13:13:38.947161 10400 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.879281 10682 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.098s
I20260709 13:13:38.979701  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:39.048209  8727 raft_consensus.cc:2410] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:13:38.826269 10630 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:38.984174  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.809597 10636 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:39.055869  8059 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:39.061115  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:39.064484  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:38.863961 10683 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:39.066179  8059 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 6: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:38.980998 10081 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:38.866997 10697 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:39.147053 10697 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.148479 10697 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:39.083600  8727 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:38.980149 10696 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:39.048054 10705 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.060328 10636 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:39.084146 10081 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:39.136829  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:39.137058 10657 consensus_meta.cc:287] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 1.057s	user 0.008s	sys 0.006s
I20260709 13:13:39.167305  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:39.168094  8135 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 6: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:39.168969  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.146580  8721 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:13:38.954813 10701 consensus_queue.cc:1059] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:13:38.980325 10676 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:38.979262  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:39.164158 10636 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:39.065227  8058 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:39.216666  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:39.165125  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:39.060860 10400 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.167392  8102 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:38.958843 10704 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.179088  8693 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:39.240458  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 61429.5us
I20260709 13:13:39.241365  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:38.874372 10684 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:39.164995 10689 consensus_meta.cc:287] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.656s	user 0.010s	sys 0.001s
W20260709 13:13:39.169358 10702 consensus_meta.cc:287] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.000s
I20260709 13:13:38.983166 10737 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.210836 10426 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.163228  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:39.166167  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:39.257575  8059 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:39.173245  8721 pending_rounds.cc:85] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Aborting all ops after (but not including) 6
I20260709 13:13:39.229210 10081 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.235241  8063 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:39.290432  8063 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:13:39.291182  8063 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:39.291982  8063 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:13:39.292166  8063 pending_rounds.cc:85] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 3
I20260709 13:13:39.292290  8063 pending_rounds.cc:107] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:13:39.292461  8063 raft_consensus.cc:2889] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:39.242405  8060 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:13:39.156657  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:39.294386  8723 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:39.163594 10705 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.234489 10636 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:39.260159  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 92858us
I20260709 13:13:39.231040 10728 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:39.250802  8693 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:39.288439  8721 pending_rounds.cc:107] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:13:39.335922  8721 raft_consensus.cc:2889] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:39.336978 10740 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:39.339658 10705 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:39.344183 10081 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:39.327126 10636 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:39.322263 10444 consensus_meta.cc:287] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.594s	user 0.006s	sys 0.002s
W20260709 13:13:39.349323 10750 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.103s	user 0.001s	sys 0.001s
I20260709 13:13:39.356822 10610 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.357861 10610 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:39.344183 10635 consensus_meta.cc:287] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.604s	user 0.005s	sys 0.003s
I20260709 13:13:39.360754 10683 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.363189 10683 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:39.367115  8059 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
W20260709 13:13:39.376045  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.369295  8205 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:13:39.384505 10738 log.cc:927] Time spent T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.765s	user 0.005s	sys 0.000s
W20260709 13:13:39.388178 10751 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 13:13:39.388342 10743 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.235s	user 0.000s	sys 0.000s
W20260709 13:13:39.388795  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.410652  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:39.304497  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 53777.2us
W20260709 13:13:39.305931 10741 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.220s	user 0.000s	sys 0.000s
W20260709 13:13:39.414177  8695 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.053s	sys 0.108s
W20260709 13:13:39.414911  8695 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.053s	sys 0.108s
I20260709 13:13:39.309478 10400 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:39.302897 10719 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:39.325575 10713 consensus_meta.cc:287] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.642s	user 0.002s	sys 0.002s
I20260709 13:13:39.175911 10657 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.327126 10426 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:39.327474 10443 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:39.218143 10676 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:39.331848 10217 consensus_meta.cc:287] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.000s
I20260709 13:13:39.257632 10521 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:39.288108 10681 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:39.368865 10081 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:39.369248  8210 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:39.518566  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:39.519202  8210 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:39.412463  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:39.520006  8133 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:39.256186 10689 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:39.249431 10717 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:39.520977  8211 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:39.521248 10630 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:39.519586 10687 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.004s	sys 0.014s
I20260709 13:13:39.354048 10540 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:39.522509  8206 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:39.446108 10400 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.419569 10565 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.436323 10644 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.487164 10657 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:39.416457 10552 consensus_meta.cc:287] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.746s	user 0.004s	sys 0.004s
W20260709 13:13:39.487769 10744 log.cc:927] Time spent T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:13:39.420372 10705 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:13:39.488477  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:39.523471 10687 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.922s	user 0.004s	sys 0.014s
I20260709 13:13:39.534332  8131 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:39.511097  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.153412 10710 consensus_queue.cc:237] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:39.414219  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 273115us
I20260709 13:13:39.536844  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:39.545526  8132 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
W20260709 13:13:39.545936  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 78456.4us
I20260709 13:13:39.525295  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:39.521468 10717 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:39.511394 10683 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:39.517141 10700 consensus_meta.cc:287] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.775s	user 0.007s	sys 0.003s
W20260709 13:13:39.532966  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.504673  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:39.255939 10702 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.533648 10552 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.532294 10644 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.548026 10717 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:39.570772 10644 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:39.531248 10565 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.547195  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:39.572558 10565 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:39.548708  8207 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:39.489125 10713 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.551954  8210 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:39.515198 10217 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.580407 10217 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:39.581784 10713 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:39.445842 10719 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.578547  8210 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:13:39.544844 10519 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:39.566906 10760 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.566998  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:39.568176 10552 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:39.568783  8055 raft_consensus.cc:2393] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 6: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:39.535631 10761 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.585170  8138 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:39.623363  8203 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:39.531424 10400 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.584494 10426 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.655128  8207 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:13:39.577278 10689 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:39.637056  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2288) took 1024 ms. Trace:
I20260709 13:13:39.627319  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:39.661092  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2286) took 1048 ms. Trace:
I20260709 13:13:39.675242  8054 rpcz_store.cc:276] 0709 13:13:38.612470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.628301 (+ 15831us) service_pool.cc:224] Handling call
0709 13:13:38.662294 (+ 33993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.955829 (+293535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.642632 (+686803us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b5400052154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:39.661016 (+ 18384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503293312}
W20260709 13:13:39.717738  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 305358us
I20260709 13:13:39.622931 10636 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.557400 10683 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.627060 10697 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.564999 10700 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.793085 10759 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:39.674642 10764 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:13:39.811671  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:39.793599 10759 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.819533  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2374) took 1249 ms. Trace:
I20260709 13:13:39.819988  8211 rpcz_store.cc:276] 0709 13:13:38.570400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.572119 (+  1719us) service_pool.cc:224] Handling call
0709 13:13:38.597826 (+ 25707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.521349 (+923523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.819308 (+297959us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b540010c054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:39.819481 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2122641664}
I20260709 13:13:39.673723  8056 rpcz_store.cc:276] 0709 13:13:38.612910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.666797 (+ 53887us) service_pool.cc:224] Handling call
0709 13:13:38.683588 (+ 16791us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:38.698528 (+ 14940us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:38.698563 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:38.699010 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 13:13:38.878904 (+179894us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:38.878948 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:38.879020 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:39.636799 (+757779us) raft_consensus.cc:1685] finished
0709 13:13:39.636826 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.636984 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19243,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":106,"thread_start_us":179539,"threads_started":1,"wal-append.queue_time_us":538857}
I20260709 13:13:39.621968 10717 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.829020  8132 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:39.623943 10760 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.831030  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 361) took 1559 ms. Trace:
I20260709 13:13:39.832006  8693 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:39.813078 10519 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:39.812829 10719 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:39.848083  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:39.782536 10689 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.833827  8212 rpcz_store.cc:276] 0709 13:13:38.270732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.270844 (+   112us) service_pool.cc:224] Handling call
0709 13:13:38.271116 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.285134 (+ 14018us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:38.285162 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:38.285199 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:38.285226 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.829896 (+1544670us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b54002413d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:39.829987 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:39.830114 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31869824}
W20260709 13:13:39.825968 10547 consensus_meta.cc:287] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.966s	user 0.005s	sys 0.005s
I20260709 13:13:39.836182  8693 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:39.763355 10443 raft_consensus.cc:697] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:39.674841 10426 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.625293 10761 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.824359 10769 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:39.796027 10636 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.835189 10717 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:39.834887 10713 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:39.864030 10716 consensus_meta.cc:287] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 1.178s	user 0.009s	sys 0.001s
I20260709 13:13:39.866739 10676 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:39.873643 10769 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:39.875053 10630 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:39.877574 10687 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:13:39.905263  8175 leader_election.cc:336] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 2.058s (SENT)
I20260709 13:13:39.905701  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:39.865092 10547 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.906848  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:39.857746 10519 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.911777  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:39.906785 10610 consensus_meta.cc:287] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.000s
I20260709 13:13:39.916795  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:39.898402 10769 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:39.888012 10716 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.908614 10702 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:39.953606  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:39.926162  8129 raft_consensus.cc:1240] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:39.913744 10630 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:39.920920 10763 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260709 13:13:39.976629  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:39.976972  8730 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:13:39.982847 10760 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:39.987336 10761 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:39.924757  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:39.991714  8726 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:39.991564  8138 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
W20260709 13:13:39.978775 10714 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 1.718s	user 0.003s	sys 0.014s
I20260709 13:13:39.994853  8205 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:39.978104 10400 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:40.001284  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 128530us
I20260709 13:13:39.977351 10676 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.001909  8176 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:13:40.005908  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.007728  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:39.993173 10636 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:40.011883 10784 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.012382 10784 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.997463  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:40.010356  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:40.015188 10659 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:40.015599  8205 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:40.015682 10659 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:39.999130  8724 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:40.016595 10659 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:39.996517 10700 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:40.019235 10716 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:40.020378 10547 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:40.020937 10547 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:40.021296  8102 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:40.021384 10547 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.010236  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:40.027794 10692 consensus_meta.cc:287] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.957s	user 0.010s	sys 0.000s
I20260709 13:13:40.028198 10692 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.029307  8208 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:13:40.030282 10692 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:40.030376  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2388) took 1364 ms. Trace:
I20260709 13:13:40.030864  8208 rpcz_store.cc:276] 0709 13:13:38.665667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.670754 (+  5087us) service_pool.cc:224] Handling call
0709 13:13:38.671017 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:40.029661 (+1358644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.030150 (+   489us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400110654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:40.030296 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3123270912}
W20260709 13:13:40.032706 10777 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
W20260709 13:13:40.034001 10772 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
I20260709 13:13:39.993707 10630 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.035074  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:40.036023 10630 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:40.038930 10081 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.039422 10081 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.041322 10081 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:40.041788 10081 consensus_queue.cc:1059] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:13:40.042608  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.009569 10759 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:40.043485 10701 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:40.043871 10701 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.044194 10701 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:40.044420 10683 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:40.046154  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:40.046766  8724 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:40.014945 10519 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:40.048187  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:40.048727  8727 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:40.048648  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:40.049346  8724 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:40.050684  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:40.051036  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.052228 10547 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:40.052517  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:40.052894  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:40.053418 10547 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:40.053690  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:40.054426  8130 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:13:40.053913 10547 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.055737  8133 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:40.054457  8138 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:40.059868  8102 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:40.063963 10682 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:40.064110  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:40.064507 10682 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:40.064770  8130 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:40.063915  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:40.065342 10682 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:39.995379 10702 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.067723  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:40.066926  8133 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:13:40.072953  8138 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:40.077585  8134 raft_consensus.cc:2410] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:13:40.076372  8102 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:40.060159  8726 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
W20260709 13:13:40.091379  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 186135us
W20260709 13:13:40.067703 10774 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:13:40.096722  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:40.096921  8130 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:40.053716 10511 raft_consensus.cc:3055] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:40.100817 10783 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:40.084740  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:40.066262 10682 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:40.014304 10784 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:40.099181  8176 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:40.066483  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:40.052517 10674 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.114794 10674 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.109035  8102 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:40.116823  8102 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:40.116962 10674 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:40.117508 10674 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:40.117830 10565 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:40.090819 10700 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:40.111312 10717 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:13:40.118824 10740 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.119268 10740 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:40.119591 10717 raft_consensus.cc:3055] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:40.119969 10717 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:40.120733 10717 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.121794 10717 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:39.993607 10443 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.117865 10674 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.096469 10610 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:40.113399 10369 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:40.111392 10081 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.124385 10081 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.008478 10676 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.112625  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:40.113219 10719 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:40.125859  8727 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:40.118178 10547 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:40.134469  8211 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:13:40.143345  8207 raft_consensus.cc:2393] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:40.106495 10511 raft_consensus.cc:740] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:40.136721  8104 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:40.147571  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 368) took 1387 ms. Trace:
I20260709 13:13:40.130733  8176 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:40.147919  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:40.148106  8206 rpcz_store.cc:276] 0709 13:13:38.759735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.134895 (+375160us) service_pool.cc:224] Handling call
0709 13:13:39.137152 (+  2257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.522875 (+385723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.147369 (+624494us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b540006f9d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:40.147520 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":885874560}
I20260709 13:13:40.149166  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:40.149693  8206 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:40.150318  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2395) took 1451 ms. Trace:
W20260709 13:13:40.150803  8062 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 269) took 1420 ms (client timeout 1535 ms). Trace:
I20260709 13:13:40.153774  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2285) took 1541 ms. Trace:
I20260709 13:13:40.156136  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:40.156759  8206 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:40.150684  8209 rpcz_store.cc:276] 0709 13:13:38.699031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.857795 (+158764us) service_pool.cc:224] Handling call
0709 13:13:38.879937 (+ 22142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.880195 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:40.049386 (+1169191us) spinlock_profiling.cc:243] Waited 66.5 ms on lock 0x7b18003adce0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf
0709 13:13:40.049503 (+   117us) log.cc:844] Serialized 14 byte log entry
0709 13:13:40.150036 (+100533us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:40.150080 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:40.150122 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.150183 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.150273 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5416,"prepare.queue_time_us":175093,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"spinlock_wait_cycles":152982912,"thread_start_us":295385,"threads_started":2}
I20260709 13:13:40.091185 10702 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.164165  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:40.151276  8062 rpcz_store.cc:269] 0709 13:13:38.730536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.736808 (+  6272us) service_pool.cc:224] Handling call
0709 13:13:40.150656 (+1413848us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b5400430154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:40.150753 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":519980288}
I20260709 13:13:40.091003 10609 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:13:40.167663  8104 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:40.154119  8057 rpcz_store.cc:276] 0709 13:13:38.612252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.623940 (+ 11688us) service_pool.cc:224] Handling call
0709 13:13:38.624162 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:38.624346 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:38.659517 (+ 35171us) log.cc:844] Serialized 14 byte log entry
0709 13:13:38.804060 (+144543us) log.cc:844] Serialized 14 byte log entry
0709 13:13:39.351249 (+547189us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b180035b960. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:13:40.135931 (+784682us) spinlock_profiling.cc:243] Waited 31.5 ms on lock 0x7b180035b960. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f29522 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:13:40.153342 (+ 17411us) spinlock_profiling.cc:243] Waited 55.7 ms on lock 0x7b180035b960. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ea3c0a 00007f6dc15e3219 00007f6db7f2ba40 00007f6db7f264e4 00007f6db7f26455 00007f6dbb00e060 00007f6dbb020716 00007f6dbb031287 00007f6dbb031208
0709 13:13:40.153472 (+   130us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:40.153515 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:40.153558 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.153628 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.153724 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3745,"prepare.queue_time_us":35133,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":97,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":14774,"spinlock_wait_cycles":204295296,"thread_start_us":143859,"threads_started":1,"wal-append.queue_time_us":320655}
I20260709 13:13:40.110303 10689 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:40.127431 10674 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:40.127704  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:40.169579  8206 raft_consensus.cc:2393] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:40.117506 10779 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:40.181512  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:40.182276  8206 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:40.136494 10081 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:40.138042 10719 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:40.186139  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:40.186964  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2373) took 1618 ms. Trace:
I20260709 13:13:40.136270 10759 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.097755 10769 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.188120 10759 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.187323  8204 rpcz_store.cc:276] 0709 13:13:38.568845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.569593 (+   748us) service_pool.cc:224] Handling call
0709 13:13:38.597104 (+ 27511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:38.601655 (+  4551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:38.601684 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:38.613660 (+ 11976us) log.cc:844] Serialized 26 byte log entry
0709 13:13:39.543325 (+929665us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b18003a4260. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:13:39.739113 (+195788us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:39.739179 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:39.739272 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:40.067058 (+327786us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:40.186748 (+119690us) raft_consensus.cc:1685] finished
0709 13:13:40.186778 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.186906 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4124,"raft.queue_time_us":16471,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":37218,"spinlock_wait_cycles":15120384,"thread_start_us":195570,"threads_started":1,"wal-append.queue_time_us":1264247}
I20260709 13:13:40.189424 10769 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:40.164531 10784 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:40.161298 10630 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:40.189954  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:40.192643  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:40.169970 10609 consensus_queue.cc:1243] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:40.159933 10710 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.136932 10610 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:13:40.160318 10760 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:40.202765 10784 raft_consensus.cc:697] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:40.203001  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2281) took 1593 ms. Trace:
I20260709 13:13:40.203049 10702 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:40.183527 10701 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.180225 10705 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:13:40.179071 10713 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:40.205165  8058 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:13:40.187240  8206 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:40.170153  9980 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:40.136077 10683 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:13:40.192559  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:40.196525 10667 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.196408 10511 consensus_queue.cc:260] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.175626 10697 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:40.180104 10565 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:40.267400 10565 raft_consensus.cc:697] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:40.203471  8055 rpcz_store.cc:276] 0709 13:13:38.609229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:38.609352 (+   123us) service_pool.cc:224] Handling call
0709 13:13:40.202820 (+1593468us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b54001e82d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:40.202951 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2099866368}
I20260709 13:13:40.192145 10719 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:40.193217 10689 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:40.265450  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:40.317587  8209 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:40.184240 10757 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:13:40.197899 10630 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.324112  8723 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
W20260709 13:13:40.245291  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.111742 10783 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.261205 10511 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:40.290934  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.238992 10769 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:40.242899 10667 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.313519 10740 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.203743 10784 consensus_queue.cc:237] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.205577 10710 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:40.326014  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.328649  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:40.347046  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:40.326538  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 154921us
I20260709 13:13:40.338793  8101 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:40.390108  8209 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
W20260709 13:13:40.402069  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 53056.4us
I20260709 13:13:40.407883  8059 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:13:40.411295  7945 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a003b4584d3747a381522d67299aaed2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 84d775d1767e4cc4aa7b9c627d8f0fb8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:40.409169  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:40.440042  8727 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:40.440320  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:40.402060  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:40.426267  8054 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:13:40.322180 10689 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.425405  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:40.441097  8055 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:40.449936  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 65281us
I20260709 13:13:40.456514  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:40.457651  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:40.401679 10728 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.422333 10521 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.440412 10471 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.440272 10435 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.401851 10623 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.401553 10610 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:40.422075 10540 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.465272  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:40.410485  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:40.460281  8026 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:40.346566 10769 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.451875 10683 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.442835 10369 consensus_queue.cc:1059] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:13:40.480875 10435 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.502260  8026 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:40.502857  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:40.477849  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.473482  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:40.478725  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:40.478051  8055 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
W20260709 13:13:40.506186  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.506806  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:40.507414  8055 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:40.508360  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:40.508925  8055 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:40.441995 10730 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:40.509915  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:40.511561  8130 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 7: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:40.511620  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:40.517772  8055 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:40.519819  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:40.462399 10700 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:13:40.484205 10626 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.521552  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:40.535022  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:40.531963  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:40.487188 10540 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.521917 10683 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.537099 10552 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.537563 10552 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.538122 10540 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:40.538576  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 59947.3us
I20260709 13:13:40.538712 10540 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.538913 10797 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:40.539111 10540 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.539532 10552 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:40.539590 10797 raft_consensus.cc:2764] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:13:40.539750  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:40.541136 10683 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:40.541590  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:40.478863 10521 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.542286 10716 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:40.542971 10369 raft_consensus.cc:3055] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:40.536185 10799 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:40.549058  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:40.554422  8721 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:40.475692 10728 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.554896 10516 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:40.555789  8056 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:13:40.556058 10516 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:40.556409 10630 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:40.556619 10728 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:40.557294  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:40.556910 10516 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.557879  8056 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:40.557494 10667 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:40.558692  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:40.487730 10701 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:40.554932  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:40.547514  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:40.481117 10623 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.474813 10682 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:13:40.583694  8056 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 6: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:40.553104 10369 raft_consensus.cc:740] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:40.484629 10713 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.136s
I20260709 13:13:40.585989  8055 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:40.548125 10787 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:13:40.535221 10730 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:40.847517 10674 consensus_meta.cc:287] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.662s	user 0.008s	sys 0.000s
I20260709 13:13:40.847998  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:40.847959 10674 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.848723  8212 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:40.849567  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:40.850579  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:40.629249  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 227714us
I20260709 13:13:40.481161 10471 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.695708  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:40.679718  8059 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:40.646700  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:40.785964  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:40.874625  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:40.696081  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:40.887147  8134 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:40.857761  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1214) took 2182 ms. Trace:
I20260709 13:13:40.888480  8058 rpcz_store.cc:276] 0709 13:13:38.675668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.227325 (+551657us) service_pool.cc:224] Handling call
0709 13:13:40.204882 (+977557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:40.205607 (+   725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.857535 (+651928us) spinlock_profiling.cc:243] Waited 977 ms on lock 0x7b54001e82d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:40.857693 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2247483648}
I20260709 13:13:40.889111  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:40.889747  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:40.890491  8058 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:40.548769  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:40.891527  8133 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
W20260709 13:13:40.600878  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.788151  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:40.683120  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:40.628754  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:40.487130 10565 consensus_queue.cc:237] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.887646  8136 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:40.915581  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:40.536537 10435 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:40.843979 10793 log.cc:927] Time spent T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.259s	user 0.000s	sys 0.000s
I20260709 13:13:40.921805  8104 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:40.941776  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.922902  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:40.943701  8061 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
I20260709 13:13:40.944687  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:40.547415 10636 raft_consensus.cc:3055] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:40.946635 10804 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:40.946882 10636 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:40.947203 10804 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.947901 10636 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.949290 10636 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:40.954457 10081 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:40.954795  8212 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:40.954978 10081 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:40.955508 10081 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:40.956054  8212 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:40.957203  8212 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:40.949313 10804 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:40.620237 10630 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.959501 10630 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.961700  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2402) took 1562 ms. Trace:
I20260709 13:13:40.962172  8210 rpcz_store.cc:276] 0709 13:13:39.398949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.584959 (+186010us) service_pool.cc:224] Handling call
0709 13:13:39.585237 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:39.977718 (+392481us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:40.102321 (+124603us) log.cc:844] Serialized 14 byte log entry
0709 13:13:40.321779 (+219458us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:40.322184 (+   405us) log.cc:844] Serialized 26 byte log entry
0709 13:13:40.322297 (+   113us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:40.322325 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.961272 (+638947us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54002bf054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:40.961328 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:40.961380 (+    52us) raft_consensus.cc:1685] finished
0709 13:13:40.961488 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:40.961639 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4708,"prepare.queue_time_us":48650,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":32,"raft.queue_time_us":10056,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":2430,"spinlock_wait_cycles":552097664,"thread_start_us":406157,"threads_started":3,"wal-append.queue_time_us":376498}
I20260709 13:13:40.965095  8210 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:40.967360 10689 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:40.971979 10471 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:40.547211 10769 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:40.972949  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 369) took 1571 ms. Trace:
I20260709 13:13:40.973318  8211 rpcz_store.cc:276] 0709 13:13:39.401256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.821092 (+419836us) service_pool.cc:224] Handling call
0709 13:13:39.831215 (+ 10123us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:40.143778 (+312563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.972711 (+828933us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54001540d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:40.972896 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":696827648}
I20260709 13:13:40.974766  8211 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:40.914170 10540 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:40.976260  8211 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:13:40.976693  8211 pending_rounds.cc:85] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 7
I20260709 13:13:40.976959  8211 pending_rounds.cc:107] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:13:40.977236  8211 raft_consensus.cc:2889] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:40.978641 10369 consensus_queue.cc:260] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.913625 10697 consensus_queue.cc:1059] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:13:40.983363 10697 raft_consensus.cc:3055] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:40.983769 10697 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:40.984133  8723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 458) took 1077 ms. Trace:
I20260709 13:13:40.985361  8730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 455) took 1084 ms. Trace:
I20260709 13:13:40.985714  8730 rpcz_store.cc:276] 0709 13:13:39.900660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.900785 (+   125us) service_pool.cc:224] Handling call
0709 13:13:39.901047 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:39.991824 (+ 90777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.985157 (+993333us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b5400580f54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:40.985309 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66646784}
I20260709 13:13:40.987632 10644 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:40.988121 10644 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:40.988893 10644 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:40.989995 10644 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:40.984601  8723 rpcz_store.cc:276] 0709 13:13:39.906234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.906629 (+   395us) service_pool.cc:224] Handling call
0709 13:13:40.983931 (+1077302us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b5400730854. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:40.984061 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":808796032}
W20260709 13:13:40.995854 10511 consensus_meta.cc:287] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.661s	user 0.004s	sys 0.004s
I20260709 13:13:40.583773 10443 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:40.958130  8212 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:40.543078 10740 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:40.545883 10710 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:40.933175  8055 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:40.984724 10697 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:40.999336  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:40.928948 10667 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.014853  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:40.923750  8063 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.006s	sys 0.014s
I20260709 13:13:40.920238  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:40.850219 10674 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:41.007731  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:40.544406 10783 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:41.008468  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:40.961660 10630 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:41.010535  8057 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.004s	sys 0.020s
I20260709 13:13:40.924131  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:40.584563 10626 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.025528  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:41.026376  8726 raft_consensus.cc:2410] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:13:41.014463  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:41.027243  8055 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
W20260709 13:13:41.009238  8060 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.004s	sys 0.010s
W20260709 13:13:41.016031  8063 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.007s	sys 0.014s
I20260709 13:13:41.022007  8056 raft_consensus.cc:2410] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:13:41.028084  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:41.018827  8133 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:41.014140  8203 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:41.000164  8210 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:13:41.021311  8104 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:41.031638  8060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.005s	sys 0.010s
I20260709 13:13:41.033339  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:41.035179  8056 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
W20260709 13:13:41.037472  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:40.942999 10705 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.085s
W20260709 13:13:41.024542  8057 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.005s	sys 0.020s
I20260709 13:13:41.017079 10799 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.045469 10547 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:41.048498 10799 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.050107 10705 raft_consensus.cc:3055] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:41.050501 10705 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:41.050580 10799 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:41.019279 10697 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:41.051334 10705 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.052454 10705 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:41.054677 10547 raft_consensus.cc:697] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:41.055631 10547 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:41.056090 10426 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:41.044658  8104 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:41.016970 10435 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.046468  8029 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:41.060125  8026 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.020s	sys 0.033s
W20260709 13:13:41.063975  8026 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.020s	sys 0.033s
I20260709 13:13:41.060942  8056 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:13:41.064651  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 562444us
I20260709 13:13:41.029343 10702 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.032156 10731 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:41.032780 10540 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.033501 10659 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.013399 10443 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:41.023480 10667 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.025540 10674 consensus_queue.cc:1059] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:13:41.067309  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:41.044008 10610 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:41.090036 10471 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:41.104971 10730 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:41.090040 10770 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:41.104926 10609 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.006s	sys 0.028s
I20260709 13:13:41.064404 10521 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:41.065990  9980 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:13:41.121796  8101 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.013s	sys 0.025s
W20260709 13:13:41.126751 10784 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.003s	sys 0.016s
W20260709 13:13:41.175526  8101 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.014s	sys 0.025s
I20260709 13:13:41.065999 10681 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 13:13:41.068405 10435 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:41.185235  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.197299  8210 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:13:41.220973  8693 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:41.183434  8058 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
W20260709 13:13:41.219312  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 793602us
I20260709 13:13:41.190146  8055 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:41.127736 10799 consensus_queue.cc:1059] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:13:41.089780 10701 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 13:13:41.126770 10702 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.219264  8696 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:41.181054 10609 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.007s	sys 0.029s
I20260709 13:13:41.233920  8210 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:41.239003  8058 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:41.143896 10810 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 13:13:41.151216 10610 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.245327  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:41.176234 10784 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.003s	sys 0.016s
I20260709 13:13:41.243221  8058 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:41.128489 10731 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.237160 10701 consensus_queue.cc:1243] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:13:41.150645 10659 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.176003  9980 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:41.229323 10687 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:41.144269 10540 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:41.299377  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.334436  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 77243.8us
I20260709 13:13:41.342518 10687 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.343015 10687 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:41.246721 10610 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:41.347484  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:41.356315  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:41.415139  8723 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:41.415261  8730 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:41.358453 10519 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:13:41.360416 10809 log.cc:927] Time spent T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
W20260709 13:13:41.361848 10812 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.303s	user 0.001s	sys 0.000s
I20260709 13:13:41.386067 10702 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:41.386497 10626 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:41.346870  9980 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:41.280601  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:41.395416 10667 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:41.396966 10808 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:41.400490 10659 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:41.402896 10623 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:41.404623 10443 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:41.258315 10713 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.174s
I20260709 13:13:41.404898 10687 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:41.407148 10719 consensus_meta.cc:287] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 1.086s	user 0.010s	sys 0.000s
I20260709 13:13:41.407650  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:41.255774 10426 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:41.431972 10689 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:41.444952  8694 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.020s	sys 0.021s
I20260709 13:13:41.425985 10519 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:41.384356 10435 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:41.345402 10676 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:41.433374  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 152891us
I20260709 13:13:41.339673 10716 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.175s
I20260709 13:13:41.338635 10717 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.097s
W20260709 13:13:41.462918  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.418283  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:41.445441  8694 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.020s	sys 0.021s
I20260709 13:13:41.238828 10740 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:41.438272 10760 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:41.466439  8176 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:41.449355 10519 consensus_queue.cc:1243] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:13:41.476683 10565 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.015s	sys 0.024s
W20260709 13:13:41.462055 10636 consensus_meta.cc:287] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.001s
I20260709 13:13:41.496480  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:41.512771  8058 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:41.499708  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2301) took 1096 ms. Trace:
I20260709 13:13:41.513672  8062 rpcz_store.cc:276] 0709 13:13:40.403444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.422438 (+ 18994us) service_pool.cc:224] Handling call
0709 13:13:40.422779 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:40.423159 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:40.423196 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:40.423695 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:13:40.915107 (+491412us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:40.915157 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:40.915230 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:41.499458 (+584228us) raft_consensus.cc:1685] finished
0709 13:13:41.499485 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:41.499641 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18634,"raft.queue_time_us":3403,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":13513,"spinlock_wait_cycles":1161600,"thread_start_us":435991,"threads_started":1,"wal-append.queue_time_us":593006}
I20260709 13:13:41.444733 10081 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.445281 10719 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:41.449666 10426 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.492327 10784 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:41.459862 10689 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:41.476279 10702 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
W20260709 13:13:41.516373 10565 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.015s	sys 0.024s
I20260709 13:13:41.481434 10760 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.513221 10443 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.508939 10740 consensus_queue.cc:1059] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:13:41.520628 10819 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:41.567028  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 285) took 1095 ms. Trace:
I20260709 13:13:41.538568 10784 raft_consensus.cc:697] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 6 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:41.578824  8059 rpcz_store.cc:276] 0709 13:13:40.472058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.473206 (+  1148us) service_pool.cc:224] Handling call
0709 13:13:41.566817 (+1093611us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400219b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:41.566959 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399872768}
W20260709 13:13:41.571241  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 335766us
W20260709 13:13:41.585268  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:41.547323 10540 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:41.575544  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 349678us
I20260709 13:13:41.565471 10731 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:41.566660 10630 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.591122 10819 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:41.604249  8101 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:41.568037 10443 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:41.609911 10813 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:41.613236 10081 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:41.613363 10697 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:41.614331 10426 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:41.600180 10700 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:41.619892  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.620316  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.620703  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:41.620754  8212 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.007s	sys 0.022s
W20260709 13:13:41.621162  8212 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.007s	sys 0.022s
I20260709 13:13:41.625015  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:41.625835  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:41.626423  8136 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:41.606215 10687 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:41.622563  8133 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:13:41.622787  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 168783us
I20260709 13:13:41.633665  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:41.597805 10565 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:13:41.628129  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:41.635483  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:41.639614  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:41.606014 10630 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.630054  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:41.626204  8101 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:41.641724  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:41.644217  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:41.645211 10702 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:41.645571 10702 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.645905 10702 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:41.645887  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:41.648386  8101 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:41.651551  8101 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:41.649753  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:41.641739 10820 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:41.621857 10700 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.657321  8101 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:41.657689  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:41.652940 10817 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:13:41.658834  8211 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:41.649003 10426 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:41.652577 10682 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:41.664173  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:41.663003  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:41.665827  8211 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:41.656754 10818 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:13:41.654354 10717 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:41.654198 10636 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:41.653616 10835 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:41.671248  8101 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:41.678524  8211 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:41.680083  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:41.682835  8211 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:13:41.661936 10719 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:41.663568 10426 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:41.663364 10834 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:41.663738 10836 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:41.685706  8104 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:41.655190 10547 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:13:41.658521 10519 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:13:41.660058 10740 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:13:41.670025 10717 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:41.671804 10636 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:41.670784 10840 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:41.694959  8211 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:41.648102 10784 consensus_queue.cc:237] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:41.684963 10609 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:41.683440 10760 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:41.693822 10426 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:41.696071 10825 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:13:41.699599 10827 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:13:41.706275  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 50837.9us
I20260709 13:13:41.654930 10819 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:41.696485 10644 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:41.718729 10644 raft_consensus.cc:697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:41.707177 10836 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:41.719800 10836 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:41.719606 10644 consensus_queue.cc:237] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.720639 10836 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.693591 10834 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:41.722652 10834 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:41.722890 10823 log.cc:927] Time spent T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
W20260709 13:13:41.724566 10828 log.cc:927] Time spent T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:13:41.725739  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2299) took 2778 ms. Trace:
W20260709 13:13:41.710980  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 50928.6us
I20260709 13:13:41.678133 10839 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:13:41.674235 10820 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:13:41.715387  8693 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:41.702792 10719 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.703459 10717 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:41.705281 10840 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.704768 10636 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:41.733553  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:41.726086  8063 rpcz_store.cc:276] 0709 13:13:38.946891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.293613 (+346722us) service_pool.cc:224] Handling call
0709 13:13:39.293790 (+   177us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:41.035966 (+1742176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:41.036009 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:41.036617 (+   608us) log.cc:844] Serialized 52 byte log entry
0709 13:13:41.505883 (+469266us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:41.505939 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.506023 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:41.725522 (+219499us) raft_consensus.cc:1685] finished
0709 13:13:41.725543 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:41.725664 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":903333,"raft.queue_time_us":898400,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":5577,"thread_start_us":1291326,"threads_started":2,"wal-append.queue_time_us":559027}
I20260709 13:13:41.744382  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2414) took 1599 ms. Trace:
I20260709 13:13:41.750999  8204 pending_rounds.cc:85] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 1
I20260709 13:13:41.710584 10609 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.761693  8204 pending_rounds.cc:107] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 13:13:41.765362  8204 raft_consensus.cc:2889] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:41.759665  8209 rpcz_store.cc:276] 0709 13:13:40.145246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.917900 (+772654us) service_pool.cc:224] Handling call
0709 13:13:40.998110 (+ 80210us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:40.998780 (+   670us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:41.037047 (+ 38267us) log.cc:844] Serialized 14 byte log entry
0709 13:13:41.598238 (+561191us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:41.605218 (+  6980us) log.cc:844] Serialized 26 byte log entry
0709 13:13:41.605373 (+   155us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:41.605418 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.605500 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:41.744170 (+138670us) raft_consensus.cc:1685] finished
0709 13:13:41.744187 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:41.744314 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3341,"prepare.queue_time_us":33673,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":12380,"raft.queue_time_us":7606,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":10829,"thread_start_us":486340,"threads_started":1,"wal-append.queue_time_us":588101}
W20260709 13:13:41.730991 10826 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:13:41.726315 10700 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:41.742511 10820 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:13:41.735311 10830 log.cc:927] Time spent T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 13:13:41.739076 10824 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 13:13:41.747051 10705 consensus_meta.cc:287] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.694s	user 0.008s	sys 0.003s
I20260709 13:13:41.747284 10443 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:41.742458 10689 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:41.751824 10630 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:41.737732  9980 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.835932  9980 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:41.733814 10719 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.842772 10719 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:41.746865 10717 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:41.844869 10717 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:41.767647 10609 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:41.767748 10845 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:41.734861 10840 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:41.748934 10636 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.833001  8132 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:13:41.800630 10813 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:41.829550 10700 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.833024 10674 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:41.825186 10820 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:41.865075 10820 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:41.864634 10674 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:13:41.866847 10674 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:41.730471 10834 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:41.869669 10832 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
I20260709 13:13:41.870105 10834 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:41.854513 10681 raft_consensus.cc:3055] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:41.829895  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:41.856515 10426 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:41.859783 10811 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.625s	user 0.004s	sys 0.000s
I20260709 13:13:41.860157 10636 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:41.864189  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1344) took 1806 ms. Trace:
I20260709 13:13:41.876770  8207 rpcz_store.cc:276] 0709 13:13:40.057588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.856336 (+798748us) service_pool.cc:224] Handling call
0709 13:13:40.929723 (+ 73387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:40.930090 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:13:40.930118 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:40.942661 (+ 12543us) log.cc:844] Serialized 52 byte log entry
0709 13:13:41.477133 (+534472us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:41.477188 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.477277 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:41.863976 (+386699us) raft_consensus.cc:1685] finished
0709 13:13:41.864001 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:41.864132 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7215,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":34428,"thread_start_us":457628,"threads_started":1,"wal-append.queue_time_us":580908}
I20260709 13:13:41.683597  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:41.870723 10681 raft_consensus.cc:740] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:41.879731 10681 consensus_queue.cc:260] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.880793 10681 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:41.824755 10610 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:41.883163 10829 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:13:41.872016 10700 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:41.884589  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 333) took 1377 ms. Trace:
I20260709 13:13:41.884903  8135 rpcz_store.cc:276] 0709 13:13:40.507322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.507582 (+   260us) service_pool.cc:224] Handling call
0709 13:13:40.507957 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:40.508358 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:40.508397 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:41.057891 (+549494us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b2c00011f88. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db990fa11 00007f6db991f372 00007f6dbb051a01 00007f6dbb01643a 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2
0709 13:13:41.058243 (+   352us) log.cc:844] Serialized 26 byte log entry
0709 13:13:41.538462 (+480219us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:41.538506 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.538602 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:41.882489 (+343887us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:41.884426 (+  1937us) raft_consensus.cc:1685] finished
0709 13:13:41.884444 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:41.884548 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36262,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":106,"spinlock_wait_cycles":35935488,"thread_start_us":479831,"threads_started":1,"wal-append.queue_time_us":585232}
I20260709 13:13:41.886201  8129 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:41.874902  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2305) took 1470 ms. Trace:
I20260709 13:13:41.887275  8138 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
W20260709 13:13:41.891562 10789 log.cc:927] Time spent T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 1.566s	user 0.011s	sys 0.025s
I20260709 13:13:41.896255  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2312) took 1484 ms. Trace:
I20260709 13:13:41.896595  8055 rpcz_store.cc:276] 0709 13:13:40.411591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.040832 (+629241us) service_pool.cc:224] Handling call
0709 13:13:41.041112 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.224258 (+183146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.896050 (+671792us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b540033efd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:41.896202 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341709696}
I20260709 13:13:41.887018  8061 rpcz_store.cc:276] 0709 13:13:40.404608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.408940 (+  4332us) service_pool.cc:224] Handling call
0709 13:13:41.874758 (+1465818us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b540001b154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:41.874851 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1199569280}
I20260709 13:13:41.851131 10845 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:41.901760  7961 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37774 (request call id 138) took 14935 ms (client timeout 15000 ms). Trace:
I20260709 13:13:41.911456  8726 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: [3.6-7.7]
I20260709 13:13:41.911602  8723 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
W20260709 13:13:41.915580  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 795985us
W20260709 13:13:41.916741  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 849512us
I20260709 13:13:41.917785  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:41.922468  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:41.922480  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:41.925364  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:41.925895  8135 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:41.934713  8134 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:41.939473  8723 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:41.940726  8134 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:13:41.944418  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:41.945253  8723 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:41.950356  8102 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:41.953029  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:41.953291  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:41.953925  8134 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:41.954062  8137 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:41.913666 10845 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:41.962632  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:41.963454  8134 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:41.964323  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:41.901455 10519 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:41.972301  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:41.902729  7961 rpcz_store.cc:269] 0709 13:13:26.966683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:26.983356 (+ 16673us) service_pool.cc:224] Handling call
0709 13:13:41.901695 (+14918339us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:36.390561 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:36.391204 (+   643us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:36.391258 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:13:36.391276 (+    18us) tablet.cc:662] Decoding operations
0709 13:13:36.392284 (+  1008us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:36.392360 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:36.392384 (+    24us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:13:36.393193 (+   809us) tablet.cc:701] Row locks acquired
0709 13:13:36.393242 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:13:36.393385 (+   143us) write_op.cc:270] Start()
0709 13:13:36.393531 (+   146us) write_op.cc:276] Timestamp: P: 1783602816393321 usec, L: 0
0709 13:13:36.393552 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:13:36.514130 (+120578us) log.cc:844] Serialized 3632 byte log entry
0709 13:13:36.903054 (+388924us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6db7f0c0ac 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b
0709 13:13:37.691490 (+788436us) op_driver.cc:464] REPLICATION: finished
0709 13:13:37.899387 (+207897us) write_op.cc:301] APPLY: starting
0709 13:13:37.904748 (+  5361us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:37.921945 (+ 17197us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:37.921975 (+    30us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:37.923054 (+  1079us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:37.923300 (+   246us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:13:37.923345 (+    45us) write_op.cc:312] APPLY: finished
0709 13:13:37.923595 (+   250us) log.cc:844] Serialized 71 byte log entry
0709 13:13:37.923766 (+   171us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:37.923908 (+   142us) write_op.cc:454] Released schema lock
0709 13:13:37.923953 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:37.924292 (+   339us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:37.924323 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":528239,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":188575,"apply.run_cpu_time_us":2727,"apply.run_wall_time_us":28325,"num_ops":9,"prepare.queue_time_us":199367,"prepare.run_cpu_time_us":38152,"prepare.run_wall_time_us":1063027,"raft.queue_time_us":221916,"raft.run_cpu_time_us":30341,"raft.run_wall_time_us":157890,"replication_time_us":1287279,"spinlock_wait_cycles":10763648,"thread_start_us":1324769,"threads_started":4,"wal-append.queue_time_us":844187,"wal-append.run_cpu_time_us":14267,"wal-append.run_wall_time_us":1678657}]]}
I20260709 13:13:41.983485 10471 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:41.985059 10443 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:41.986654  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:41.986981  8027 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:41.987809  8027 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:41.989039  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:41.990005  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:41.995932  8723 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:42.001673  8135 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:42.037480  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2300) took 1634 ms. Trace:
I20260709 13:13:42.039839  8057 rpcz_store.cc:276] 0709 13:13:40.403099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.423824 (+ 20725us) service_pool.cc:224] Handling call
0709 13:13:40.424173 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:40.424535 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:40.424572 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:40.450317 (+ 25745us) log.cc:844] Serialized 26 byte log entry
0709 13:13:41.046442 (+596125us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:41.046522 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.046635 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:41.425732 (+379097us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b48000801d8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:13:41.859096 (+433364us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:42.037193 (+178097us) raft_consensus.cc:1685] finished
0709 13:13:42.037223 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:42.037408 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":581780,"raft.queue_time_us":2089,"raft.run_cpu_time_us":22634,"raft.run_wall_time_us":983927,"spinlock_wait_cycles":392765440,"thread_start_us":567633,"threads_started":1,"wal-append.queue_time_us":687100}
I20260709 13:13:42.045795  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:41.972802 10853 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:42.053787  8134 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:42.056624  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.057174  8057 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:42.058295  8061 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:42.044296 10727 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.062096  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:42.060329  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 142670us
W20260709 13:13:42.060887  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 173133us
I20260709 13:13:42.062899  8058 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 7: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:42.062961  8132 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:42.063022  8029 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:42.061965 10730 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:42.048092 10674 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:42.064790  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.037216 10719 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:13:42.065868 10805 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:42.066038  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:42.066668  8059 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:42.066807 10521 consensus_queue.cc:1059] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:42.066406 10805 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:42.050536 10623 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:42.067543  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.068620  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.050393 10667 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:42.036609 10443 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:13:41.996052 10705 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
W20260709 13:13:42.079788 10848 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:13:42.007218 10519 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.084098  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:42.060632  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:42.096153  8057 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:13:42.100766  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:42.080981 10623 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:42.088514 10540 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:42.084858 10644 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:42.106048  8029 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:42.119146  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.119385  8029 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:42.080575 10667 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:42.088826 10716 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.081712 10674 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:42.113448  8134 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:42.080638 10626 raft_consensus.cc:3055] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:42.127290  8057 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:42.071635 10727 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.130863  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1338) took 2085 ms. Trace:
I20260709 13:13:42.132659  8693 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:42.129570  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.135509  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.110286 10705 consensus_queue.cc:1243] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:42.110759 10521 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:42.118070 10623 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:42.156383  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.129415 10667 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:42.174278  8208 rpcz_store.cc:276] 0709 13:13:40.045189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.061516 (+ 16327us) service_pool.cc:224] Handling call
0709 13:13:40.061793 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:40.187288 (+125495us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:40.963379 (+776091us) spinlock_profiling.cc:243] Waited 7.25 ms on lock 0x7b180023ab20. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ea3c0a 00007f6dc15e3219 00007f6db7f2ba40 00007f6db7f264e4 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf 00007f6dbb030038
0709 13:13:40.963894 (+   515us) log.cc:844] Serialized 14 byte log entry
0709 13:13:41.462075 (+498181us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:41.462564 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 13:13:41.462695 (+   131us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:41.462728 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.130595 (+667867us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b540025d5d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:42.130644 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:42.130687 (+    43us) raft_consensus.cc:1685] finished
0709 13:13:42.130699 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:42.130811 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10625,"prepare.queue_time_us":776187,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":4065,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":16566,"spinlock_wait_cycles":304409216,"thread_start_us":497809,"threads_started":1,"wal-append.queue_time_us":569788}
I20260709 13:13:42.170373  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.168486  8722 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:42.123247 10731 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:42.213191  8134 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:42.213212  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.219913  8136 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:42.137355 10626 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:42.116468 10835 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:42.122466 10730 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:42.174602 10819 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:42.136695 10737 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:13:42.210569 10799 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:42.128552 10644 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.137848 10716 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.247051 10426 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:42.336683  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.216629 10516 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.333130  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:42.333933  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 280) took 1919 ms. Trace:
I20260709 13:13:42.098593 10853 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.191s
I20260709 13:13:42.364579  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.337182  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.279358  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2313) took 1867 ms. Trace:
I20260709 13:13:42.361979  8134 raft_consensus.cc:2393] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:42.365803  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.366112  8136 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:42.366174  8056 rpcz_store.cc:276] 0709 13:13:40.411995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.044572 (+632577us) service_pool.cc:224] Handling call
0709 13:13:41.044816 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.064882 (+ 20066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.279095 (+1214213us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b540021a554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:42.279290 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36213120}
I20260709 13:13:42.366747  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.367367  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.367636  8134 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:42.363333  8059 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:42.368041  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.368901  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.363554  8054 rpcz_store.cc:276] 0709 13:13:40.414776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.124749 (+709973us) service_pool.cc:224] Handling call
0709 13:13:41.125084 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:41.916306 (+791222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:41.916334 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:41.916735 (+   401us) log.cc:844] Serialized 26 byte log entry
0709 13:13:42.121080 (+204345us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:42.174917 (+ 53837us) log.cc:844] Serialized 14 byte log entry
0709 13:13:42.175134 (+   217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.175216 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:42.333714 (+158498us) raft_consensus.cc:1685] finished
0709 13:13:42.333736 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:42.333877 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":874191,"prepare.queue_time_us":126819,"prepare.run_wall_time_us":2872,"raft.queue_time_us":847372,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":34065,"spinlock_wait_cycles":23162112,"thread_start_us":134446,"threads_started":1,"wal-append.queue_time_us":320954}
I20260709 13:13:42.369984  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:42.370177  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.372278  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.371320  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.373752  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.374595  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.350881 10799 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.259250 10700 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:42.373124  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:42.364331 10856 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 13:13:42.270107 10819 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:42.352438 10759 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:42.392941  8134 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:42.396059  8137 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:42.379994 10805 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:42.401511  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:42.401767  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:42.253149 10835 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.403354  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:42.279384 10674 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.332907 10626 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.372638 10516 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.412010 10835 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:42.412514 10626 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:42.429482 10644 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:42.430604 10681 consensus_meta.cc:287] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.549s	user 0.011s	sys 0.000s
W20260709 13:13:42.414105 10862 log.cc:927] Time spent T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:13:42.414885 10727 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:42.415393 10861 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:13:42.417443 10819 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:42.424260 10609 consensus_meta.cc:287] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.561s	user 0.002s	sys 0.008s
I20260709 13:13:42.438573 10609 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:42.437395 10822 raft_consensus.cc:3055] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:42.439759 10822 raft_consensus.cc:740] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:13:42.440680 10820 consensus_meta.cc:287] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.575s	user 0.005s	sys 0.003s
I20260709 13:13:42.440809 10609 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:42.440526 10822 consensus_queue.cc:260] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.441483 10822 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:42.445856 10716 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:42.446751 10821 raft_consensus.cc:3055] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:42.447240 10821 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:42.402661 10769 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:42.437829 10702 consensus_meta.cc:287] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.791s	user 0.009s	sys 0.000s
I20260709 13:13:42.448069 10821 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.441077 10820 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.449245 10799 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:42.435523 10845 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.452061 10769 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:42.455554 10816 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 1.116s	user 0.002s	sys 0.000s
I20260709 13:13:42.454739 10821 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:42.453284 10820 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:42.453648 10845 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:42.457814 10840 consensus_meta.cc:287] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.000s
I20260709 13:13:42.460160 10840 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:42.461776 10840 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:42.498615  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:42.504096  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:42.498667  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:42.500568  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.504237  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:42.379053  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:42.457599 10769 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:42.506034  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.508774  8722 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:42.510878  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.512408  8132 raft_consensus.cc:2393] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:42.514284  8137 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:13:42.519088  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.519495  8721 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:13:42.522406  8730 raft_consensus.cc:2393] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:42.415306 10519 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:42.523581  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:42.546720  8055 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:42.546918 10799 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:42.548079  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:42.547783 10799 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.549288  8055 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:42.524622  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 472) took 2175 ms. Trace:
I20260709 13:13:42.550769  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.551638  8055 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:42.552059  8058 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:42.550657  8724 rpcz_store.cc:276] 0709 13:13:40.348772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.402154 (+ 53382us) service_pool.cc:224] Handling call
0709 13:13:40.471009 (+ 68855us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:40.536864 (+ 65855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:40.536904 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:40.537519 (+   615us) log.cc:844] Serialized 52 byte log entry
0709 13:13:41.045270 (+507751us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:41.062482 (+ 17212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.096164 (+ 33682us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:42.455016 (+1358852us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:42.520090 (+ 65074us) raft_consensus.cc:1685] finished
0709 13:13:42.520135 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:42.524233 (+  4098us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31840,"raft.queue_time_us":36680,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"spinlock_wait_cycles":60562304,"thread_start_us":494453,"threads_started":1,"wal-append.queue_time_us":687468}
I20260709 13:13:42.555927 10682 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:42.557104  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.558842  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.559958  8026 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:42.560236  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:42.505009  8059 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:42.561710  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.561906  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:42.562476 10814 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 1.310s	user 0.003s	sys 0.000s
I20260709 13:13:42.562827  8059 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:42.561939 10810 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:13:42.566430 10805 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:42.567584 10805 raft_consensus.cc:3055] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:42.568073 10805 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:42.570960  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 598585us
I20260709 13:13:42.535966  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:42.536873  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:42.538782  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:42.544994 10516 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:42.556795 10682 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.452170 10702 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.611913 10630 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:42.612648 10630 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:42.613148 10630 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:42.616019 10610 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:42.623229  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:42.629616  8211 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:13:42.632278  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 388) took 1097 ms. Trace:
I20260709 13:13:42.632737  8211 rpcz_store.cc:276] 0709 13:13:41.534880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.704607 (+169727us) service_pool.cc:224] Handling call
0709 13:13:41.704915 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:42.630228 (+925313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.631821 (+  1593us) spinlock_profiling.cc:243] Waited 924 ms on lock 0x7b540018a454. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:42.632133 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2124778880}
I20260709 13:13:42.654029 10689 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:42.655146 10689 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:42.663430  8026 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:42.664695 10815 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:42.665117 10815 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:42.665463 10815 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:42.670964  8061 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:42.673278 10799 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:42.675050 10837 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.982s	user 0.001s	sys 0.003s
I20260709 13:13:42.569751 10805 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:42.697532  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2412) took 2552 ms. Trace:
I20260709 13:13:42.697901  8205 rpcz_store.cc:276] 0709 13:13:40.144744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:40.917422 (+772678us) service_pool.cc:224] Handling call
0709 13:13:41.000081 (+ 82659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:41.014825 (+ 14744us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:41.014857 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:41.037266 (+ 22409us) log.cc:844] Serialized 26 byte log entry
0709 13:13:41.596935 (+559669us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:41.596990 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:41.597080 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:42.674057 (+1076977us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:42.683408 (+  9351us) raft_consensus.cc:1685] finished
0709 13:13:42.683436 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:42.697345 (+ 13909us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3958,"raft.queue_time_us":11686,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":6517,"spinlock_wait_cycles":3307264,"thread_start_us":490367,"threads_started":1,"wal-append.queue_time_us":593159}
I20260709 13:13:42.658210 10689 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:42.724383 10623 consensus_meta.cc:287] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.510s	user 0.013s	sys 0.004s
I20260709 13:13:42.728164 10623 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.726585 10799 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:42.767030 10799 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:42.808159  8055 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
W20260709 13:13:42.770241 10521 consensus_meta.cc:287] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.583s	user 0.005s	sys 0.013s
I20260709 13:13:42.823073  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2434) took 1237 ms. Trace:
W20260709 13:13:43.160702 10730 consensus_meta.cc:287] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.901s	user 0.013s	sys 0.001s
I20260709 13:13:43.133929 10810 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:43.175464 10682 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:43.176470 10810 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:43.177424 10369 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:43.190369 10799 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:43.191030 10822 consensus_meta.cc:287] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.746s	user 0.009s	sys 0.005s
I20260709 13:13:43.099611 10521 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:43.201947 10805 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:43.017872  8209 rpcz_store.cc:276] 0709 13:13:41.585312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.766613 (+181301us) service_pool.cc:224] Handling call
0709 13:13:41.822769 (+ 56156us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.823022 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:41.829914 (+  6892us) log.cc:844] Serialized 14 byte log entry
0709 13:13:42.822206 (+992292us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:42.822363 (+   157us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:42.822491 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:42.822614 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:42.822846 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":638908,"prepare.queue_time_us":6632,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":683241,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":206,"thread_start_us":352433,"threads_started":2,"wal-append.queue_time_us":215882}
I20260709 13:13:43.258164  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.229430 10730 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:43.274175 10471 raft_consensus.cc:3055] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:43.275545  8721 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:43.275432  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:43.276432  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:43.276487  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:43.276823  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:43.277209  8727 raft_consensus.cc:2393] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:43.278812 10626 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.279692 10626 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:43.280421  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2431) took 1695 ms. Trace:
I20260709 13:13:43.280954  8210 rpcz_store.cc:276] 0709 13:13:41.584547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.731691 (+147144us) service_pool.cc:224] Handling call
0709 13:13:41.731975 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:43.162462 (+1430487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:43.162514 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:43.163170 (+   656us) log.cc:844] Serialized 26 byte log entry
0709 13:13:43.201110 (+ 37940us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:43.201162 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.226561 (+ 25399us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:43.279997 (+ 53436us) raft_consensus.cc:1685] finished
0709 13:13:43.280018 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:43.280198 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":849096,"raft.queue_time_us":893495,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":169,"thread_start_us":344402,"threads_started":2,"wal-append.queue_time_us":114534}
W20260709 13:13:43.282574  8755 heartbeater.cc:646] Failed to heartbeat to 127.6.198.126:38543 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.6.198.126:38543 timed out after 15.000s (SENT)
I20260709 13:13:43.282649 10610 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.283510 10610 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:43.285805 10610 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:43.291406  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:43.292637  8136 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:43.266091  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.289151  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:43.293274  8056 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:43.293717  8134 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:43.294152  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:43.288132  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:43.290247 10682 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.294893  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:43.295976  8138 raft_consensus.cc:2393] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:43.296830  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.298683 10810 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:43.301090  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:13:43.257413  8175 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.903s (SENT) [suppressed 20 similar messages]
I20260709 13:13:43.294943 10682 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:43.303185  8136 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:43.304596 10682 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:43.306768 10623 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:43.307864 10886 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:43.308732 10630 consensus_meta.cc:287] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.695s	user 0.012s	sys 0.000s
I20260709 13:13:43.308650 10886 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:43.309569 10667 consensus_meta.cc:287] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 1.082s	user 0.017s	sys 0.000s
I20260709 13:13:43.309216 10630 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:43.330912  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.332020 10630 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:43.332744 10630 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:43.333477  8101 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:43.333390 10630 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:43.275712  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.298830  8056 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:43.310055 10667 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:43.313961  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:43.314448 10886 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:43.314610  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:43.317134 10888 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:43.317760 10884 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.322021 10887 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.323993  8206 raft_consensus.cc:1217] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Deduplicated request from leader. Original: 2.4->[3.5-8.7]   Dedup: 3.5->[6.6-8.7]
W20260709 13:13:43.325268 10821 consensus_meta.cc:287] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.869s	user 0.009s	sys 0.003s
I20260709 13:13:43.278841 10471 raft_consensus.cc:740] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:43.327244  8210 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:13:43.351538  8206 pending_rounds.cc:85] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 5
I20260709 13:13:43.339679  8026 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:43.338203  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:43.349174  8058 consensus_meta.cc:287] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.796s	user 0.012s	sys 0.000s
W20260709 13:13:43.353879  8059 consensus_meta.cc:287] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.790s	user 0.008s	sys 0.003s
I20260709 13:13:43.355940  8058 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:43.356127  8059 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:43.354743  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:43.355898  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:43.358486  8056 raft_consensus.cc:2376] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6. Re-sending same reply.
I20260709 13:13:43.359488  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:43.359927  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1219) took 1235 ms. Trace:
I20260709 13:13:43.361155  8207 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:43.360355  8059 rpcz_store.cc:276] 0709 13:13:42.121863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:42.561471 (+439608us) service_pool.cc:224] Handling call
0709 13:13:43.356743 (+795272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:43.355507  8206 pending_rounds.cc:107] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:13:43.364660  8205 raft_consensus.cc:2410] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:43.366195  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:43.347877 10630 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:43.368227  8055 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:43.371716  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1225) took 1245 ms. Trace:
I20260709 13:13:43.369557 10757 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:13:43.365588  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 99497.2us
I20260709 13:13:43.364289  8206 raft_consensus.cc:2889] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:43.375362  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:43.375208 10471 consensus_queue.cc:260] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:43.375871  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:43.376360  8205 raft_consensus.cc:2410] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:13:43.375025 10887 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:43.371080 10730 raft_consensus.cc:2764] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:13:43.376760  8131 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:43.367470 10667 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:43.349531 10689 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:43.372177  8055 rpcz_store.cc:276] 0709 13:13:42.126070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.225466 (+1099396us) service_pool.cc:224] Handling call
0709 13:13:43.371666 (+146200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:43.378438  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.380003  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.380896 10884 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:43.381685  8204 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.013s	sys 0.000s
I20260709 13:13:43.381807 10887 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:43.376704 10471 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:43.390674  8204 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.016s	sys 0.000s
I20260709 13:13:43.388638  8730 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:43.396891  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:43.397377  8055 raft_consensus.cc:2376] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5. Re-sending same reply.
I20260709 13:13:43.397907  8026 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:43.400789  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:43.402058  8211 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:43.389346 10702 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:43.395385 10821 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.396601 10369 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:43.416293  8136 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:43.403724 10626 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:43.441367  8029 connection.cc:256] RPC call timeout handler was delayed by -1.8714707139999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
I20260709 13:13:43.442261  8029 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:43.443924 10884 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:43.444023  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:43.445428 10541 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:43.398618 10888 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:43.446223 10541 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:43.415969  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 83302.7us
W20260709 13:13:43.414554 10815 consensus_meta.cc:287] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.746s	user 0.011s	sys 0.002s
I20260709 13:13:43.437124  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.438313  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 278) took 4351 ms. Trace:
I20260709 13:13:43.433225  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.437867  8205 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:43.421133  8056 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:43.416883 10667 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:43.470829  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2339) took 1361 ms. Trace:
I20260709 13:13:43.471350  8055 rpcz_store.cc:276] 0709 13:13:42.109482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.401422 (+1291940us) service_pool.cc:224] Handling call
0709 13:13:43.470755 (+ 69333us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:43.479926  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 85562.2us
I20260709 13:13:43.480222  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:43.480604  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:43.481351 10674 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:13:43.481539  8726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 621) took 1513 ms (client timeout 1619 ms). Trace:
I20260709 13:13:43.481652  8026 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:43.405659 10689 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:43.493799  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:43.482709  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:43.493768  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:43.482093  8726 rpcz_store.cc:269] 0709 13:13:41.967891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.969777 (+  1886us) service_pool.cc:224] Handling call
0709 13:13:43.480280 (+1510503us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:43.481442 (+  1162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10909440}
I20260709 13:13:43.462754 10810 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:43.504953  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.499435  8057 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:43.483912 10626 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:43.507170  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1246) took 1005 ms. Trace:
I20260709 13:13:43.470160 10623 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:13:43.493192 10885 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:13:43.464390 10815 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:43.520744 10815 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:43.523104  8056 raft_consensus.cc:2410] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:13:43.523602  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1249) took 1020 ms. Trace:
I20260709 13:13:43.523936  8056 rpcz_store.cc:276] 0709 13:13:42.503297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.468970 (+965673us) service_pool.cc:224] Handling call
0709 13:13:43.523550 (+ 54580us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:43.524472 10730 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:43.524534 10521 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:43.524912  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.525326 10730 consensus_queue.cc:1243] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:43.525651  8056 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:43.526484 10799 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.527374 10730 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:43.527225 10799 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:43.527783 10730 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:43.528719 10730 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:43.529496 10799 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:43.529933 10730 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:43.489180  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:43.415748 10821 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:43.472718 10369 raft_consensus.cc:697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:43.513386  8208 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:43.526158  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1236) took 1030 ms. Trace:
I20260709 13:13:43.542784  8056 rpcz_store.cc:276] 0709 13:13:42.496465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.524618 (+1028153us) service_pool.cc:224] Handling call
0709 13:13:43.526107 (+  1489us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:43.507624  8057 rpcz_store.cc:276] 0709 13:13:42.501926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.493499 (+991573us) service_pool.cc:224] Handling call
0709 13:13:43.507102 (+ 13603us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:43.539806  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.540421  8058 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:43.544793  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1252) took 1040 ms. Trace:
I20260709 13:13:43.490911 10547 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:43.564946  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1234) took 1228 ms. Trace:
I20260709 13:13:43.570924  8055 rpcz_store.cc:276] 0709 13:13:42.337020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.472142 (+1135122us) service_pool.cc:224] Handling call
0709 13:13:43.564879 (+ 92737us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:43.553586  8058 rpcz_store.cc:276] 0709 13:13:42.504435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.366025 (+861590us) service_pool.cc:224] Handling call
0709 13:13:43.544587 (+178562us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:43.506521 10888 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:43.582031  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.584568  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:43.457620  8060 rpcz_store.cc:276] 0709 13:13:39.084521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:39.296343 (+211822us) service_pool.cc:224] Handling call
0709 13:13:39.342734 (+ 46391us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:40.186053 (+843319us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:40.202387 (+ 16334us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b48000801d8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:13:40.458866 (+256479us) log.cc:844] Serialized 14 byte log entry
0709 13:13:41.059476 (+600610us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:41.096493 (+ 37017us) log.cc:844] Serialized 52 byte log entry
0709 13:13:41.096695 (+   202us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:41.097846 (+  1151us) log.cc:844] Serialized 14 byte log entry
0709 13:13:41.915053 (+817207us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b1800389200. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ea3c0a 00007f6dc15e3219 00007f6db7f2ba40 00007f6db7f264e4 00007f6db7f26455 00007f6dbb00e060 00007f6dbb020716 00007f6dbb031287 00007f6dbb031208
0709 13:13:42.526519 (+611466us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:43.390566 (+864047us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b1800389200. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf
0709 13:13:43.398759 (+  8193us) log.cc:844] Serialized 14 byte log entry
0709 13:13:43.418248 (+ 19489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.427691 (+  9443us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:43.427761 (+    70us) raft_consensus.cc:1685] finished
0709 13:13:43.427777 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:43.436157 (+  8380us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1043789,"prepare.queue_time_us":400368,"prepare.run_cpu_time_us":4752,"prepare.run_wall_time_us":47882,"raft.queue_time_us":1689357,"raft.run_cpu_time_us":2827,"raft.run_wall_time_us":42149,"spinlock_wait_cycles":403396864,"thread_start_us":862271,"threads_started":3,"wal-append.queue_time_us":698969}
I20260709 13:13:43.482134 10674 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:43.586625  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:43.530284 10884 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:13:43.598057 10889 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:13:43.599810 10369 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:43.600817 10667 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:43.583528 10815 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:43.617451  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.621714  8205 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:43.623577  8057 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:43.624986  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1242) took 1124 ms. Trace:
I20260709 13:13:43.625335  8057 rpcz_store.cc:276] 0709 13:13:42.500217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.544303 (+1044086us) service_pool.cc:224] Handling call
0709 13:13:43.624923 (+ 80620us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:43.627372  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 138307us
I20260709 13:13:43.630685  8059 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:43.633738  8056 raft_consensus.cc:2410] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:13:43.635156  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2340) took 1524 ms. Trace:
I20260709 13:13:43.635576  8056 rpcz_store.cc:276] 0709 13:13:42.110964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.565063 (+1454099us) service_pool.cc:224] Handling call
0709 13:13:43.635073 (+ 70010us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:43.627015 10884 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:43.652501  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.655680  8057 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:43.656605  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:43.673132  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.683866  8056 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
W20260709 13:13:43.690212  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 210082us
I20260709 13:13:43.693177  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:43.694309  8057 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:43.681468 10689 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:43.711552  8208 raft_consensus.cc:2393] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:43.735062  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 136142us
I20260709 13:13:43.736315  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:43.733007  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.759436  8056 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 5: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:43.766772  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:43.768036 10884 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:43.768996  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:43.773332 10810 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:43.774593 10815 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.775336 10815 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:43.779783 10896 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.780332 10896 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:43.782533  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 59844.1us
I20260709 13:13:43.785478  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:43.791414 10547 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:43.791486  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:43.791883 10547 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:43.792239 10547 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:43.792694  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:43.787528 10701 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:43.794425  8726 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:43.768118  8176 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:43.796030  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 114466us
I20260709 13:13:43.798280 10884 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:43.770148  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:43.790167 10674 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:43.799156 10892 log.cc:927] Time spent T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:13:43.800927  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:43.802619  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:43.803299  8057 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:43.803418 10667 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:43.805389 10810 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:13:43.807006  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.808025 10893 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.808856 10893 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:43.808920 10884 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:43.810675  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:43.811087 10884 consensus_queue.cc:1243] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:43.811635  8724 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:43.811771 10689 raft_consensus.cc:3055] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:43.812374 10689 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:43.798792 10896 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:43.806859 10815 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:43.816315  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:43.817057  8208 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:43.823092  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.823900  8056 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:43.824888  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:43.825556  8056 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:43.825706  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:43.826642  8061 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:43.827104  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1250) took 1323 ms. Trace:
I20260709 13:13:43.827409  8061 rpcz_store.cc:276] 0709 13:13:42.503637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.825379 (+1321742us) service_pool.cc:224] Handling call
0709 13:13:43.827046 (+  1667us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:43.828287  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:43.829077  8061 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:43.836112  8205 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:43.813766 10689 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:43.855397  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:43.860715  8175 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.000s	sys 0.017s
I20260709 13:13:43.862124  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:43.865296  8205 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:43.865830  8693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.005s	sys 0.012s
W20260709 13:13:43.874614  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 335369us
W20260709 13:13:43.920882  8693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.005s	sys 0.012s
I20260709 13:13:43.907845  8726 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:43.899129  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:43.918717  8175 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.000s	sys 0.017s
W20260709 13:13:43.927527  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 626604us
I20260709 13:13:43.929061 10906 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:43.930245 10898 raft_consensus.cc:3055] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:43.930905 10898 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:43.932173 10898 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:43.923316  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 669010us
I20260709 13:13:43.933900 10898 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:43.958150  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:43.955771  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 161757us
I20260709 13:13:43.960454  8205 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
W20260709 13:13:43.966220  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 444196us
I20260709 13:13:43.966960  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:43.965940  8208 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
W20260709 13:13:44.008798  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.005605  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.014088  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.011439  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2433) took 2420 ms. Trace:
I20260709 13:13:44.031476  8205 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:44.033339 10426 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.028811 10909 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.034185 10426 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:44.034765 10521 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:44.034900 10909 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:44.035354 10521 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.036960 10426 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:44.037204  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:44.037333 10909 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:44.036962  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1410) took 2206 ms. Trace:
I20260709 13:13:44.046380  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.047268  8208 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:44.052645  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:44.054206  8057 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:44.017345  8204 rpcz_store.cc:276] 0709 13:13:41.585065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.750411 (+165346us) service_pool.cc:224] Handling call
0709 13:13:41.750781 (+   370us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:13:42.571021 (+820240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:42.571081 (+    60us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:13:42.581004 (+  9923us) log.cc:844] Serialized 78 byte log entry
0709 13:13:43.393269 (+812265us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:43.597371 (+204102us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:43.861660 (+264289us) log.cc:844] Serialized 14 byte log entry
0709 13:13:43.866481 (+  4821us) log.cc:844] Serialized 14 byte log entry
0709 13:13:43.922393 (+ 55912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.959233 (+ 36840us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:43.998817 (+ 39584us) raft_consensus.cc:1685] finished
0709 13:13:43.998848 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.005082 (+  6234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":759244,"prepare.queue_time_us":568944,"raft.queue_time_us":816710,"raft.run_cpu_time_us":454,"raft.run_wall_time_us":39129,"spinlock_wait_cycles":7022848,"thread_start_us":1258228,"threads_started":2,"wal-append.queue_time_us":811871}
I20260709 13:13:44.022320  8724 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:44.045377  8129 rpcz_store.cc:276] 0709 13:13:41.830570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.830702 (+   132us) service_pool.cc:224] Handling call
0709 13:13:41.831043 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.886797 (+ 55754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.036546 (+2149749us) spinlock_profiling.cc:243] Waited 55 ms on lock 0x7b54001315d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:44.036777 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126408064}
W20260709 13:13:44.030889 10889 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:13:44.026383 10443 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.070588  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:44.071689  8062 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
W20260709 13:13:44.072408  8062 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1241) took 1572 ms (client timeout 1846 ms). Trace:
W20260709 13:13:44.072857  8062 rpcz_store.cc:269] 0709 13:13:42.499732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.070194 (+1570462us) service_pool.cc:224] Handling call
0709 13:13:44.072279 (+  2085us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:44.077564  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.079258  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:44.080375  8728 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:44.085764  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.089047  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1412) took 1966 ms. Trace:
W20260709 13:13:44.097707  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 292909us
I20260709 13:13:44.099426  8721 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:13:44.100663  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:44.089557  8132 rpcz_store.cc:276] 0709 13:13:42.112395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:42.114269 (+  1874us) service_pool.cc:224] Handling call
0709 13:13:44.062294 (+1948025us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b5400174154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:44.078882 (+ 16588us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":691249408}
I20260709 13:13:44.133235  8205 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:44.064456 10893 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:44.139923  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:44.147338  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:44.148151  8723 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:44.151815  8727 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:13:44.154882  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.155617  8205 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:44.156641  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.158529 10888 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.159853  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1235) took 1795 ms. Trace:
I20260709 13:13:44.159598 10888 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:44.160548  8027 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:44.160262  8059 rpcz_store.cc:276] 0709 13:13:42.364475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.410507 (+1046032us) service_pool.cc:224] Handling call
0709 13:13:44.159699 (+749192us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b540025f3d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:44.159799 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398103424}
I20260709 13:13:44.163528 10541 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:44.164636 10913 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:13:44.167924 10667 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:44.168511 10667 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:44.169476 10667 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:44.170456  9980 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:44.170923  9980 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:44.172112  9980 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:44.173408  9980 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:44.174469  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 297) took 2078 ms. Trace:
I20260709 13:13:44.175068  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.174827  8063 rpcz_store.cc:276] 0709 13:13:42.055715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:42.056792 (+  1077us) service_pool.cc:224] Handling call
0709 13:13:42.057090 (+   298us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:42.254076 (+196986us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:42.254101 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:42.254651 (+   550us) log.cc:844] Serialized 52 byte log entry
0709 13:13:42.559683 (+305032us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:42.559761 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.091958 (+1532197us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b54003ba1d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:44.092002 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.092053 (+    51us) raft_consensus.cc:1685] finished
0709 13:13:44.092072 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.133912 (+ 41840us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3373,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":3055,"spinlock_wait_cycles":452189440,"thread_start_us":288325,"threads_started":1,"wal-append.queue_time_us":1035898}
I20260709 13:13:44.176333  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.177004  8063 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:44.177143  8209 raft_consensus.cc:2410] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:44.177601  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2337) took 2073 ms. Trace:
I20260709 13:13:44.177963  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:44.175094 10799 raft_consensus.cc:3055] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:44.178694 10369 consensus_queue.cc:1059] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:44.181550  8176 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:44.177980  8063 rpcz_store.cc:276] 0709 13:13:42.103902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.176513 (+2072611us) service_pool.cc:224] Handling call
0709 13:13:44.176867 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.177376 (+   509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.177521 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:44.200892  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.221701  8205 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
W20260709 13:13:44.227627  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.164557 10541 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:44.095359 10815 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.130s
I20260709 13:13:44.136812 10426 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:44.239526  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:44.080603 10443 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:44.251659 10541 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.187819 10799 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:44.158174  7956 catalog_manager.cc:5697] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:44.256045 10511 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.256417  7956 catalog_manager.cc:5697] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:44.256851 10511 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:44.254240 10541 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:44.276245 10888 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:44.277029 10888 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:44.278115  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:44.279759  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.281193  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.280417  7956 catalog_manager.cc:5697] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 4, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 4 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:44.284754  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:44.285569  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.286244  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.301097  8210 raft_consensus.cc:2410] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:13:44.277697 10888 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.300415  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.293570  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.292868  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2432) took 2707 ms. Trace:
I20260709 13:13:44.303027  8727 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:44.292898  7956 catalog_manager.cc:5697] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 3 to 5, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 5 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:44.320336  8207 raft_consensus.cc:2410] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:13:44.321225  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.322599  8203 raft_consensus.cc:2410] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:44.316321  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 347) took 1032 ms. Trace:
I20260709 13:13:44.287979 10884 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:13:44.326653  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 302740us
I20260709 13:13:44.326150  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 389) took 1003 ms. Trace:
I20260709 13:13:44.328773  8205 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:44.237155 10547 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.314694  7956 catalog_manager.cc:5697] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 2 to 5, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 5 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:44.335506  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2332) took 2262 ms. Trace:
I20260709 13:13:44.305771  8212 rpcz_store.cc:276] 0709 13:13:41.584832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.741418 (+156586us) service_pool.cc:224] Handling call
0709 13:13:41.741685 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:41.741864 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:41.742100 (+   236us) log.cc:844] Serialized 14 byte log entry
0709 13:13:43.487755 (+1745655us) spinlock_profiling.cc:243] Waited 6.24 ms on lock 0x7b18003adec0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ef72a6 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f29522 00007f6db7f2b7c1
0709 13:13:43.606571 (+118816us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b18003adec0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f29522 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:13:44.292183 (+685612us) spinlock_profiling.cc:243] Waited 6.27 ms on lock 0x7b18003adec0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbb00e060 00007f6dbb020716 00007f6dbb031287
0709 13:13:44.292343 (+   160us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:44.292425 (+    82us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:44.292487 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.292572 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.292699 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":790589,"prepare.queue_time_us":187,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":828744,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":16577,"spinlock_wait_cycles":38751488,"thread_start_us":170373,"threads_started":2,"wal-append.queue_time_us":55279,"wal-append.run_cpu_time_us":1374,"wal-append.run_wall_time_us":1024829}
W20260709 13:13:44.349418  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 333574us
I20260709 13:13:44.327510 10884 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.373361  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:44.328413  8206 rpcz_store.cc:276] 0709 13:13:43.322773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.322992 (+   219us) service_pool.cc:224] Handling call
0709 13:13:43.323476 (+   484us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:13:43.399834 (+ 76358us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:43.819396 (+419562us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.038172 (+218776us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:44.052924 (+ 14752us) log.cc:844] Serialized 52 byte log entry
0709 13:13:44.053171 (+   247us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:44.053206 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.053267 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.324536 (+271269us) raft_consensus.cc:1685] finished
0709 13:13:44.324559 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.326088 (+  1529us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":514695,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":2090,"raft.queue_time_us":16201,"raft.run_cpu_time_us":536,"raft.run_wall_time_us":24328,"thread_start_us":580746,"threads_started":2,"wal-append.queue_time_us":382336}
W20260709 13:13:44.378516  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 285220us
I20260709 13:13:44.355566  8063 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:44.380671  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.382942  8205 raft_consensus.cc:2410] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:13:44.384431  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.385309  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.319072 10426 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.389956  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1226) took 2259 ms. Trace:
I20260709 13:13:44.387144  8212 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 4.
I20260709 13:13:44.390573  8063 rpcz_store.cc:276] 0709 13:13:42.126403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.247793 (+2121390us) service_pool.cc:224] Handling call
0709 13:13:44.329622 (+ 81829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.385862 (+ 56240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.386136 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:44.392187  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 457497us
I20260709 13:13:44.393869  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.364471 10884 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:44.398577  8205 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:44.404740  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:44.425379  8209 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:13:44.427826  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:44.429405  8063 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:44.431300  8027 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:44.399061  8693 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:44.427127  8727 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
W20260709 13:13:44.442828  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 112799us
W20260709 13:13:44.444095  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 262688us
I20260709 13:13:44.443670  8029 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:44.450878  8212 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:44.451541  8207 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:13:44.451948  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1358) took 1036 ms. Trace:
I20260709 13:13:44.452312  8212 rpcz_store.cc:276] 0709 13:13:43.415352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.391342 (+975990us) service_pool.cc:224] Handling call
0709 13:13:44.427038 (+ 35696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.451679 (+ 24641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.451885 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:44.454923  8205 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:44.456391  8029 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:44.444244 10626 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:44.458236  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 63212.4us
W20260709 13:13:44.462755  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 59662us
I20260709 13:13:44.435915 10740 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.459551  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:44.445166 10757 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.464255  8027 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:44.461759  7956 catalog_manager.cc:5697] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 5, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 5 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:44.469604  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:44.455829 10805 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:44.473989  8212 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:13:44.474829  8063 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:44.475945 10834 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.476763 10822 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:44.476651 10834 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.477231 10822 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.477629 10822 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:44.324921  8130 rpcz_store.cc:276] 0709 13:13:43.283985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.284134 (+   149us) service_pool.cc:224] Handling call
0709 13:13:43.284521 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:43.284908 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:43.701527 (+416619us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.314902 (+613375us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.315837 (+   935us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:44.315884 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:44.315935 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.316006 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.316126 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":365288,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":84,"raft.queue_time_us":264049,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":18654,"thread_start_us":976813,"threads_started":3,"wal-append.queue_time_us":366848}
I20260709 13:13:44.478751 10834 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:44.478926 10443 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:44.476670 10915 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:44.480557 10821 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:44.480345 10915 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.481400  8722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 467) took 2079 ms. Trace:
I20260709 13:13:44.481779  8722 rpcz_store.cc:276] 0709 13:13:42.401739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:42.401886 (+   147us) service_pool.cc:224] Handling call
0709 13:13:42.402167 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:42.534447 (+132280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.481130 (+1946683us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54002e04d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:44.481329 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":243807104}
I20260709 13:13:44.482365 10915 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:44.483515 10928 log.cc:927] Time spent T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:13:44.485096  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2330) took 2424 ms. Trace:
I20260709 13:13:44.486418 10541 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:44.485430  8058 rpcz_store.cc:276] 0709 13:13:42.060820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.573666 (+1512846us) service_pool.cc:224] Handling call
0709 13:13:43.574095 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:43.853795 (+279700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:43.853836 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:43.864714 (+ 10878us) log.cc:844] Serialized 26 byte log entry
0709 13:13:44.141408 (+276694us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:44.175293 (+ 33885us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.175838 (+   545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.176001 (+   163us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.176765 (+   764us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b48000801d8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:13:44.484813 (+308048us) raft_consensus.cc:1685] finished
0709 13:13:44.484911 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.485027 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33352,"prepare.queue_time_us":36497,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":21571,"raft.run_cpu_time_us":2882,"raft.run_wall_time_us":581048,"spinlock_wait_cycles":40297216,"thread_start_us":500619,"threads_started":2,"wal-append.queue_time_us":413576}
I20260709 13:13:44.486804 10541 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:44.487197 10541 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:44.487550  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:44.488677  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:44.489672  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:44.465216 10511 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:44.470880 10623 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:44.494702 10623 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:44.494943  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.495258 10623 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:44.497738 10547 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:44.472987  7956 catalog_manager.cc:5697] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 4, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 4 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:44.494774  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1353) took 1184 ms. Trace:
I20260709 13:13:44.499739  8211 rpcz_store.cc:276] 0709 13:13:43.309955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.310324 (+   369us) service_pool.cc:224] Handling call
0709 13:13:44.494541 (+1184217us) spinlock_profiling.cc:243] Waited 56 ms on lock 0x7b540010c054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:44.494675 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128851712}
I20260709 13:13:44.475916 10944 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:13:44.474632  8212 pending_rounds.cc:85] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 3
I20260709 13:13:44.504540  7956 catalog_manager.cc:5697] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 1 to 4, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 4 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:44.505265  8212 pending_rounds.cc:107] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:13:44.505602  8212 raft_consensus.cc:2889] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:13:44.505846  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 117055us
W20260709 13:13:44.509203  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 50233us
I20260709 13:13:44.515846  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:44.514788  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:44.500067 10805 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.525183  8696 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:44.512741 10471 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:44.515147 10757 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.542551  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:44.543124  8129 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:44.543648 10757 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:44.546738  7956 catalog_manager.cc:5697] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:44.548295  7956 catalog_manager.cc:5697] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:44.552341  8130 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:44.552660  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:44.553395  8137 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:44.554920 10426 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:44.555050 10924 log.cc:927] Time spent T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:13:44.504089 10547 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:44.499243 10940 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:44.471920 10626 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:44.564100  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 104614us
I20260709 13:13:44.534590  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:44.571532 10905 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:44.572225 10905 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:44.572788 10623 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.571456 10843 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:44.574060 10843 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:44.574949 10843 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:44.575736  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2329) took 2515 ms. Trace:
I20260709 13:13:44.572687 10905 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:44.576129  8055 rpcz_store.cc:276] 0709 13:13:42.060566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.571805 (+1511239us) service_pool.cc:224] Handling call
0709 13:13:43.572419 (+   614us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:43.572972 (+   553us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:43.837746 (+264774us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.225656 (+387910us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.225991 (+   335us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:44.312356 (+ 86365us) log.cc:844] Serialized 26 byte log entry
0709 13:13:44.312540 (+   184us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:44.312589 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.312683 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.495954 (+183271us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b48000801d8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:13:44.575536 (+ 79582us) raft_consensus.cc:1685] finished
0709 13:13:44.575558 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.575677 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58137,"prepare.queue_time_us":487934,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":74,"raft.queue_time_us":9743,"raft.run_cpu_time_us":1576,"raft.run_wall_time_us":307582,"spinlock_wait_cycles":5285248,"thread_start_us":543967,"threads_started":2,"wal-append.queue_time_us":504347}
W20260709 13:13:44.577608 10898 consensus_meta.cc:287] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.001s
I20260709 13:13:44.537834 10805 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:44.580662  8133 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:44.580920  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:44.581794  8138 raft_consensus.cc:2410] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:13:44.582679  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:44.583060  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:44.583271  8131 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:44.583726 10626 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:44.583007 10905 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:44.584014  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:44.567203 10547 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.583752  8133 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:44.586371 10623 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:44.586573 10547 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:44.525034 10888 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:44.589084  8055 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:44.594769  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1233) took 2322 ms. Trace:
I20260709 13:13:44.590884 10541 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.591964  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2334) took 2502 ms. Trace:
I20260709 13:13:44.604741  8057 rpcz_store.cc:276] 0709 13:13:42.089943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.059592 (+1969649us) service_pool.cc:224] Handling call
0709 13:13:44.084358 (+ 24766us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:44.094301 (+  9943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:44.094360 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:44.136633 (+ 42273us) log.cc:844] Serialized 26 byte log entry
0709 13:13:44.397174 (+260541us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:44.397209 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.397271 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.591702 (+194431us) raft_consensus.cc:1685] finished
0709 13:13:44.591731 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.591903 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35699,"raft.queue_time_us":1380,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":216876,"threads_started":1,"wal-append.queue_time_us":333086}
I20260709 13:13:44.607820  8058 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:44.614820 10945 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:13:44.616050 10541 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:44.627628  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:44.628389  8063 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:44.564646 10940 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:44.630676 10940 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:44.631552  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:49202 (request call id 82) took 2136 ms. Trace:
I20260709 13:13:44.631907  8054 rpcz_store.cc:276] 0709 13:13:42.495667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.606297 (+2110630us) service_pool.cc:224] Handling call
0709 13:13:44.631458 (+ 25161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:44.635808 10822 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:44.636610  8058 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:44.637190  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1239) took 2139 ms. Trace:
I20260709 13:13:44.637521  8058 rpcz_store.cc:276] 0709 13:13:42.498150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.636040 (+2137890us) service_pool.cc:224] Handling call
0709 13:13:44.636397 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.636986 (+   589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.637143 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:44.639115  8058 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:13:44.639714  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1243) took 2139 ms. Trace:
I20260709 13:13:44.640447  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2441) took 1157 ms. Trace:
I20260709 13:13:44.640828  8203 rpcz_store.cc:276] 0709 13:13:43.483237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.285941 (+802704us) service_pool.cc:224] Handling call
0709 13:13:44.640248 (+354307us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b54000ba9d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:44.640391 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26253568}
I20260709 13:13:44.568923 10426 consensus_queue.cc:1059] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:13:44.642843 10426 consensus_queue.cc:1243] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:13:44.614204  8056 consensus_meta.cc:287] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.788s	user 0.009s	sys 0.001s
W20260709 13:13:44.647274  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 91375.3us
I20260709 13:13:44.647575  8056 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:44.648356  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2347) took 1045 ms. Trace:
I20260709 13:13:44.648711  8209 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:44.592360  7956 catalog_manager.cc:5697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:44.592608 10905 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:44.593915  8693 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:13:44.571583  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 61749us
I20260709 13:13:44.595320  8055 rpcz_store.cc:276] 0709 13:13:42.272371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.578102 (+2305731us) service_pool.cc:224] Handling call
0709 13:13:44.588879 (+ 10777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.594504 (+  5625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.594697 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:44.471701 10740 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:44.648782  8056 rpcz_store.cc:276] 0709 13:13:43.603166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.824588 (+221422us) service_pool.cc:224] Handling call
0709 13:13:44.648301 (+823713us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:44.640038  8058 rpcz_store.cc:276] 0709 13:13:42.500658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.638414 (+2137756us) service_pool.cc:224] Handling call
0709 13:13:44.638724 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.639476 (+   752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.639660 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:44.660187  7956 catalog_manager.cc:5697] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:44.662002  8063 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:44.663586  8063 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:44.664338  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 299) took 1059 ms. Trace:
I20260709 13:13:44.664775  8063 rpcz_store.cc:276] 0709 13:13:43.604677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.663005 (+1058328us) service_pool.cc:224] Handling call
0709 13:13:44.663373 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.664082 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.664279 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:44.666826  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 72972us
I20260709 13:13:44.647970  8104 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:44.685846  8055 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:13:44.648479 10888 raft_consensus.cc:3055] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:44.683462 10369 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:44.690918  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:44.692414  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2343) took 1862 ms. Trace:
I20260709 13:13:44.692764  8055 rpcz_store.cc:276] 0709 13:13:42.824120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.651904 (+1827784us) service_pool.cc:224] Handling call
0709 13:13:44.678228 (+ 26324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.686310 (+  8082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.686571 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:44.696506  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.690537  7956 catalog_manager.cc:5697] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 3 to 5, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 5 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
W20260709 13:13:44.697049  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:44.698191  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 50286us
I20260709 13:13:44.683684 10944 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:44.683934  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:44.705678  8055 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:13:44.686985 10689 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:44.693882 10369 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.710788  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 302) took 1105 ms. Trace:
W20260709 13:13:44.711804  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 103931us
I20260709 13:13:44.692994  8129 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:44.711730  8055 rpcz_store.cc:276] 0709 13:13:43.605509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.693636 (+1088127us) service_pool.cc:224] Handling call
0709 13:13:44.704444 (+ 10808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.706194 (+  1750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.710720 (+  4526us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:44.712647  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:44.689287 10888 raft_consensus.cc:740] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:44.709570  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:44.713960  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 260192us
W20260709 13:13:44.717567  8730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 484) took 1443 ms (client timeout 1720 ms). Trace:
W20260709 13:13:44.733624  8730 rpcz_store.cc:269] 0709 13:13:43.274242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.276057 (+  1815us) service_pool.cc:224] Handling call
0709 13:13:44.717425 (+1441368us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b540017a054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:44.717518 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256230016}
W20260709 13:13:44.792687  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 80114.3us
W20260709 13:13:44.735387  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 44548.2us
I20260709 13:13:44.797610  7956 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 3 to 4, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 4 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:44.805536  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:13:44.779657  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:44.797235  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 61669us
I20260709 13:13:44.713871 10369 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:44.734899  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:44.765512  8055 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:13:44.702893 10933 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:13:44.767122  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 63178.9us
I20260709 13:13:44.811753  8101 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:13:44.812639  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 81172.8us
W20260709 13:13:44.816087  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.719141 10898 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:44.818281  8104 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:44.717022 10547 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:44.817623  8027 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:44.764995 10541 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:44.766921  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:44.735219 10944 raft_consensus.cc:697] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:44.767370 10682 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:44.807729 10822 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:44.832384  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 320494us
I20260709 13:13:44.831919  8135 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:44.831914  8693 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:44.818789 10948 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:44.825528 10511 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:44.840158  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.847901  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:44.827065 10898 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:44.827395 10905 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:44.849114  8101 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:44.853092  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.854988  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:44.766685  7945 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a003b4584d3747a381522d67299aaed2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 84d775d1767e4cc4aa7b9c627d8f0fb8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:13:44.856786  7945 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 171109us
I20260709 13:13:44.857638  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:44.837627 10682 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:13:44.859963  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.861003  8204 raft_consensus.cc:2410] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:13:44.861321  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:44.862083  8207 raft_consensus.cc:2393] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:44.845587 10730 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:44.848783 10948 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.864881 10944 consensus_queue.cc:237] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.848301  8724 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:44.870332 10740 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:44.854166 10898 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:44.851755 10905 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.860472  9980 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:44.872080 10805 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:44.872680  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:44.874431 10689 consensus_queue.cc:1059] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:44.863765  8104 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:44.882109  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 64170.7us
I20260709 13:13:44.848124 10511 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.890893  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:44.840088 10945 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:13:44.895982 10511 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:44.861910 10674 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:44.896227  8730 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 4.
I20260709 13:13:44.898705  8130 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:44.901237  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:44.903208  8136 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:44.904136  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2335) took 2806 ms. Trace:
I20260709 13:13:44.880381 10682 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:44.904524  8062 rpcz_store.cc:276] 0709 13:13:42.098228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.073778 (+1975550us) service_pool.cc:224] Handling call
0709 13:13:44.083341 (+  9563us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:13:44.445241 (+361900us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:44.445271 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:13:44.464012 (+ 18741us) log.cc:844] Serialized 78 byte log entry
0709 13:13:44.777665 (+313653us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:44.777698 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.777756 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.903922 (+126166us) raft_consensus.cc:1685] finished
0709 13:13:44.903940 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.904085 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":42240,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":32520,"thread_start_us":531318,"threads_started":2,"wal-append.queue_time_us":339775}
W20260709 13:13:44.921288  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 60038.2us
I20260709 13:13:44.848647 10547 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:44.825673 10888 consensus_queue.cc:260] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:44.906101  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2346) took 1303 ms. Trace:
I20260709 13:13:44.884932 10906 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:44.929016  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:44.922557  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58902.8us
I20260709 13:13:44.925210  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2451) took 1435 ms. Trace:
I20260709 13:13:44.893383 10905 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:44.893148  9980 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.930574  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:44.927727  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:44.927074  8058 rpcz_store.cc:276] 0709 13:13:43.602886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.652785 (+1049899us) service_pool.cc:224] Handling call
0709 13:13:44.653184 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:44.688552 (+ 35368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:44.688599 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:44.711565 (+ 22966us) log.cc:844] Serialized 26 byte log entry
0709 13:13:44.882075 (+170510us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:44.882115 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.882180 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:44.905920 (+ 23740us) raft_consensus.cc:1685] finished
0709 13:13:44.905942 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.906053 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15648,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":5296,"thread_start_us":89646,"threads_started":1,"wal-append.queue_time_us":131928}
I20260709 13:13:44.929697  8209 rpcz_store.cc:276] 0709 13:13:43.486308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.177721 (+691413us) service_pool.cc:224] Handling call
0709 13:13:44.916672 (+738951us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b5400189554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:44.922020 (+  5348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":559580544}
I20260709 13:13:44.905705 10541 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:44.885154 10948 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:44.922091 10701 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:44.909281 10674 raft_consensus.cc:697] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:44.934089  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:44.934675  8058 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:13:44.960274  8138 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:44.936010  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:44.936184  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:44.936973  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:44.933246  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:44.923029 10942 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.933300  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:44.922191 10956 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.934198  9980 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:44.968746  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 96145.9us
W20260709 13:13:44.972193  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 48076.6us
I20260709 13:13:44.973187  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:44.933986 10740 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:44.942050 10888 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:44.936084 10834 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:44.978695  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:44.936975 10906 raft_consensus.cc:697] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:44.980232  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:44.976281 10950 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:13:44.974989 10942 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.969918 10623 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:13:44.985621 10674 consensus_queue.cc:237] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.987027 10805 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:44.963060 10541 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:44.979532  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:44.998840  8138 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:13:44.970505 10701 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.003942 10966 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:45.004180  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:45.013358  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:45.013923  8138 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:45.014024  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:45.007858  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:45.014842  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:45.009677  8026 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:45.007323 10884 consensus_meta.cc:287] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.000s
I20260709 13:13:45.012280  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:45.022434  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.006597  8723 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:13:45.016718  8727 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:45.010519  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:45.017498  8138 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:13:45.023420  8728 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:13:45.024027  8131 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:13:44.998174 10834 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:44.970311 10956 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:45.034765  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:45.012555 10511 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:44.999914 10471 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:13:45.039110  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:45.040666  8026 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:45.013300 10966 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:45.061179  8135 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:44.989702 10740 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.028366 10834 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:45.076293  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 64053.1us
I20260709 13:13:45.077557  8029 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:45.034574 10623 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:45.082865  8131 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:13:45.082916  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2336) took 2982 ms. Trace:
I20260709 13:13:45.083881  8059 rpcz_store.cc:276] 0709 13:13:42.100877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.161725 (+2060848us) service_pool.cc:224] Handling call
0709 13:13:44.162095 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:44.178923 (+ 16828us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:44.178953 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:44.492725 (+313772us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b2c00037918. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db990fa11 00007f6db991f372 00007f6dbb051a01 00007f6dbb01643a 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2
0709 13:13:44.493254 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:13:44.693499 (+200245us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:44.693531 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.884512 (+190981us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b540021a054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:44.884562 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:45.082732 (+198170us) raft_consensus.cc:1685] finished
0709 13:13:45.082751 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.082864 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":342,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":66,"spinlock_wait_cycles":40957696,"thread_start_us":199796,"threads_started":1,"wal-append.queue_time_us":225644}
I20260709 13:13:45.037611 10805 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:45.091598  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:45.093284 10964 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:45.093876 10964 raft_consensus.cc:697] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:45.094833 10964 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:45.096697  8730 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
W20260709 13:13:45.097514  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 56947.5us
I20260709 13:13:45.098644  8029 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:45.026929 10884 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.034281 10907 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.162s
I20260709 13:13:45.063920 10757 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:45.076584 10626 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:45.085829 10471 consensus_queue.cc:1243] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:13:45.109724  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:45.112130  8723 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: [3.6-7.7]
I20260709 13:13:45.084017 10541 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:45.113394  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:45.117707 10907 consensus_queue.cc:1243] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:13:45.122289  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:45.123003  8131 raft_consensus.cc:2368] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
W20260709 13:13:45.125003  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 55734.5us
I20260709 13:13:45.125767  8104 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:45.112496 10623 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:45.115228 10547 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:45.117957 10541 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:45.131716 10907 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:45.138046  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:45.147595  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:45.114696 10906 consensus_queue.cc:237] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.154575  8726 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:13:45.156280 10962 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
W20260709 13:13:45.157136 10961 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:13:45.158466  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 300) took 1553 ms. Trace:
I20260709 13:13:45.158788  8057 rpcz_store.cc:276] 0709 13:13:43.604944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.679496 (+1074552us) service_pool.cc:224] Handling call
0709 13:13:44.695910 (+ 16414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:44.705514 (+  9604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:44.705549 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:44.719540 (+ 13991us) log.cc:844] Serialized 26 byte log entry
0709 13:13:44.891396 (+171856us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:44.894019 (+  2623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.894096 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:45.158262 (+264166us) raft_consensus.cc:1685] finished
0709 13:13:45.158277 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.158406 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7841,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":22519,"thread_start_us":80057,"threads_started":1,"wal-append.queue_time_us":183388}
I20260709 13:13:44.344566  8061 rpcz_store.cc:276] 0709 13:13:42.072866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.829895 (+1757029us) service_pool.cc:224] Handling call
0709 13:13:43.830284 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:43.853404 (+ 23120us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:44.057035 (+203631us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.335065 (+278030us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:44.335137 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:44.335205 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:44.335295 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:44.335434 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4541,"prepare.queue_time_us":248143,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":17989,"thread_start_us":422020,"threads_started":2}
I20260709 13:13:45.162657 10701 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:45.165639 10940 consensus_meta.cc:287] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.535s	user 0.003s	sys 0.007s
I20260709 13:13:45.166258 10939 raft_consensus.cc:3055] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:45.166658 10939 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:45.137940 10897 raft_consensus.cc:2764] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:13:45.148288  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:45.131340 10805 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:13:45.177094 10975 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:45.177944 10942 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:45.178323  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.151098 10907 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.167444 10939 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:45.145762 10541 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:45.198282  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:45.198850  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 100287us
I20260709 13:13:45.201066  8693 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:45.203060  8102 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:45.207126 10740 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:45.208127 10967 log.cc:927] Time spent T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:13:45.209369 10893 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:45.210040 10822 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:45.217314 10511 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:45.219285 10956 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:45.221258 10966 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.222707  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:45.223394  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:45.223614 10884 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:45.224061  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:45.219296 10843 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:45.230567  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.230952 10907 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:45.166031 10940 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.232380  8131 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:45.235921 10821 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:45.236244  8029 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:45.234771  8726 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
W20260709 13:13:45.250661  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 47673.3us
I20260709 13:13:45.251747  8138 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:45.234110  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:45.245980 10929 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:45.254611  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:45.270716  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:45.271582  8138 raft_consensus.cc:2410] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:13:45.272289  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:45.273838  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:45.270318  8721 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:45.278424  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:45.279112  8209 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:45.278493 10976 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:45.277666 10905 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.278445 10982 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.282122  8058 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:13:45.313599  8728 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:45.283135 10945 raft_consensus.cc:3055] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:45.283334  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:45.315478  8061 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:45.285079 10975 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:45.286731  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:45.287528 10940 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:45.287652 10799 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.291960  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:45.293689 10939 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:45.294003 10966 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:45.295161  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:45.335523  8138 raft_consensus.cc:2368] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 6. Current term is 7.
I20260709 13:13:45.287559  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2331) took 3217 ms. Trace:
I20260709 13:13:45.333986  8029 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:45.231002  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 135495us
W20260709 13:13:45.306557  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 208381us
W20260709 13:13:45.307490  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.282716  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:45.337754  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.315356  8135 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 5.
I20260709 13:13:45.285833 10973 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:45.340406  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:45.317934  8027 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:45.315037 10945 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:45.318730  8696 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:45.322386  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:45.323092  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:45.324335  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:45.344496  8057 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:45.280035 10821 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:45.321153  8059 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:45.342731  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:45.336197  8060 rpcz_store.cc:276] 0709 13:13:42.070472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.601140 (+1530668us) service_pool.cc:224] Handling call
0709 13:13:43.601649 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:43.647808 (+ 46159us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:43.858832 (+211024us) log.cc:844] Serialized 14 byte log entry
0709 13:13:44.142296 (+283464us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:45.158229 (+1015933us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b2c00037918. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db990fa11 00007f6db991f372 00007f6dbb051a01 00007f6dbb01643a 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2
0709 13:13:45.158584 (+   355us) log.cc:844] Serialized 26 byte log entry
0709 13:13:45.158717 (+   133us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:45.158928 (+   211us) log.cc:844] Serialized 14 byte log entry
0709 13:13:45.169643 (+ 10715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.287281 (+117638us) spinlock_profiling.cc:243] Waited 45.7 ms on lock 0x7b5400052154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:45.287339 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:45.287379 (+    40us) raft_consensus.cc:1685] finished
0709 13:13:45.287392 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.287508 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83079,"prepare.queue_time_us":434043,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":76,"raft.queue_time_us":76852,"raft.run_cpu_time_us":401,"raft.run_wall_time_us":26268,"spinlock_wait_cycles":109356160,"thread_start_us":396240,"threads_started":2,"wal-append.queue_time_us":345165}
I20260709 13:13:45.351886  8206 raft_consensus.cc:2393] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:45.355204  8026 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:45.357343  8208 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:45.357722  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:45.359176  9980 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:45.360039 10976 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:45.360203  8721 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
W20260709 13:13:45.362556  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 695071us
I20260709 13:13:45.362428 10948 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.363281  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:45.363499 10898 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.341737 10940 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:13:45.367252  8175 connection.cc:256] RPC call timeout handler was delayed by -2.22097485900008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24580000000000002s for a response
I20260709 13:13:45.370780  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:45.375931  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:45.381134  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:45.378993  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 253288us
I20260709 13:13:45.354648 10893 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:45.384897  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:45.407686  8176 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:45.385175 10730 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:45.387120 10898 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:45.383126 10940 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.388497  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:45.421471  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:45.389873  8175 connection.cc:256] RPC call timeout handler was delayed by -2.1446895940000559s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21640000000000004s for a response
I20260709 13:13:45.356118 10521 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:45.390152 10945 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:45.378741 10907 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.392184 10905 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:45.444852 10907 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:45.445163 10682 consensus_meta.cc:287] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.514s	user 0.002s	sys 0.005s
I20260709 13:13:45.347658 10977 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:45.445763 10977 raft_consensus.cc:697] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:45.445595 10682 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.446985  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:45.446777 10977 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.378877 10972 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:45.398597 10821 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.450448 10972 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.450850 10972 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:45.451910  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:45.401858  8727 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:45.446733  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:45.453347  8210 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:45.455039  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:45.315181 10982 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:45.410786  8061 raft_consensus.cc:2410] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:13:45.422171  8175 connection.cc:256] RPC call timeout handler was delayed by -1.8290532959999837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1543s for a response
W20260709 13:13:45.458750  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7704050859999825s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
W20260709 13:13:45.459126  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7586651809999694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
W20260709 13:13:45.460000  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7425050859999374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1542s for a response
W20260709 13:13:45.460387  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6841532959999768s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
W20260709 13:13:45.460664  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6471050860000105s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1648s for a response
W20260709 13:13:45.460927  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6246050859999741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1673s for a response
W20260709 13:13:45.461196  8175 connection.cc:256] RPC call timeout handler was delayed by -1.5826176849998319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19340000000000002s for a response
W20260709 13:13:45.461452  8175 connection.cc:256] RPC call timeout handler was delayed by -1.5788532960000339s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1821s for a response
W20260709 13:13:45.461695  8175 connection.cc:256] RPC call timeout handler was delayed by -1.535653295999964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1869s for a response
W20260709 13:13:45.461941  8175 connection.cc:256] RPC call timeout handler was delayed by -1.5068532959999175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19010000000000002s for a response
I20260709 13:13:45.459357 10931 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:45.459116 10674 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:13:45.464479 10667 raft_consensus.cc:3055] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:45.464840 10667 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:45.456305 10973 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:13:45.465641 10667 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:45.407106 10369 consensus_meta.cc:287] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.576s	user 0.006s	sys 0.003s
W20260709 13:13:45.423107  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 187696us
I20260709 13:13:45.423741  8060 raft_consensus.cc:2368] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 6. Current term is 7.
I20260709 13:13:45.427876 10521 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:45.427945 10471 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:13:45.428601  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 48787.7us
I20260709 13:13:45.423153 10940 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:45.430402 10541 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:45.431066 10626 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.438073  8725 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:45.438921 10948 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:45.440030  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:45.412317 10976 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:45.383723 10834 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.419253 10888 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:45.457584  8130 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:13:45.460938 10511 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:13:45.466457  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:45.512733  8063 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:13:45.467760 10988 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:13:45.468696 10982 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:45.515533  8027 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:45.469969  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.516050  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:45.442274  8724 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:13:45.470203  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:45.462190  8175 connection.cc:256] RPC call timeout handler was delayed by -1.5048763689999305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3252s for a response
I20260709 13:13:45.470881  8729 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:45.471158 10907 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.532259 10907 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.473759  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:45.533915  8728 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:45.534246  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:45.534888  8208 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:45.476176 10972 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:45.419770  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2350) took 1179 ms. Trace:
I20260709 13:13:45.480981  8210 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:45.478044  8730 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:45.483698  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:45.487517  8212 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:13:45.490617  8061 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:13:45.492496  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:45.498647  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:45.503499  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:45.506022  8203 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:45.509426  8723 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:13:45.468876  9980 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:45.490327 10978 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:13:45.524194  8175 connection.cc:256] RPC call timeout handler was delayed by -1.4137511459998677s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33590000000000003s for a response
I20260709 13:13:45.531472  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:45.501344 10521 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:45.508095 10626 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:45.498011 10369 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:45.935788  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 396161us
W20260709 13:13:45.611932 10990 log.cc:927] Time spent T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
W20260709 13:13:45.602977  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.943125  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:45.905957 10674 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.944722  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:45.936652  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.944805 10674 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:45.945586  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.945859  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:45.946286 10948 consensus_queue.cc:1059] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:45.947119  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:45.947721  8203 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:45.948586  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:45.631438 10893 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.335s
I20260709 13:13:45.949421  8203 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:45.512246  8721 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:45.950465  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:45.951122  8210 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:45.952045  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:45.952674  8210 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:13:45.953905 10626 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:45.954424 10521 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:45.954936  8722 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 6. Current term is 7. Ops: [6.6-6.6]
W20260709 13:13:45.955343  8175 connection.cc:256] RPC call timeout handler was delayed by -1.4044176849999985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21320000000000003s for a response
W20260709 13:13:46.138579  8175 connection.cc:256] RPC call timeout handler was delayed by -1.3573050859999967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.197s for a response
W20260709 13:13:46.139022  8175 connection.cc:256] RPC call timeout handler was delayed by -1.3336021099999016s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
W20260709 13:13:46.139376  8175 connection.cc:256] RPC call timeout handler was delayed by -1.1707021099998656s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21450000000000002s for a response
I20260709 13:13:45.958220 10369 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:46.141705 10915 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.030s	sys 0.044s
I20260709 13:13:45.631929  8728 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:45.613480  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:45.724732 10834 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:45.938097 10682 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:45.535935  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:45.942044  8209 raft_consensus.cc:2393] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:45.938717  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:46.142742  8208 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
W20260709 13:13:46.143234  8205 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.001s	sys 0.010s
I20260709 13:13:46.145560  8056 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:46.146013  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:46.146498  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:45.958899  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:46.147202  8137 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:46.147847  8729 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 13:13:46.148253  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.142829  8211 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.024s	sys 0.051s
W20260709 13:13:46.149077  8211 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.024s	sys 0.051s
I20260709 13:13:46.149187  8722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 479) took 1269 ms. Trace:
I20260709 13:13:46.149583  8722 rpcz_store.cc:276] 0709 13:13:44.880085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.882594 (+  2509us) service_pool.cc:224] Handling call
0709 13:13:44.929132 (+ 46538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:46.136741 (+1207609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.148962 (+ 12221us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400730854. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:46.149127 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2353097472}
W20260709 13:13:46.142104 10915 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.030s	sys 0.044s
I20260709 13:13:46.151245  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:46.151844  8209 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:46.152627  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:46.155515 10944 raft_consensus.cc:3055] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:46.155956 10944 raft_consensus.cc:740] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:45.943814  8212 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:46.151321 11007 consensus_queue.cc:1059] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:13:46.141892  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:46.160763 10944 consensus_queue.cc:260] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:46.162123 10944 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:46.164835  8212 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:46.165812  8207 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:46.147149  8056 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:45.936322 10945 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:45.960058  8061 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:46.170368  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:46.171224  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:46.171514  8693 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:45.960963  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:45.630211 10973 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.955027  8055 rpcz_store.cc:276] 0709 13:13:44.240260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.910606 (+670346us) service_pool.cc:224] Handling call
0709 13:13:44.939382 (+ 28776us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.965872 (+ 26490us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:45.150261 (+184389us) log.cc:844] Serialized 14 byte log entry
0709 13:13:45.419057 (+268796us) log.cc:844] Serialized 14 byte log entry
0709 13:13:45.419358 (+   301us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:45.419419 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:45.419480 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:45.419572 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:45.419700 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30884,"prepare.queue_time_us":109221,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":70,"raft.queue_time_us":7140,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":9601,"thread_start_us":224848,"threads_started":2,"wal-append.queue_time_us":108850}
I20260709 13:13:46.172978  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:46.172574 10973 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:46.173513  8055 raft_consensus.cc:2376] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7. Re-sending same reply.
I20260709 13:13:46.174396  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:46.175344  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:46.176187 10973 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:45.523239 10888 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:46.177053 10843 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:46.177928  8138 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:45.541100 10933 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.430s
I20260709 13:13:46.178304 10843 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:46.177604 10888 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:46.179059 10843 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:46.179409 10843 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.179724 10843 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:45.939146 10907 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:45.948609 10674 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:45.966858  8062 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:45.968158 10972 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:45.969215  8210 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
W20260709 13:13:45.975526  8026 leader_election.cc:336] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.578s (SENT)
I20260709 13:13:46.197418  8026 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:45.611932 10993 log.cc:927] Time spent T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 13:13:45.984020 10995 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:45.982324 10730 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:45.988128 10547 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:45.994581  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.974586 10977 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:46.002707 10471 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:46.010849  8696 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:46.216480  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 205736us
I20260709 13:13:46.217784  8696 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:46.033196 10978 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:13:46.056064 10940 consensus_meta.cc:287] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.000s
I20260709 13:13:46.090320  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:46.116334  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 600881us
I20260709 13:13:46.117362  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 358) took 1211 ms. Trace:
I20260709 13:13:46.119012  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:46.120141  8134 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:46.121989 11009 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.124702 10667 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:46.125828  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 356) took 1225 ms. Trace:
I20260709 13:13:46.269500  8130 rpcz_store.cc:276] 0709 13:13:44.900862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.900996 (+   134us) service_pool.cc:224] Handling call
0709 13:13:46.125647 (+1224651us) spinlock_profiling.cc:243] Waited 556 ms on lock 0x7b54000d20d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:46.125764 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1278145792}
I20260709 13:13:46.128417  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:46.129828  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:46.269520  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 81127.4us
I20260709 13:13:46.270913  8063 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:46.271529  8104 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:46.145731  8205 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.001s	sys 0.010s
I20260709 13:13:46.146595  8208 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:46.156018  8131 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:45.813681 10541 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:46.273921  8029 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:46.168423  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:46.169466  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:46.171788  8061 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:46.176291  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:46.275157  8056 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:46.275733  8055 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:45.610811  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:46.276724  8206 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:46.261731  8136 rpcz_store.cc:276] 0709 13:13:44.905553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.905672 (+   119us) service_pool.cc:224] Handling call
0709 13:13:44.923363 (+ 17691us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:44.959583 (+ 36220us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:44.976752 (+ 17169us) log.cc:844] Serialized 14 byte log entry
0709 13:13:45.203564 (+226812us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:45.203608 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:45.203651 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.117075 (+913424us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b540035d9d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:46.117131 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.117272 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20314,"prepare.queue_time_us":18854,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":20679,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":36449536,"thread_start_us":195622,"threads_started":1,"wal-append.queue_time_us":202517,"wal-append.run_cpu_time_us":1233,"wal-append.run_wall_time_us":1003591}
W20260709 13:13:45.612287 10989 log.cc:927] Time spent T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
W20260709 13:13:46.181300  8102 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.049s	sys 0.122s
I20260709 13:13:45.514204 10976 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:46.182435  8207 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
W20260709 13:13:46.281360  8102 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.049s	sys 0.122s
I20260709 13:13:45.809842 10799 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:46.195250  8137 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:46.286748  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:46.287227  8722 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:46.287408  8208 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:46.225119 10471 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:46.197867  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:46.199731 10834 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:13:46.200975  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 225477us
I20260709 13:13:46.208604 10995 raft_consensus.cc:697] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:46.208880  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 67123.3us
I20260709 13:13:46.303469  9980 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:46.303776  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:46.261173  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:46.303862 10995 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:46.304760 10973 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:46.305142 10973 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.305478 10973 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:46.310299  8054 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:46.277499  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:46.311033  8062 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:46.311661  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:46.312124  8205 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:46.280385  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:46.260349  8060 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:46.194199  8138 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:46.314781  8728 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
W20260709 13:13:46.281720  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 656544us
I20260709 13:13:46.292572  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:46.258509 10893 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:46.292654  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:46.326157  8056 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:46.262490 11006 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:46.316520  8104 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:46.279397 10972 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:46.301478  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:46.330782  8696 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:46.330942  8210 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
W20260709 13:13:46.262159  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.281023 10757 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:13:46.335975  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:46.314009  8026 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:46.317453  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:46.319123  8062 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:46.320134  8205 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:46.323724  8061 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:46.278982 10369 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.319888  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:46.235069 11007 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:46.329264  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:46.299144 10948 raft_consensus.cc:3055] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:46.220779 10730 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:46.340751  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:46.339872  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2358) took 1703 ms. Trace:
I20260709 13:13:46.367416  8054 rpcz_store.cc:276] 0709 13:13:44.626732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:44.925414 (+298682us) service_pool.cc:224] Handling call
0709 13:13:46.330016 (+1404602us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540025f8d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:46.330113 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3099433216}
I20260709 13:13:46.225353 10547 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:46.205879 10907 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:46.268904 11009 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:46.340471 10898 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:46.390789  8212 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:46.358508  8207 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:46.333871  8027 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:46.358660  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:46.361735  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 47807.4us
I20260709 13:13:46.393253  8026 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:46.270138 10940 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:46.364950  8063 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:46.314504 10939 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.264813 10977 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:46.326995 10951 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.330116 10893 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.343770  8206 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:46.380738 10521 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:46.380431 10978 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:46.328712 10821 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:13:46.299582 10906 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:13:46.437139  8058 raft_consensus.cc:2368] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 6. Current term is 7.
I20260709 13:13:46.337513 10972 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.389384  8055 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:46.391944  8721 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:46.449913  8721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 482) took 1010 ms. Trace:
I20260709 13:13:46.450083  8057 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:46.451485  8726 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:13:46.362430 10948 raft_consensus.cc:740] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:13:46.386154 11004 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:13:46.334735 10931 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:13:46.374603 10740 raft_consensus.cc:2764] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:13:46.409729  8056 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:46.328680 11005 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:13:46.360716 10369 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:46.369542 10942 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:13:46.407251  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 73474.1us
I20260709 13:13:46.492378  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:46.365793 11020 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:46.400213 10898 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.406819 10893 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:46.438714 10907 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:46.382304 10674 raft_consensus.cc:3055] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:13:46.385174  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.401073  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 95747.8us
I20260709 13:13:46.450395  8721 rpcz_store.cc:276] 0709 13:13:45.439573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.439734 (+   161us) service_pool.cc:224] Handling call
0709 13:13:46.449828 (+1010094us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:46.455679 10972 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:46.479616  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:13:46.483367  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.494481  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:46.526454  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 46910.3us
I20260709 13:13:46.402746 10933 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:46.497306  8212 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 13:13:46.499641  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 195953us
I20260709 13:13:46.504894  8054 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:46.369695 10982 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:46.519484  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:46.529232  8729 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 6: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:46.435137 10521 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:46.503697  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:46.381847  8210 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:48498
I20260709 13:13:46.530141  8722 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:46.520826  8057 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:46.365784 10623 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
W20260709 13:13:46.522712  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.477421 10994 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:46.432426 10939 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:46.523532  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 130396us
I20260709 13:13:46.537469  8728 raft_consensus.cc:2368] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9.
I20260709 13:13:46.492465 10942 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:46.541376 10942 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:46.542584  8693 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:46.505816 10898 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:46.543730  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.545071  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:46.531800  8210 raft_consensus.cc:606] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:46.545655  8210 raft_consensus.cc:614] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:46.591197  8721 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:46.593516  8212 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:13:46.520910 10674 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:46.529510 10933 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:46.613476  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 69017.9us
I20260709 13:13:46.614184  8212 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:13:46.614665  8212 pending_rounds.cc:85] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 6
I20260709 13:13:46.615267  8026 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:46.614945  8212 pending_rounds.cc:107] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:13:46.616324  8212 raft_consensus.cc:2889] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:46.616792  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2469) took 1119 ms. Trace:
I20260709 13:13:46.524204 11022 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:46.432201 10951 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:46.620327  8026 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:46.622809  8060 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:13:46.628225  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 106838us
W20260709 13:13:46.618083  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 124632us
I20260709 13:13:46.629751  8026 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:46.630264  8027 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:46.617426  8205 rpcz_store.cc:276] 0709 13:13:45.497221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.338007 (+840786us) service_pool.cc:224] Handling call
0709 13:13:46.367564 (+ 29557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.384434 (+ 16870us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:46.433427 (+ 48993us) log.cc:844] Serialized 14 byte log entry
0709 13:13:46.616153 (+182726us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:46.616243 (+    90us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:46.616348 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.616474 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.616680 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":48909,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"thread_start_us":175051,"threads_started":1}
I20260709 13:13:46.607569 10740 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:13:46.639709  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 90128.8us
I20260709 13:13:46.641921  8212 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:46.610432 11020 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:13:46.659715  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 52060.9us
I20260709 13:13:46.660706  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:46.664625  8212 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:46.669262  8212 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:46.669940  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 313) took 1182 ms. Trace:
I20260709 13:13:46.619659 10799 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.670656  8062 rpcz_store.cc:276] 0709 13:13:45.487178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.337981 (+850803us) service_pool.cc:224] Handling call
0709 13:13:46.370360 (+ 32379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.370606 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:46.540941 (+170335us) log.cc:844] Serialized 14 byte log entry
0709 13:13:46.668508 (+127567us) log.cc:844] Serialized 14 byte log entry
0709 13:13:46.669336 (+   828us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:46.669436 (+   100us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:46.669510 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.669640 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.669834 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2402,"prepare.queue_time_us":288299,"thread_start_us":262146,"threads_started":2}
I20260709 13:13:46.620049 10933 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:46.621151 11022 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:13:46.683552  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.678419  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.619119 10757 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 13:13:46.671557 10740 raft_consensus.cc:697] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:46.684198 10905 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:46.775090 11040 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:13:46.826956  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 131244us
I20260709 13:13:46.692572 11022 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:46.695879 10907 raft_consensus.cc:697] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:46.780552 10623 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:46.647043 10948 consensus_queue.cc:260] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:46.832841 10976 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:46.817324 10730 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:46.862293  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 201699us
W20260709 13:13:46.859097 10667 consensus_meta.cc:287] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.569s	user 0.007s	sys 0.002s
I20260709 13:13:46.865154  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2470) took 1367 ms. Trace:
I20260709 13:13:46.866041  8101 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:46.692693 10799 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:46.838889 11007 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:46.848548 10547 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:46.844081  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 214422us
I20260709 13:13:46.672537 10674 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:46.842813 10905 raft_consensus.cc:697] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:46.847517 10940 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:46.880968  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 192193us
I20260709 13:13:46.729090 11005 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.136s
I20260709 13:13:46.863039 11040 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.892541  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2471) took 1394 ms. Trace:
I20260709 13:13:46.894641  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:46.865144 11049 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:46.876585  8211 rpcz_store.cc:276] 0709 13:13:45.497500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.386609 (+889109us) service_pool.cc:224] Handling call
0709 13:13:46.403550 (+ 16941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.438357 (+ 34807us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:46.499496 (+ 61139us) log.cc:844] Serialized 14 byte log entry
0709 13:13:46.864733 (+365237us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:46.864797 (+    64us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:46.864854 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.864937 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.865086 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":62586,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":46931,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":5371,"thread_start_us":364728,"threads_started":2}
W20260709 13:13:46.951081  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:46.948328  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 68135.5us
I20260709 13:13:46.904597  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 637) took 1385 ms. Trace:
W20260709 13:13:46.919250  8058 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1270) took 1597 ms (client timeout 1521 ms). Trace:
W20260709 13:13:46.975951  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:46.975003  8026 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:46.903995  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 309) took 1426 ms. Trace:
I20260709 13:13:46.976562  8726 rpcz_store.cc:276] 0709 13:13:45.519225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.519445 (+   220us) service_pool.cc:224] Handling call
0709 13:13:46.451160 (+931715us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:46.452005 (+   845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.904409 (+452404us) spinlock_profiling.cc:243] Waited 842 ms on lock 0x7b54004d2e50. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:46.904546 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1936587008}
I20260709 13:13:46.871613 11048 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:46.882829 10682 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:46.929301  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:46.980264 11043 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:46.908895 11050 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:13:47.028316 10931 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:46.907348 10730 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:47.029423  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 100397us
I20260709 13:13:47.029579 10730 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.030903  8101 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:47.031811 10931 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:47.032406 10730 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:46.917330 10843 consensus_meta.cc:287] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.737s	user 0.009s	sys 0.001s
W20260709 13:13:46.900171 10471 consensus_meta.cc:287] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.585s	user 0.008s	sys 0.000s
I20260709 13:13:47.033231 10843 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:46.893036  8207 rpcz_store.cc:276] 0709 13:13:45.498207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.391628 (+893421us) service_pool.cc:224] Handling call
0709 13:13:46.391920 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.495435 (+103515us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:46.726472 (+231037us) log.cc:844] Serialized 14 byte log entry
0709 13:13:46.892197 (+165725us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:46.892243 (+    46us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:46.892293 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:46.892366 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:46.892480 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5515,"prepare.queue_time_us":48349,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":12177,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"thread_start_us":315536,"threads_started":2}
I20260709 13:13:47.033555  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:46.981467 10942 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.035305 10843 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:46.976365  8058 rpcz_store.cc:269] 0709 13:13:45.322030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.322142 (+   112us) service_pool.cc:224] Handling call
0709 13:13:46.919068 (+1596926us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400219154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:46.919185 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2513866624}
I20260709 13:13:46.984930  8059 rpcz_store.cc:276] 0709 13:13:45.477163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.128132 (+650969us) service_pool.cc:224] Handling call
0709 13:13:46.903798 (+775666us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b5400334854. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:46.903935 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":781596416}
I20260709 13:13:46.880617 11009 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:46.978518 10945 consensus_meta.cc:287] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.791s	user 0.005s	sys 0.004s
I20260709 13:13:47.033296 10471 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:46.897703 10799 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:46.902259 10944 consensus_meta.cc:287] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.740s	user 0.008s	sys 0.002s
I20260709 13:13:46.894286 10976 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.893292 10369 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:47.035655 10942 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:46.979105 11049 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.915916 10521 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:46.917060 10907 consensus_queue.cc:237] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.062474  8203 raft_consensus.cc:2410] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:13:47.062785 10905 consensus_queue.cc:237] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:46.972142 10805 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:13:47.064232 10471 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:47.023736 10940 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.059361  8204 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:47.068017 11033 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:47.068681 10976 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:47.069510  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 181879us
I20260709 13:13:46.906916 11032 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.163s
I20260709 13:13:46.906477 11040 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.070019  9980 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:46.927591 10674 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:46.902768 10951 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:47.027791 10682 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:47.038599 11052 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:47.062117 11049 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:47.085844  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.041142 11043 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:47.116117  8104 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.001s	sys 0.011s
I20260709 13:13:47.118780  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:47.121168  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:47.121706  8210 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:13:47.121781  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:47.127308  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 93824.1us
I20260709 13:13:47.128183  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:47.129801  8101 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:47.131127  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:47.131927  8726 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:47.133385  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:47.088842  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2474) took 1575 ms. Trace:
W20260709 13:13:47.134645  8102 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.052s	sys 0.081s
W20260709 13:13:47.135089  8102 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.052s	sys 0.081s
I20260709 13:13:47.084582 10906 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:47.084609  9980 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:13:47.161291  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 275791us
W20260709 13:13:47.116828  8206 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.001s	sys 0.010s
W20260709 13:13:47.163391  8206 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.002s	sys 0.010s
I20260709 13:13:47.165351  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:47.114676 10682 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:47.088924 10973 consensus_meta.cc:287] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.783s	user 0.009s	sys 0.001s
I20260709 13:13:47.169466  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:47.170708  8102 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:13:47.116505  8104 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.001s	sys 0.011s
I20260709 13:13:47.084728 10626 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:13:47.173987  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 832267us
I20260709 13:13:47.179735  8101 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:47.182022  8210 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:47.137876 10931 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:13:47.170722 10898 consensus_meta.cc:287] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.000s
I20260709 13:13:47.137521 10511 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:47.137308 10471 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:47.189914  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.191674  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:47.201129  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 425371us
I20260709 13:13:47.159552 10906 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:47.192982  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:47.202330  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:47.083153 10940 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:47.203488  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.193419  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:46.909463 10740 consensus_queue.cc:237] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.196139  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:47.185878  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:47.204656  8210 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:47.204829  8205 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:47.205072  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:47.198489  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:47.199237  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:47.205866  8057 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:47.207506  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:47.207700  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:47.208218  8135 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:47.208529  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:47.210237  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:47.179684 11052 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:47.200693  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:47.211604  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:47.174324 11020 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:47.212159  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:47.212772  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:47.134895  8208 rpcz_store.cc:276] 0709 13:13:45.513032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.401012 (+887980us) service_pool.cc:224] Handling call
0709 13:13:46.401339 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.412413 (+ 11074us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:46.433821 (+ 21408us) log.cc:844] Serialized 14 byte log entry
0709 13:13:47.088447 (+654626us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:47.088509 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:47.088570 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.088650 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:47.088780 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48052,"prepare.queue_time_us":21971,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":118198,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":41429,"thread_start_us":627277,"threads_started":2,"wal-append.queue_time_us":475232}
I20260709 13:13:47.179078 10975 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:47.231107  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:47.232295  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:47.235939  8210 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:47.232308  8138 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:47.239338  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:47.239933  8129 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:47.233222  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:47.240988  8132 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:47.242470  8696 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:47.234778  8137 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
W20260709 13:13:47.207670 10972 consensus_meta.cc:287] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.668s	user 0.010s	sys 0.000s
I20260709 13:13:47.168744 10822 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:13:47.085378 11033 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:47.253472  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 82957us
I20260709 13:13:47.117794 11022 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.242960  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:47.253928  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:47.232364  8058 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:13:47.173667 10843 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:13:47.214426 10931 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:47.167367 10933 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.271939 10931 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.273885 10933 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:47.262760  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 146567us
I20260709 13:13:47.275627  8175 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:47.265766 11043 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.231698 11052 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:47.276926 11052 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:47.277428  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:47.278318  8136 raft_consensus.cc:2368] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8.
I20260709 13:13:47.278899 11034 consensus_queue.cc:1059] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:47.201967 10511 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:47.279968 10511 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:47.209759 10898 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.263573  8057 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:47.228727 11044 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:47.283360 11044 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:47.256860 10972 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.284067 10898 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:47.254259  9980 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.284230 11044 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.183856 10973 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.267174  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:47.270823  8138 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:47.260656 10843 consensus_queue.cc:1243] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:47.189805 10626 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:47.228577 11006 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.037s	sys 0.033s
I20260709 13:13:47.232965 10471 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:47.472327  8061 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.005s	sys 0.005s
I20260709 13:13:47.570943  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:47.571388  8061 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.007s	sys 0.005s
I20260709 13:13:47.472198  8132 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:47.572458  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2364) took 1607 ms. Trace:
W20260709 13:13:47.471653  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 240180us
I20260709 13:13:47.305549  8130 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: []
W20260709 13:13:47.389530  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 112206us
W20260709 13:13:47.472031  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 341241us
I20260709 13:13:47.253906 10547 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:47.287470  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:47.416790  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 154744us
I20260709 13:13:47.237125 10939 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:47.570417  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:47.581408  8134 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:47.572810  8061 rpcz_store.cc:276] 0709 13:13:45.965294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.338433 (+373139us) service_pool.cc:224] Handling call
0709 13:13:46.365788 (+ 27355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.365992 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:46.844880 (+478888us) log.cc:844] Serialized 14 byte log entry
0709 13:13:47.572129 (+727249us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:47.572185 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:47.572237 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.572300 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:47.572410 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19820,"prepare.queue_time_us":241637,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":39,"thread_start_us":1205415,"threads_started":2}
I20260709 13:13:47.587487  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:47.569089 11063 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.272s	user 0.000s	sys 0.000s
I20260709 13:13:47.602778  8131 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:13:47.393635 11019 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 1.096s	user 0.005s	sys 0.000s
I20260709 13:13:47.260937 10975 raft_consensus.cc:697] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:13:47.604200  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 242953us
W20260709 13:13:47.533623 10893 consensus_meta.cc:287] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 1.008s	user 0.007s	sys 0.003s
I20260709 13:13:47.591223  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:47.284276 11033 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:47.576525 11006 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.182s	user 0.056s	sys 0.038s
I20260709 13:13:47.304219 11008 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:47.289669 10945 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:47.584537  8102 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:47.324242 10757 raft_consensus.cc:3055] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:47.323748 10888 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:47.582235 10978 raft_consensus.cc:2764] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:13:47.580108  9980 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:47.629428  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:47.622584 10940 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:47.589862 11052 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:47.633747  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.635859  8135 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:47.373956 11030 consensus_queue.cc:1059] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.366s
I20260709 13:13:47.621389 10939 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.638300 10972 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:47.647473  8210 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:47.285694 10369 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:13:47.694325 11073 log.cc:927] Time spent T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:13:47.694597 10888 raft_consensus.cc:697] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 6 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:47.695624 10888 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.697242 10471 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:47.697897 11022 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:47.639727 10757 raft_consensus.cc:740] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:47.630190 10893 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.374449 10906 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.698504 10945 raft_consensus.cc:2764] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:13:47.630359 10976 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:47.631876 11006 raft_consensus.cc:697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:47.706300 10511 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.706588 10975 consensus_queue.cc:237] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:47.706717 10682 consensus_meta.cc:287] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.522s	user 0.006s	sys 0.002s
I20260709 13:13:47.734510 11006 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.734772 10682 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.735562 10511 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:47.632164 10547 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:47.737581 10682 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:47.738286 10547 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.739226 10757 consensus_queue.cc:260] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.740912 10757 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:47.637223 11049 consensus_meta.cc:287] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.000s
I20260709 13:13:47.741798 11049 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.744107 10893 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:47.744267 11049 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:47.702719 11068 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:47.374787 11020 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.745223 11068 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:47.745785 11068 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:47.748936 10906 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:47.751513  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:47.705305  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 321) took 1424 ms. Trace:
I20260709 13:13:47.752089  8063 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:47.704214 11075 log.cc:927] Time spent T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:13:47.752271  8056 rpcz_store.cc:276] 0709 13:13:46.280755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.475983 (+195228us) service_pool.cc:224] Handling call
0709 13:13:46.476332 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:46.693945 (+217613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:46.694135 (+   190us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:46.828962 (+134827us) log.cc:844] Serialized 26 byte log entry
0709 13:13:47.083182 (+254220us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:47.083235 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.083353 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:47.705095 (+621742us) raft_consensus.cc:1685] finished
0709 13:13:47.705115 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:47.705244 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91551,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":6399,"thread_start_us":377670,"threads_started":2,"wal-append.queue_time_us":332667}
I20260709 13:13:47.642985 11008 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:13:47.698732  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 109041us
I20260709 13:13:47.754930 11008 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.756045  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:47.757087 11071 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:47.757493  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:47.758422 11071 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:47.707175 10973 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:47.758792 11071 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:47.759191 11071 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:47.717691  8210 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:13:47.303791 11047 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:47.780148  8063 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
W20260709 13:13:47.781428  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 152264us
I20260709 13:13:47.702533 10369 raft_consensus.cc:697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:13:47.783754  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.709144 11052 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:47.808557  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:47.730352 10939 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.811105  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:47.813560  8211 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:13:47.813660  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:47.814642  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:47.815616  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:47.816628  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:47.801533 10905 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:47.818388 10905 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.818897  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:47.820366 10905 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:47.821987  8211 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:13:47.829187  8728 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 640) took 1382 ms (client timeout 1682 ms). Trace:
I20260709 13:13:47.833461  8104 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:47.835784  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 251326us
I20260709 13:13:47.836020  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:47.785880 11047 raft_consensus.cc:697] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:47.840467  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:47.814177  8208 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:47.829689  8728 rpcz_store.cc:269] 0709 13:13:46.446668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.447508 (+   840us) service_pool.cc:224] Handling call
0709 13:13:47.828956 (+1381448us) spinlock_profiling.cc:243] Waited 44.1 ms on lock 0x7b540041e6d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:47.829119 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101398400}
W20260709 13:13:47.846155  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.854664  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:47.856112  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:47.842394 11022 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:47.865599 11086 log.cc:927] Time spent T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:13:47.865494 11022 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.866978 10511 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:47.867520 11022 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:47.859674 11029 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:47.868717 11029 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.869222 11047 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.871568 10995 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:47.872198 10995 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.848328 10888 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:47.811193 11034 consensus_queue.cc:1059] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:13:47.879807 10369 consensus_queue.cc:237] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.880550 11029 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:47.881577 11057 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.882112 11057 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.883376 11071 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.884076 11057 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:47.885334 11071 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:47.890668  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:47.891361  8057 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 6: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:47.892234  8059 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:47.893848  8061 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:47.891387  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:47.895897  8063 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:47.898411  8026 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:47.901823  8102 leader_election.cc:400] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9.
I20260709 13:13:47.842931 11062 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.902669  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:47.902269  8102 leader_election.cc:403] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:13:47.900701  8026 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:47.852974 11078 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:47.907204  8058 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:13:47.906071  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:13:47.908155  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 45989.9us
I20260709 13:13:47.909945 10976 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:47.926309 10976 raft_consensus.cc:697] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:47.927259 10976 consensus_queue.cc:237] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:47.928738  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.929530 10888 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.930127  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:47.930044 10888 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.931258  8056 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:47.932610 10888 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:47.932696  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:47.914237  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:47.934044  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:47.934470  8204 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2459) took 2454 ms (client timeout 1545 ms). Trace:
I20260709 13:13:47.934785 10945 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:47.935176  8175 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:47.935433 10945 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:47.935884 10945 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:47.936316  8129 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: []
W20260709 13:13:47.935434 10674 consensus_meta.cc:287] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.849s	user 0.010s	sys 0.000s
I20260709 13:13:47.938642 10888 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:47.939297 11027 raft_consensus.cc:2764] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:13:47.907223 11071 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:47.918493 10972 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:47.904495 10940 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:47.854487 10682 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:47.907116 11062 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:47.924600 11028 raft_consensus.cc:3055] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:47.924465 11068 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.915611  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:47.964032  8058 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:47.951200 10843 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:47.935052 10939 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:47.958909 11071 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:47.952935  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:47.929502  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:47.991364  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:47.993016  8176 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:47.934841  8204 rpcz_store.cc:269] 0709 13:13:45.479745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.479921 (+   176us) service_pool.cc:224] Handling call
0709 13:13:47.934276 (+2454355us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400189054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:47.934386 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2539102080}
I20260709 13:13:47.957161  8208 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:47.957994 10730 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:13:47.956516 10757 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:47.951340 10945 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:47.960212 10940 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.962198 10972 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:48.011482  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:47.961542 11028 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:13:48.001987  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 49119.7us
I20260709 13:13:48.014523  8136 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:47.998219 11057 raft_consensus.cc:3055] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:47.961294 10682 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.018677  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:47.996661 11071 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:48.020287  8693 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:48.009555  8058 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:48.000733 11020 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:48.011085 10940 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:48.013281 10843 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.045650  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:48.056296  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:48.014855 10906 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:48.057206  8058 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:48.063161  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:48.063791  8058 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:48.064401  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.020231 11057 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:48.065145  8134 raft_consensus.cc:2393] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:48.042912 11020 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.046525 11074 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:13:48.067140 11024 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.459s	user 0.017s	sys 0.031s
I20260709 13:13:48.044453 11030 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:13:48.046635 11008 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:13:48.075807  8101 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:48.076663  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:48.068763 11020 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:48.059490 11025 consensus_queue.cc:1059] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
W20260709 13:13:48.079901 11024 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.547s	user 0.017s	sys 0.031s
I20260709 13:13:48.086521  8058 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:48.096030  8101 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:48.062222 10843 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.098945 11077 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:48.098629  8101 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:48.101961  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 242565us
I20260709 13:13:48.083140 11008 consensus_queue.cc:1243] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:13:48.081463 11074 raft_consensus.cc:697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:13:48.105152  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 170038us
I20260709 13:13:48.106853  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:48.097586 11025 consensus_queue.cc:1243] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:13:48.070621 11028 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.101718 10975 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:48.121308  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:48.122301 11040 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:48.123261 10995 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:48.109452 11077 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:48.141184 11077 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:48.144645  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.145248  8135 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:48.146669  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:48.146750  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.148110 10977 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:48.148233 10945 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:48.148989  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:48.125744 10757 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9.
I20260709 13:13:48.149253 10682 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:48.150763  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 320) took 1874 ms. Trace:
I20260709 13:13:48.137303 11068 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:48.151336  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:48.151912  8135 raft_consensus.cc:2376] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6. Re-sending same reply.
I20260709 13:13:48.152758  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.153383  8135 raft_consensus.cc:2376] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7. Re-sending same reply.
I20260709 13:13:48.154193  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:48.154445  8203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55942 (request call id 410) took 2255 ms (client timeout 1719 ms). Trace:
W20260709 13:13:48.154803  8203 rpcz_store.cc:269] 0709 13:13:45.898805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.960748 (+ 61943us) service_pool.cc:224] Handling call
0709 13:13:48.154234 (+2193486us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b540019e6d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:48.154370 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2046511360}
I20260709 13:13:48.155205  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:48.151130  8055 rpcz_store.cc:276] 0709 13:13:46.276764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.448843 (+172079us) service_pool.cc:224] Handling call
0709 13:13:46.449269 (+   426us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:46.459157 (+  9888us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:46.459199 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:46.518372 (+ 59173us) log.cc:844] Serialized 52 byte log entry
0709 13:13:46.920464 (+402092us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b180041f680. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:13:47.234701 (+314237us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:47.254199 (+ 19498us) log.cc:844] Serialized 14 byte log entry
0709 13:13:47.955819 (+701620us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b180041f680. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f29557 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:13:48.150034 (+194215us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b180041f680. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ea3c0a 00007f6dc15e3219 00007f6db7f2ba40 00007f6db7f264e4 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf 00007f6dbb030038
0709 13:13:48.150257 (+   223us) log.cc:844] Serialized 14 byte log entry
0709 13:13:48.150478 (+   221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:48.150551 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:48.150587 (+    36us) raft_consensus.cc:1685] finished
0709 13:13:48.150597 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:48.150713 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":872888,"prepare.queue_time_us":918983,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":3428,"raft.queue_time_us":91943,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":53609,"spinlock_wait_cycles":63152512,"thread_start_us":290786,"threads_started":1,"wal-append.queue_time_us":1048719}
I20260709 13:13:48.130882  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:48.161648  8208 raft_consensus.cc:2393] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:48.117604  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 311) took 2640 ms. Trace:
I20260709 13:13:48.133720  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:48.103474  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:48.162096  8060 rpcz_store.cc:276] 0709 13:13:45.477644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.314964 (+837320us) service_pool.cc:224] Handling call
0709 13:13:46.330193 (+ 15229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:46.629305 (+299112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.931300 (+1301995us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b540025f8d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:48.117446 (+186146us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b540025f8d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:48.117558 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":690066048}
W20260709 13:13:48.162941  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 59553us
I20260709 13:13:48.137223  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.138011  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.138121  8205 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:48.107064 11089 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:48.077754 10972 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.152863  8722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 484) took 1064 ms. Trace:
I20260709 13:13:48.175827  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:48.128453  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:48.177139  8210 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:48.172793  8726 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:48.179889 11028 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:48.182452  8210 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:48.183542  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:48.184286  8058 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:48.189564  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:48.190478  8730 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:48.175017  8722 rpcz_store.cc:276] 0709 13:13:47.102308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.115610 (+ 13302us) service_pool.cc:224] Handling call
0709 13:13:47.830576 (+714966us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:47.830943 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:47.830980 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:47.831031 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:47.831074 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.831148 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:48.152696 (+321548us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b540041e6d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:48.152802 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1167,"raft.queue_time_us":9857,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":8815,"spinlock_wait_cycles":1542040320}
I20260709 13:13:48.095305 11057 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.194906  8210 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:13:48.195832  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 62190.2us
I20260709 13:13:48.180455 10945 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:48.202558 10945 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:48.202983 10945 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:48.206689 11090 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:13:48.208287  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.208987  8138 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:48.209949  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.156046 10939 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:48.175019 11068 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:48.217298  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.214895  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.217922  8130 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:48.218070  8138 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:48.216363  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:48.222779  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.219645  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:48.230585  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:48.217132 11040 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.221894 11057 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:48.221776 10730 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.225701 11007 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:48.245018 10945 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:48.227777 10369 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:48.226210 11068 raft_consensus.cc:697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:13:48.257753  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:48.259604  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 141923us
I20260709 13:13:48.265033  8104 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:48.225615 10977 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:13:48.270666  8104 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:48.271250 11102 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:48.271983 11102 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:48.274708  8208 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:48.276374  8208 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:13:48.223778 10939 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.219956 11074 consensus_queue.cc:237] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.252401 10369 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.240207 11040 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:48.249919 10730 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:48.290489  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:48.277712 11008 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:48.281184 11089 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:48.282321 10369 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:48.315478  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.293614 11007 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:48.284143  9980 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:13:48.283843 10975 consensus_queue.cc:1059] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:13:48.319859 10730 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:48.320313 10945 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:48.323225 11071 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:48.299700 11008 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.327862  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:48.328168  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:48.305935 11089 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:13:48.338738 11095 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:13:48.353152 11089 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:48.340869 11097 log.cc:927] Time spent T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:13:48.306231 11068 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:48.343139 11093 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.245s	user 0.000s	sys 0.004s
I20260709 13:13:48.344717 11040 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:48.293862 11034 consensus_queue.cc:1059] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:13:48.293923 11088 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
W20260709 13:13:48.368521  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.373219 10945 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:48.377274 11089 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.380499 11020 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.398197  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:48.399408  8726 raft_consensus.cc:2393] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
W20260709 13:13:48.385013  8131 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1462) took 1191 ms (client timeout 1525 ms). Trace:
I20260709 13:13:48.394757  8210 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
W20260709 13:13:48.400228  8131 rpcz_store.cc:269] 0709 13:13:47.193408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.193517 (+   109us) service_pool.cc:224] Handling call
0709 13:13:48.384814 (+1191297us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b540036ea54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:48.384956 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":851985664}
I20260709 13:13:48.403530  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:48.361312 11008 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:48.411329  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.387036 10843 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:48.423375  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:48.424597  8728 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:48.431929  8725 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:48.471915  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:48.471938  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:48.472847  8721 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:48.479940 11111 log.cc:927] Time spent T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 13:13:48.482208 11118 log.cc:927] Time spent T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
W20260709 13:13:48.483000 11100 log.cc:927] Time spent T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:13:48.485394 11071 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:48.492636  8726 raft_consensus.cc:2410] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:13:48.458968 10940 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:48.506187  8061 raft_consensus.cc:2393] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:48.520764 11020 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:48.520891  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 193108us
I20260709 13:13:48.521652  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:48.522719 11126 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:48.522816 10940 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:48.523387 11126 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:48.523494 11089 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:48.524312 11126 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:48.525461 11020 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:48.526361  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:48.526973  8204 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:48.528570  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:48.531580 10369 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.532987  8728 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:13:48.533161  8175 leader_election.cc:400] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 6. Current term is 7.
I20260709 13:13:48.533454  8175 leader_election.cc:403] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:48.532191 11020 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:48.535153  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:48.535456 11071 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 6. Current term is 7.
I20260709 13:13:48.535861  8131 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:48.536790  8175 leader_election.cc:400] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 6. Current term is 7.
I20260709 13:13:48.537110  8175 leader_election.cc:403] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:48.538378  8130 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:48.538947 11020 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 6. Current term is 7.
I20260709 13:13:48.540057 10740 consensus_queue.cc:1059] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:48.540838  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.541137 11057 raft_consensus.cc:3055] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:48.541515 11057 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:48.542519 11057 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.543481 11057 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:48.543541  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:48.544528  8131 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:48.545652  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:48.546739 10821 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:48.547204 10821 raft_consensus.cc:697] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:48.547724 11008 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:48.548105 10821 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.550172 11008 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:48.550311  8138 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:13:48.551622  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:48.558646  8721 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:13:48.560675 11102 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:48.534183 10369 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:48.571130 10939 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:48.573985 10626 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:48.575462 10905 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.576026 10905 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.579767 10948 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:48.580411  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.581266  8136 raft_consensus.cc:2410] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:13:48.582490 10905 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:48.585124  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:48.585965  8136 raft_consensus.cc:2410] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:13:48.587815  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 427868us
I20260709 13:13:48.588228  8136 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:48.592149  8102 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:48.592885  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:48.595404  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.599382  8138 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:48.599931  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:48.600440  8138 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:48.601404 11044 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:48.597272  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:48.601773  8102 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:48.602232  8131 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:48.602545 11089 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:48.602983 11089 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:48.603344 11089 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:48.604010  8729 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:48.605432  8723 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:13:48.601876 11044 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:48.606503  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:48.607399 11044 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.611692  8721 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:48.614073 11089 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.616094 11089 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:48.618101  8724 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:13:48.620326  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:48.623698 11126 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.624257 11126 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.626636 11126 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:48.627207  8175 connection.cc:256] RPC call timeout handler was delayed by -2.2106797080002707s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1502s for a response
W20260709 13:13:48.627647  8175 connection.cc:256] RPC call timeout handler was delayed by -2.193579708000243s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
I20260709 13:13:48.612814 11052 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.629846 11052 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:48.631302  8209 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2478) took 2659 ms (client timeout 1682 ms). Trace:
W20260709 13:13:48.631669  8209 rpcz_store.cc:269] 0709 13:13:45.972206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.152416 (+180210us) service_pool.cc:224] Handling call
0709 13:13:48.631128 (+2478712us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002beb54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:48.631240 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2407447424}
I20260709 13:13:48.636965  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:48.640106 11023 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:48.640362  8102 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:48.641434 11023 consensus_queue.cc:1059] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:48.642230 10757 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:48.643102 10757 raft_consensus.cc:3055] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:48.643478 10757 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:48.644225 10757 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.645259 10757 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:48.642282 11023 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:48.649101 11023 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:48.649995 11023 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.653591  8208 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:48.656147 11008 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:48.659070  8055 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:13:48.666996 11070 log.cc:927] Time spent T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 1.395s	user 0.003s	sys 0.000s
I20260709 13:13:48.668706  8204 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:13:48.669252  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2475) took 3148 ms. Trace:
I20260709 13:13:48.669623  8206 rpcz_store.cc:276] 0709 13:13:45.520859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.404929 (+884070us) service_pool.cc:224] Handling call
0709 13:13:46.437105 (+ 32176us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:46.458184 (+ 21079us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:46.458231 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:46.496436 (+ 38205us) log.cc:844] Serialized 26 byte log entry
0709 13:13:47.163884 (+667448us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:47.163939 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:47.164038 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:48.666467 (+1502429us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:48.668963 (+  2496us) raft_consensus.cc:1685] finished
0709 13:13:48.668986 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:48.669197 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":38806,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":7249,"spinlock_wait_cycles":31266176,"thread_start_us":552509,"threads_started":1,"wal-append.queue_time_us":720245}
I20260709 13:13:48.672441 11010 raft_consensus.cc:3055] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:48.627952 10973 raft_consensus.cc:3055] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:48.678647 10973 raft_consensus.cc:740] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:48.679463 10973 consensus_queue.cc:260] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.681082 10973 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:48.672848 11010 raft_consensus.cc:740] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:48.683154 11010 consensus_queue.cc:260] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:48.687474  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:48.688743  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:48.690196 11010 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:48.695425  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:48.696125  8055 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:48.700215  8055 pending_rounds.cc:85] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 2
I20260709 13:13:48.700510  8055 pending_rounds.cc:107] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 13:13:48.700719  8055 raft_consensus.cc:2889] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:48.701655  8726 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:48.702816  8102 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:48.703157  8101 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:48.702924  8175 connection.cc:256] RPC call timeout handler was delayed by -2.193579708000243s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
W20260709 13:13:48.704131  8175 connection.cc:256] RPC call timeout handler was delayed by -2.0234797080001954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
I20260709 13:13:48.704246  8059 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
W20260709 13:13:48.704514  8175 connection.cc:256] RPC call timeout handler was delayed by -1.9307797080002729s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18130000000000002s for a response
W20260709 13:13:48.704836  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7291797080001743s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2037s for a response
W20260709 13:13:48.705168  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7273797080001714s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20390000000000003s for a response
W20260709 13:13:48.705451  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7138797080001495s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2054s for a response
W20260709 13:13:48.705708  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7093797080001423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20590000000000003s for a response
W20260709 13:13:48.705974  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6409797080002591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2135s for a response
W20260709 13:13:48.706251  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6364797080002518s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
W20260709 13:13:48.706544  8175 connection.cc:256] RPC call timeout handler was delayed by -1.5608797080001295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22240000000000004s for a response
W20260709 13:13:48.706820  8175 connection.cc:256] RPC call timeout handler was delayed by -1.0766797080002561s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2762s for a response
I20260709 13:13:48.707325  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:48.708240  8063 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:48.712410  8058 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:48.717906  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:48.718315  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:48.718916  8056 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:48.714155  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:48.720155  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.720902  8212 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:48.722584  8054 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:13:48.723853  8059 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:48.724824  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:48.729322  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:48.730257  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:48.730769  8054 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:48.727756  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:48.704713 11030 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.729801  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:48.733911  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:48.735317 11038 consensus_queue.cc:1059] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:13:48.738957 10821 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:48.735216 10906 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:48.711344 10948 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:48.709764 10369 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.737371  8063 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:48.746178  8212 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:48.747875  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:48.751426  8056 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:48.753826  8175 leader_election.cc:400] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:48.754129  8175 leader_election.cc:403] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:13:48.755646 10369 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.759888  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:48.757016  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:48.726717 11008 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:48.729506 11089 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:48.764160  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:48.733003 11126 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:48.767421 10951 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:48.775942 11010 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:48.762873 11038 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:48.776727 11038 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:48.779345  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:48.753239 10888 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:48.762748 10948 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:48.783963 11089 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:48.738981 11022 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:13:48.791170 10369 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:48.791522 11062 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:48.792186 11071 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:48.745954 11030 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.793282 10888 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.795408 10888 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:48.770259 11020 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:48.789913 11102 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:48.799634  8059 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:13:48.799888 11071 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:48.789335  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:48.801986 10821 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:48.802671  8175 leader_election.cc:400] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 6. Current term is 7.
I20260709 13:13:48.804054  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.804805  8132 raft_consensus.cc:2410] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:13:48.805925  8137 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:48.806481 11030 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:48.807060 11102 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:48.803462  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:48.808080 11089 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:48.808413  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:48.809535 11020 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:48.810143 11020 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:48.810595 11020 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:48.810946 11020 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:48.813155  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 288813us
W20260709 13:13:48.814224  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:48.813787  8027 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:48.820108  8137 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:13:48.820411 11102 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:48.820914  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:48.821480  8725 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:48.821614  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 61801.9us
I20260709 13:13:48.822672  8026 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:48.823732  8026 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:48.826571 10521 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:48.826884 10547 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:48.827256 10521 raft_consensus.cc:2764] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:13:48.827198 10982 consensus_queue.cc:1059] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:13:48.828015 10521 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:48.828387 10982 raft_consensus.cc:3055] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:48.828785 10982 raft_consensus.cc:740] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:48.829588 10982 consensus_queue.cc:260] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:48.810066 11089 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:48.831918 11007 consensus_queue.cc:1059] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:48.781332 11008 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:48.834487  8726 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:48.836441  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:48.837045  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:48.810151 11038 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:48.843674  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:48.837289  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:48.844414 11038 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:48.845258  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:48.845363  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:48.846419  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:48.837751  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 384) took 1028 ms. Trace:
I20260709 13:13:48.846511  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:48.848628  8203 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:48.849175 11008 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:48.846916 10369 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:13:48.846979 11047 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:13:48.856383  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:48.826009  8026 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:48.848443  8129 rpcz_store.cc:276] 0709 13:13:47.808970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.949810 (+140840us) service_pool.cc:224] Handling call
0709 13:13:47.950053 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:48.063573 (+113520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:48.063610 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:48.063648 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:48.063676 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:48.837551 (+773875us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54003c91d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:48.837593 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:48.837702 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260696320}
I20260709 13:13:48.837983  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:48.865365  8204 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:48.838383  8208 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:48498
I20260709 13:13:48.866243  8208 raft_consensus.cc:606] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:48.866684  8208 raft_consensus.cc:614] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:48.838722  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:48.835494  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:48.869782  8207 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:48.847364  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.881282  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:48.881860  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:48.882512  8207 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:48.886695  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:48.890643  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:48.891609  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:48.892156  8205 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:48.892390  8211 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:48.893422 11022 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:48.893914  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:48.894601  8204 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:48.895177 10547 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:13:48.896468  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 82753.8us
I20260709 13:13:48.897785  8027 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:48.898576  8027 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:48.899933  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 73990.9us
I20260709 13:13:48.913100  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:48.913785  8057 raft_consensus.cc:2368] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:48.914602  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:48.915598  8056 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:48.917044  8212 pending_rounds.cc:85] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 5
I20260709 13:13:48.917320  8212 pending_rounds.cc:107] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:13:48.917536  8212 raft_consensus.cc:2889] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:48.901738  8026 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:48.923434  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:48.890661 10982 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:48.945739  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:48.951053  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:48.970623  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:48.951913  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50241.5us
I20260709 13:13:48.972146 11069 raft_consensus.cc:2764] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:13:49.074896  8137 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:49.075069  8203 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:48.899497 11077 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:13:49.075696  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:49.076535  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:49.077358  8204 raft_consensus.cc:2410] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:13:48.942631 11040 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 13:13:49.104789  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:48.945808 10977 consensus_queue.cc:1059] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 13:13:49.119123  8211 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:13:49.119554  8211 pending_rounds.cc:85] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 5
I20260709 13:13:49.119834  8211 pending_rounds.cc:107] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:13:49.120111  8211 raft_consensus.cc:2889] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:48.919054 10521 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:48.923772 11089 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:13:49.077970  8206 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
W20260709 13:13:49.080319  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 175982us
W20260709 13:13:49.080888  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 177281us
I20260709 13:13:49.122977  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:49.123116  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:49.105420  8208 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:49.126232  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:49.077260  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 176051us
I20260709 13:13:49.117578  8696 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:49.090893  8207 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:49.132303  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:49.133014  8061 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:49.127959  8060 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:49.074795 10939 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:49.091712 10995 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:49.095235 10976 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:49.133463  8176 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:49.078588 11128 raft_consensus.cc:3055] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:49.157131  8206 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:49.157830  8176 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:49.138635  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:49.130658  8101 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:49.146608  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:49.152961  8211 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:49.156481  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:49.162354  8059 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:49.125065 10948 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:49.131076 11088 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:49.182945  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 107309us
I20260709 13:13:49.098511 11068 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:13:49.136423 10521 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:49.099519 11007 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:13:49.208964  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:49.140211 10933 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:13:49.223065  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.230100  8061 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
W20260709 13:13:49.203402 11095 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:13:49.184571 10995 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.234162  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:49.247507  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 116913us
I20260709 13:13:49.162042 11128 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:49.251183  8059 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:13:49.162205 10945 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:49.199245 11126 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:49.208379 10521 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:49.195881 10976 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:49.218436 10730 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:13:49.297662  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.303923  8057 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:49.270766 10978 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:49.306367  7945 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a003b4584d3747a381522d67299aaed2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:13:49.268847 10933 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:49.212121 11088 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:49.330405  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:49.329046  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:49.336082  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:49.337800 11128 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:49.350471  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:49.350699  8137 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: [4.7-6.8]
I20260709 13:13:49.351438  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:49.405386  8129 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:49.302659 10730 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:49.353145  8029 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:49.300721 11047 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:13:49.412931  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 59895.4us
I20260709 13:13:49.414287  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:49.416464  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:49.417908  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:49.371487  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:49.374693  8131 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:49.281448 10995 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:49.423733 11052 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:49.400310  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.385461  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:49.391716 11144 log.cc:927] Time spent T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
W20260709 13:13:49.391525 11147 log.cc:927] Time spent T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:13:49.425958 10995 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:49.426652 10995 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.427138 10995 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:49.314787  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:49.429104 11151 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:13:49.429181 10995 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:49.429777 10995 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:49.401863 11128 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:49.430259 10995 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:49.434121  8725 consensus_meta.cc:287] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.612s	user 0.016s	sys 0.001s
I20260709 13:13:49.450475 10995 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:49.411576 10730 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:49.399286 11068 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
W20260709 13:13:49.487499  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69657.6us
I20260709 13:13:49.489230  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:49.464372 10940 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:49.502997  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:49.507741  8205 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:49.514098  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:49.518322  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:49.469856  8725 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:13:49.517920 11052 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:49.530570 11078 raft_consensus.cc:3055] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:49.531040 11078 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:49.530865 11052 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:49.531970 11078 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:49.532879 11052 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:49.534117 11088 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:49.534377 11008 raft_consensus.cc:3055] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:49.534840 11008 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:13:49.535046 11148 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:13:49.535702 11008 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:49.536857 11008 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:49.539165 11068 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:49.501997 10976 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:49.492254 11155 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:49.541013 10940 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:13:49.543243 11038 consensus_meta.cc:287] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.698s	user 0.011s	sys 0.000s
I20260709 13:13:49.520891  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:49.528517  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:49.607062  8205 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:49.520080 10933 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:49.490901 10995 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:49.545781 10521 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:49.542210 11020 consensus_meta.cc:287] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.731s	user 0.008s	sys 0.001s
I20260709 13:13:49.601444  8203 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:49.552685 11153 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:49.600317 11068 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:49.602053  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2507) took 1341 ms. Trace:
I20260709 13:13:49.618155  8210 rpcz_store.cc:276] 0709 13:13:48.260999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.261146 (+   147us) service_pool.cc:224] Handling call
0709 13:13:48.261494 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:48.400401 (+138907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.601837 (+1201436us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54002bf554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:49.602001 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":305952000}
W20260709 13:13:49.625439  8693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.033s	sys 0.054s
W20260709 13:13:49.626590  8693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.033s	sys 0.054s
W20260709 13:13:49.610507  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 89679.5us
W20260709 13:13:49.604121  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 486604us
W20260709 13:13:49.635481  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 513127us
I20260709 13:13:49.607946 10940 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:49.610581 11069 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:49.650686  8696 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:49.619231 10995 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:49.672034  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:49.679581  8724 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:13:49.680745  8725 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:49.681753  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:49.682552  8724 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:49.686090  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:49.687426 10757 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.687889 10757 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:49.688391  8029 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:49.689515 10977 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:49.689845 10757 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:49.690675  8060 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:13:49.692550  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:49.694171  8029 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:49.695864  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:49.697649 10972 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:49.698448 10972 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:13:49.711493 11150 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 13:13:49.724341 10626 raft_consensus.cc:3055] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:49.724861 10626 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:49.731817  8209 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
W20260709 13:13:49.732280  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.663152 10940 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:49.780365  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 82378.1us
I20260709 13:13:49.619330 11020 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:49.794740  8132 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:13:49.771148 10972 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:49.795215  8058 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:13:49.725811 10626 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:49.793401  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 440062us
I20260709 13:13:49.807899  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:49.810026  8207 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:49.864014  8130 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:49.868283  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:49.869297  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.800323 11068 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:49.797956 10972 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:49.869089  8137 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:49.779547 10995 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:13:49.809954 10740 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:49.890805  8058 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:13:49.799350 11023 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:13:49.874712 10626 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:49.878185  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:49.882313 10972 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:49.908280  8137 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:49.904243 11155 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:49.951962 11152 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.521s	user 0.000s	sys 0.008s
I20260709 13:13:49.953480  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 338) took 1140 ms. Trace:
I20260709 13:13:49.953833  8062 rpcz_store.cc:276] 0709 13:13:48.813550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.830969 (+ 17419us) service_pool.cc:224] Handling call
0709 13:13:48.842687 (+ 11718us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:48.898539 (+ 55852us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:48.898567 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:48.898994 (+   427us) log.cc:844] Serialized 26 byte log entry
0709 13:13:49.203978 (+304984us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:49.204060 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.204193 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:49.951294 (+747101us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:49.953135 (+  1841us) raft_consensus.cc:1685] finished
0709 13:13:49.953161 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.953409 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55505,"raft.queue_time_us":79933,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":3175,"spinlock_wait_cycles":7475456,"thread_start_us":304415,"threads_started":1,"wal-append.queue_time_us":531161}
I20260709 13:13:49.955221 11038 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.955741 11038 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:49.929494  8725 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:49.958120  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:49.969233  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:49.971521  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:49.973058  8175 leader_election.cc:400] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:49.972151  8054 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:49.981179  8130 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:13:49.981114  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:49.947086 11047 raft_consensus.cc:697] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:49.996333  8130 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:13:49.996796  8130 pending_rounds.cc:85] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 7
I20260709 13:13:49.997061  8130 pending_rounds.cc:107] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:13:49.997336  8130 raft_consensus.cc:2889] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:49.992065  8062 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:49.986357  8104 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:50.004925  8137 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:13:49.965735 11068 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.009490  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:50.009574  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:50.018679  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:50.026038  8176 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.012s	sys 0.016s
I20260709 13:13:49.946663 11155 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.028515  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:50.043948  8132 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:50.032666  8208 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:13:49.975153 10948 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:50.007681 10982 consensus_meta.cc:287] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 1.035s	user 0.011s	sys 0.000s
I20260709 13:13:50.030210  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:50.037421  8212 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
W20260709 13:13:50.038375  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 387740us
W20260709 13:13:50.039655  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 66773.6us
I20260709 13:13:50.035354 11020 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:50.042616  8063 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.008s	sys 0.031s
W20260709 13:13:50.079233  8063 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.008s	sys 0.031s
I20260709 13:13:50.008816 10547 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.084287 10547 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.027264 11088 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.086786 11088 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.088783 11088 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:50.088928 10521 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:50.095568 11172 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:13:50.096366 11159 consensus_queue.cc:1059] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:13:50.097119  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 396) took 1242 ms. Trace:
I20260709 13:13:50.097334 11074 consensus_queue.cc:1059] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:13:50.073634  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 64211.5us
I20260709 13:13:50.077488  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:50.104771  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:50.042323  8176 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.012s	sys 0.016s
I20260709 13:13:50.080883 11128 consensus_queue.cc:1059] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:50.107491 11128 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:50.108749 11128 raft_consensus.cc:3055] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:50.109115 11128 raft_consensus.cc:740] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:50.083660  8696 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:50.027848 11077 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:50.089560 11158 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:50.090859 10547 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:50.093405  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.070048 10975 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:50.097627  8134 rpcz_store.cc:276] 0709 13:13:48.854817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.854913 (+    96us) service_pool.cc:224] Handling call
0709 13:13:48.855184 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:48.855507 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:48.855533 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:48.855892 (+   359us) log.cc:844] Serialized 26 byte log entry
0709 13:13:49.453527 (+597635us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b180005f0e0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:13:49.892795 (+439268us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:49.892831 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.892898 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:50.096783 (+203885us) raft_consensus.cc:1685] finished
0709 13:13:50.096808 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.097051 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":96,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":22393,"spinlock_wait_cycles":3947392,"thread_start_us":439108,"threads_started":1,"wal-append.queue_time_us":1148061}
W20260709 13:13:50.114776 10978 consensus_meta.cc:287] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.777s	user 0.008s	sys 0.003s
I20260709 13:13:50.114636  8058 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:50.148114  8206 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:50.116638 11020 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:50.106806 11088 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:13:50.109887 11128 consensus_queue.cc:260] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.116561 11077 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.078939 10948 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.123461 10905 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:50.105495  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:50.123930 11023 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:13:50.177454  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 379014us
I20260709 13:13:50.175007 10978 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:50.197973  8205 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:50.188947  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:50.181495 11166 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.123265 10369 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:13:50.193971  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:50.194080  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 88691.4us
I20260709 13:13:50.179261 11168 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.200846  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:50.201476  8723 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:13:50.181792 11177 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.204144  8062 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:13:50.209218  8062 raft_consensus.cc:606] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:50.209643  8062 raft_consensus.cc:614] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:50.204438  8730 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:50.188480  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:50.207567  8728 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:50.212401  8206 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:13:50.191722 11077 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:50.200726 11174 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:50.078140 11068 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.218591  8026 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:50.173671 11047 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.229476  8205 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:50.202134 10740 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:50.211540 11128 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:50.237680  8026 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:50.211781 11168 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.242799  8026 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:50.235914 10730 consensus_meta.cc:287] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.739s	user 0.005s	sys 0.005s
I20260709 13:13:50.267346  8206 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:13:50.268667  8206 pending_rounds.cc:85] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 4
I20260709 13:13:50.268963  8206 pending_rounds.cc:107] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:13:50.269248  8206 raft_consensus.cc:2889] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:50.222460 11174 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:50.204249 11166 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.212664 11034 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:13:50.275125  8131 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:13:50.278705  8134 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:13:50.330715  8131 pending_rounds.cc:85] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 3
I20260709 13:13:50.330837  8134 pending_rounds.cc:85] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 8
I20260709 13:13:50.331223  8131 pending_rounds.cc:107] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:13:50.331359  8134 pending_rounds.cc:107] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:13:50.331661  8131 raft_consensus.cc:2889] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:50.331769  8134 raft_consensus.cc:2889] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:50.278652  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:50.333345  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2415) took 1023 ms. Trace:
I20260709 13:13:50.203748 11038 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:50.283607  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:13:50.280267 10976 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:13:50.238517 10982 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:13:50.206044 11023 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.242238 11167 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:50.239941 10995 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:50.271051 11163 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:50.216725 11177 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.335826  8056 rpcz_store.cc:276] 0709 13:13:49.310314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.427101 (+116787us) service_pool.cc:224] Handling call
0709 13:13:49.427450 (+   349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:49.427766 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:49.427794 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:49.428332 (+   538us) log.cc:844] Serialized 52 byte log entry
0709 13:13:49.803362 (+375030us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:50.221799 (+418437us) log.cc:844] Serialized 14 byte log entry
0709 13:13:50.311397 (+ 89598us) log.cc:844] Serialized 14 byte log entry
0709 13:13:50.311595 (+   198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.311697 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:50.311744 (+    47us) raft_consensus.cc:1685] finished
0709 13:13:50.311766 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.333273 (+ 21507us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85617,"prepare.queue_time_us":141063,"prepare.run_wall_time_us":38,"raft.queue_time_us":100254,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":63,"spinlock_wait_cycles":7279872,"thread_start_us":673581,"threads_started":2,"wal-append.queue_time_us":532184}
I20260709 13:13:50.327132 11153 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:50.281374 11174 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:50.273881 10730 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:50.386597  8061 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.012s	sys 0.009s
W20260709 13:13:50.394934  8061 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.012s	sys 0.010s
I20260709 13:13:50.395746 11160 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:50.371378 11155 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:50.242720 10740 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.324699 11069 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:50.404093  8176 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:50.381407  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 173950us
I20260709 13:13:50.366858 11020 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:50.398823  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:50.407073  8206 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:50.407043  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:50.409842  8057 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:50.410693  8208 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:13:50.411566  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:50.393702 10888 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:50.399436  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:50.401479  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 296713us
I20260709 13:13:50.382139 10905 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:13:50.420259 10940 consensus_meta.cc:287] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.621s	user 0.004s	sys 0.004s
I20260709 13:13:50.421360 10972 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.423231 11068 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:50.458918 10888 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.428815 10948 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:50.466506 10888 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:50.466612 11166 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:50.466836 10948 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:50.467360 10948 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:50.468142 11023 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:50.468415 10972 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:50.402665 11188 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:50.431304  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:50.432572 11175 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:13:50.460965 11178 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:13:50.461553 11168 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:50.462606  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:50.463749 10626 consensus_meta.cc:287] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.539s	user 0.009s	sys 0.000s
I20260709 13:13:50.414239 11069 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.468183 10948 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.459268 11193 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:13:50.387003  8059 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.000s	sys 0.022s
I20260709 13:13:50.468439  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2413) took 1158 ms. Trace:
I20260709 13:13:50.474318 11177 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:50.431003 11020 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.474890  9980 consensus_queue.cc:1059] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:50.475737  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:50.476320 11008 consensus_meta.cc:287] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.939s	user 0.008s	sys 0.003s
W20260709 13:13:50.474813  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 65464.3us
I20260709 13:13:50.482416  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2419) took 1108 ms. Trace:
I20260709 13:13:50.512213  8063 rpcz_store.cc:276] 0709 13:13:49.374077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.460872 (+ 86795us) service_pool.cc:224] Handling call
0709 13:13:49.461120 (+   248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:49.480991 (+ 19871us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:49.481033 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:49.481573 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 13:13:50.079596 (+598023us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:50.079642 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.079701 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:50.482183 (+402482us) raft_consensus.cc:1685] finished
0709 13:13:50.482212 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.482362 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19464,"raft.queue_time_us":66785,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":62,"thread_start_us":563517,"threads_started":1,"wal-append.queue_time_us":741592}
W20260709 13:13:50.488945  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 57709.7us
I20260709 13:13:50.528039  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:50.549710  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:50.566828  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.431407 11185 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.479195 11078 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:50.495782  8132 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:50.431325  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:50.489959  8055 rpcz_store.cc:276] 0709 13:13:49.309726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.421729 (+112003us) service_pool.cc:224] Handling call
0709 13:13:49.422088 (+   359us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:49.777691 (+355603us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:49.777719 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:49.778208 (+   489us) log.cc:844] Serialized 52 byte log entry
0709 13:13:50.145978 (+367770us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:50.402536 (+256558us) log.cc:844] Serialized 14 byte log entry
0709 13:13:50.409857 (+  7321us) log.cc:844] Serialized 14 byte log entry
0709 13:13:50.410010 (+   153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.410090 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:50.468236 (+ 58146us) raft_consensus.cc:1685] finished
0709 13:13:50.468255 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.468385 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":72971,"prepare.queue_time_us":311380,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":155798,"raft.run_cpu_time_us":418,"raft.run_wall_time_us":71741,"thread_start_us":854171,"threads_started":3,"wal-append.queue_time_us":576830}
I20260709 13:13:50.480396 10888 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.664170  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:50.712736  8203 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
W20260709 13:13:50.490599  8059 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.000s	sys 0.022s
I20260709 13:13:50.714221  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2411) took 1414 ms. Trace:
I20260709 13:13:50.498045  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:50.716423  8131 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:50.714653  8059 rpcz_store.cc:276] 0709 13:13:49.299391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.306277 (+  6886us) service_pool.cc:224] Handling call
0709 13:13:49.306579 (+   302us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:49.306875 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:49.306899 (+    24us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:49.319699 (+ 12800us) log.cc:844] Serialized 52 byte log entry
0709 13:13:49.795913 (+476214us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:49.875083 (+ 79170us) log.cc:844] Serialized 14 byte log entry
0709 13:13:50.713847 (+838764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.713944 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:50.713998 (+    54us) raft_consensus.cc:1685] finished
0709 13:13:50.714017 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.714156 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3792,"prepare.queue_time_us":128799,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":3115,"raft.queue_time_us":188653,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":40998,"thread_start_us":1310486,"threads_started":2,"wal-append.queue_time_us":675189}
I20260709 13:13:50.433630  8210 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:13:50.492462 11145 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 1.216s	user 0.002s	sys 0.000s
I20260709 13:13:50.481788 10730 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:50.482218 11068 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.771023 11068 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.481426 10939 raft_consensus.cc:3055] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:50.771092 11096 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:50.487862 11069 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:50.802999  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2535) took 1342 ms. Trace:
I20260709 13:13:50.810554  8204 rpcz_store.cc:276] 0709 13:13:49.460411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.464431 (+  4020us) service_pool.cc:224] Handling call
0709 13:13:49.497337 (+ 32906us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:49.497670 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:49.497697 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:49.508864 (+ 11167us) log.cc:844] Serialized 52 byte log entry
0709 13:13:50.027217 (+518353us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:50.027258 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.027321 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:50.802684 (+775363us) raft_consensus.cc:1685] finished
0709 13:13:50.802714 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.802917 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":12244,"thread_start_us":412362,"threads_started":1,"wal-append.queue_time_us":714324}
I20260709 13:13:50.479749 10940 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.809758  8026 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:13:50.812698  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 408956us
I20260709 13:13:50.813154  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:50.778455 10939 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:50.824247  8726 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:13:50.490397 10905 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.775059 11192 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:50.817373  8026 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:50.782675 11008 raft_consensus.cc:3055] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:50.478740 11188 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:50.854461  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.889137  8131 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
W20260709 13:13:50.891077  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 73762.9us
I20260709 13:13:50.775471 10888 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.843436 11068 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:50.897382  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:50.849673 10626 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:50.904707  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:50.488672 11193 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.815429 10730 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:13:50.890034 11047 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.913941  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:50.914543  8204 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:50.914873  8206 raft_consensus.cc:2410] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:13:50.893663 10740 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:50.894151 11008 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:50.896416 11183 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:50.848948 11069 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:13:50.845661 10982 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:13:50.846123 11168 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:13:50.912150 10972 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:50.803212 11174 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.890391 11128 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:50.707100 11185 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:50.915737  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.926101  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2532) took 1466 ms. Trace:
I20260709 13:13:50.937486  8209 rpcz_store.cc:276] 0709 13:13:49.459624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.464611 (+  4987us) service_pool.cc:224] Handling call
0709 13:13:49.464856 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:49.732271 (+267415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.925833 (+1193562us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b5400241dd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:50.926036 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":613441792}
I20260709 13:13:50.927984  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:50.939073  8204 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:13:50.928530 11154 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:50.929170 10940 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:50.929806 11199 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:50.941352 11199 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.930008  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1291) took 1610 ms. Trace:
I20260709 13:13:50.942641  8057 rpcz_store.cc:276] 0709 13:13:49.319391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.430020 (+110629us) service_pool.cc:224] Handling call
0709 13:13:49.430292 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:50.410227 (+979935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.929813 (+519586us) spinlock_profiling.cc:243] Waited 979 ms on lock 0x7b5400333e54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:50.929954 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2252418304}
I20260709 13:13:50.930598 11193 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:50.906894 10626 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.914108 11068 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:50.952445 11068 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.953433 11020 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:50.941658 11008 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.922156 11183 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.940548  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:50.940886 11195 log.cc:927] Time spent T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:13:50.950361 11200 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:50.950870 11187 log.cc:927] Time spent T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:13:50.954540 11068 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:50.944756 11077 consensus_meta.cc:287] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.712s	user 0.003s	sys 0.006s
I20260709 13:13:50.933385 10972 raft_consensus.cc:697] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:50.963286  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:50.978312  8207 raft_consensus.cc:2393] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 6: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:13:50.963898  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.978919 11202 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:50.919397 11047 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.979575 11202 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:50.980046 11202 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.981142 11047 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:50.960319 11183 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:50.966410  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:13:50.980455 11165 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 1.046s	user 0.003s	sys 0.014s
I20260709 13:13:50.982744 10521 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:50.983600 10521 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.986040 10976 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:50.949501 11154 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.986713 10976 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.989740 10940 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.990324 10940 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:50.991675 11199 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:50.993806 10972 consensus_queue.cc:237] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.994849 10982 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:50.995533 10982 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:50.997952 10982 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:50.954979 10626 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:51.003803 11185 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:51.004964 11174 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:50.919219 10939 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.005808 10905 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:50.982084 11202 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:51.008643  8207 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:50.946908 11128 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.006803 11068 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.086028  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:51.023869 10906 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.090059  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2414) took 1780 ms. Trace:
I20260709 13:13:51.093832  8061 rpcz_store.cc:276] 0709 13:13:49.310028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.422301 (+112273us) service_pool.cc:224] Handling call
0709 13:13:49.422580 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:49.777254 (+354674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:49.777282 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:49.795632 (+ 18350us) log.cc:844] Serialized 26 byte log entry
0709 13:13:50.395469 (+599837us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:50.395517 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.395593 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:51.089836 (+694243us) raft_consensus.cc:1685] finished
0709 13:13:51.089860 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:51.089995 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86297,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":12261,"thread_start_us":878038,"threads_started":2,"wal-append.queue_time_us":853741}
I20260709 13:13:51.074975 11007 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:13:51.101684  8027 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.68:44573 timed out after 2.194s (SENT) [suppressed 21 similar messages]
I20260709 13:13:51.101970  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:51.006929 11077 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.102550  8129 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:51.105330  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:51.102988  8132 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:51.107656  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:50.974705 11200 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.109297  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:51.083468 10939 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:51.106665  8102 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:51.086783  8211 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:51.113829  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:51.085021 11174 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:51.115866  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:51.119238  8207 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:51.109297 11185 raft_consensus.cc:3055] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:51.109835 11038 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:51.087354 11068 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.137183  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:51.110288 11202 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:51.145331  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 617351us
I20260709 13:13:51.146251  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:51.147150  8207 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:51.137679  8129 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:13:51.118840 10888 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:51.149537  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:51.151010  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:51.109966 11088 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:13:51.130409 11020 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:51.152783  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:51.123293 11185 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:51.132364 11199 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:51.154616  8133 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:13:51.105046 10906 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.111361 11007 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.138201 11038 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.164002  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2553) took 1177 ms. Trace:
I20260709 13:13:51.149181 11193 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:51.208287 11183 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.209933 10940 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:51.210736 11128 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:51.168694  8205 rpcz_store.cc:276] 0709 13:13:49.986602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.987529 (+   927us) service_pool.cc:224] Handling call
0709 13:13:49.987826 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:50.232234 (+244408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:51.163793 (+931559us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b5400189054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:51.163949 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":483010304}
I20260709 13:13:51.213980 11200 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:51.149278 11202 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.161541 11020 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.158149 10888 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:13:51.218739 11212 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:13:51.218762  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1294) took 1248 ms. Trace:
I20260709 13:13:51.243880  8058 rpcz_store.cc:276] 0709 13:13:49.970761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.970884 (+   123us) service_pool.cc:224] Handling call
0709 13:13:49.971157 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:50.178697 (+207540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:51.218552 (+1039855us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54001e8cd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:51.218710 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329466752}
I20260709 13:13:51.137846 11006 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:13:51.245585 11006 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.246063 11183 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:51.246009 11006 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.247978 11006 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:51.161864 11199 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.248529 11199 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:51.248726 11183 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.249214 11183 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.249647  8207 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:13:51.251171  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:51.251945  8211 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:51.252036 10940 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:51.148586 11074 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:13:51.252528 10940 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.253507 10821 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:51.254524 10940 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:51.254644  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:51.255185  8210 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:51.255497 10940 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:51.255873  8725 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:13:51.256124 10940 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.257946 10940 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:51.258453 11006 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.130255 11174 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.155915 10948 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:51.238504  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:51.251374 11183 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:51.282831  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:51.282744 11043 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:51.283501  8203 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:51.284195 11022 raft_consensus.cc:3055] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:51.284322  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:51.284686 11022 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:51.284440 11043 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:51.285439  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:51.286485 11211 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.260859  8209 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:13:51.287964  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:51.289839  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:51.290719  8203 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:51.154069 11207 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:51.257800 11199 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.266026  8210 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:51.266274  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:51.266597  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:51.269541  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:51.223127 11038 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:51.258926 11006 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:51.292303 11078 consensus_meta.cc:287] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.000s
W20260709 13:13:51.344527  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 242852us
W20260709 13:13:51.356020  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 89508.8us
I20260709 13:13:51.310532  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:51.362169  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:51.362375  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 96175.8us
I20260709 13:13:51.290653 11128 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:51.286998 11211 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.365422  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:51.308740 10948 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:51.376228 11209 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.376732 11209 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.363265 11193 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:51.378815 11209 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:51.379328  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 524) took 1172 ms. Trace:
I20260709 13:13:51.379376 11199 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:51.379839  8726 rpcz_store.cc:276] 0709 13:13:50.206832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.211863 (+  5031us) service_pool.cc:224] Handling call
0709 13:13:50.212116 (+   253us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:50.848709 (+636593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:51.379107 (+530398us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54006527d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:51.379275 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407246592}
I20260709 13:13:51.382810 11174 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:51.384136 11154 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:51.384939 11020 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:51.386086 11202 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:51.386997 11068 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:51.387499 10982 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:51.388032 10982 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.366313  8135 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:51.391501  8058 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:13:51.391651  8135 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1504) took 1584 ms (client timeout 1607 ms). Trace:
W20260709 13:13:51.392062  8135 rpcz_store.cc:269] 0709 13:13:49.807566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.807694 (+   128us) service_pool.cc:224] Handling call
0709 13:13:51.391494 (+1583800us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54001f09d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:51.391587 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3582715776}
I20260709 13:13:51.393715 11069 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.394178 11069 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.286628 11185 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.396426  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:51.341001 10906 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:51.366487 11183 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:13:51.367604 11192 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:13:51.367749 11207 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
W20260709 13:13:51.384209 11215 log.cc:927] Time spent T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:13:51.355801 11176 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:51.396147  8203 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:13:51.399120  8058 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:51.308494 11043 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:51.366384 10940 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:51.310045  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 165127us
I20260709 13:13:51.355156 11077 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:51.401125  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:51.402572  8027 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:51.406414  8055 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:51.407547  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:51.409312 10982 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:51.412503 10948 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.415390 11038 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.416307 11007 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:51.450738 10369 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:51.417111  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:51.453918  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:51.454429  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:51.455786  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:51.456354  8134 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:51.458006  8054 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:51.460814  8212 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:51.418627  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:51.419232  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:51.462831  8210 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
W20260709 13:13:51.424898  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.453860  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:51.464349  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.426368  8060 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:13:51.464915  8060 pending_rounds.cc:85] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 7
I20260709 13:13:51.465185  8060 pending_rounds.cc:107] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:13:51.465400  8060 raft_consensus.cc:2889] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:51.400231  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:51.433334  8058 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:13:51.437917  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:51.467326  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:51.401760 11088 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.401943 11169 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.444099  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:51.468621  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:51.469441  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:51.402077 10972 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:51.445951  8055 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:51.446048  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:51.446827  8209 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:51.427143  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:51.440299 11005 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:13:51.441440 10940 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.448824 11160 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:51.449004 11185 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:51.438825 11154 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:51.482887  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:51.483560  8723 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:51.449487 11206 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:13:51.408619 11128 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:13:51.467444  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:51.470889  8134 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:51.510279  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:51.475281  8203 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:51.510938  8056 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:51.477133  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:51.478070  8696 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:51.418936 10730 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:13:51.481870  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:51.453341 10982 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:51.440622 11040 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:51.474026 11183 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:51.522030  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:51.523001 11217 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:13:51.523850 11217 raft_consensus.cc:3055] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:51.524176 11217 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:51.524915 11217 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.526075 11217 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:51.475541 11192 raft_consensus.cc:3055] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:51.442203 11158 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
W20260709 13:13:51.520502  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 53120.2us
W20260709 13:13:51.560026  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 192883us
I20260709 13:13:51.516819  8054 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:51.561424  8055 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:51.533222  8725 raft_consensus.cc:2410] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:13:51.527949  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:51.528650 10976 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:51.599809  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:51.529942  8060 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:51.600286  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:51.530447  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:51.453131 11219 consensus_queue.cc:1059] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:13:51.533627 10888 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.480479 10972 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.531299 10626 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.532140 11022 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.532773 11211 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:51.533748  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:51.513854  8696 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:51.482518 11160 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:51.452196 11220 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
W20260709 13:13:51.562793  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.635082  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:51.560671  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:51.527350 11038 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:51.565428  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:51.598018 11007 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:51.596331  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:51.640254  8056 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:51.640357  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:51.521085 10730 consensus_queue.cc:1243] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:51.598393  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:51.598649  8729 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:51.599519  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:51.529047 11006 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:51.509641 11020 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:51.600891  8054 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:51.476351 11088 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.478942 11169 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.516738 10821 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:51.531035  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:51.527395 10940 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:51.558413 11043 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:51.559093 10973 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.621790  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:51.628345  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:51.628793  8134 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:13:51.629456  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:51.630309  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:51.633375  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.637419  8203 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:13:51.565879  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:51.637784  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 124001us
I20260709 13:13:51.597923 11207 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:51.556988 11192 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:13:51.639045  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 78455.4us
I20260709 13:13:51.648329  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:51.648464  8138 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:13:51.611281 11040 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:13:51.677086  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 274640us
I20260709 13:13:51.971067  8723 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:13:51.703002  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:51.886631  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:51.976109  8205 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:51.948390  8102 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:51.977568  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:51.978112  8026 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:51.772655  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.650547  8209 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:51.980136  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:51.985798  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:51.986514  8135 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:51.989759  8026 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:51.991487  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:51.991962  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:51.884281 11221 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:13:51.990556 10730 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:13:52.002153  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.616279 10888 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:52.010183  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:51.641635 11069 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:51.996141  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:51.637605 11222 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:52.012508 11222 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:51.640367 11022 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:52.025130 11208 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.025933  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:52.026506  8204 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 4.
W20260709 13:13:52.027267  8728 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.013s	sys 0.012s
W20260709 13:13:52.027675  8728 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.013s	sys 0.012s
I20260709 13:13:51.659543 10369 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:51.991598 11193 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:52.004101  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:51.719168 10948 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:52.005903  8727 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:13:52.006302  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.007025  8203 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:13:52.011304  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:51.958495 11005 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:52.000214 11043 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:51.825048  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:51.998792 11038 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:51.992575  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:52.079665  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 48134.5us
I20260709 13:13:52.025666 11208 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.984375 10973 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:52.131862  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:52.001772 11209 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:13:52.150892 11077 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.013s	sys 0.031s
I20260709 13:13:52.004949 10730 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:52.149679  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:52.151648  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:52.193213  8204 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:52.003553 11074 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:51.658509 11007 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:51.974289 11040 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:51.985656 11020 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:52.122587 11193 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:52.079218 10369 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:52.010672 10939 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.149s
I20260709 13:13:51.958212 11078 consensus_queue.cc:1059] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:13:51.999354 11207 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:52.190227 11199 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.190888  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:52.148070 11069 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:52.124032 10948 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:52.182986 11038 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:52.214313  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:52.203850  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:52.201682 11200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.018s	sys 0.055s
I20260709 13:13:52.220503  8203 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:52.221743  8061 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:52.198829 10730 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:52.203539 11040 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:52.201740 10626 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:52.219276 10369 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:52.272306 11193 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:52.223282 10948 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:52.273759 11207 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:52.275241 11040 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:52.277181 11040 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:52.230728 11069 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.278940 11069 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:52.280470 10730 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:52.280900 11069 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:52.203627 11074 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:52.280932  8135 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:13:52.282006  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:52.284597  8205 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:13:52.199761 11188 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:52.225225 11038 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:52.248597 11236 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
W20260709 13:13:52.249279 11233 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:13:52.221874 11199 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:52.186802 11192 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:52.257328  8129 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:13:52.304497  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:52.257864 11174 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.257879 11208 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:52.258728 11043 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:52.260107 11222 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:52.203372 11225 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:13:52.263536 11007 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:52.264842 10973 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:52.272097 11020 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:52.282590 10730 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:52.279237 10369 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:52.183807 11005 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:52.363312  8726 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:52.283093 11074 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:52.237191  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 289913us
W20260709 13:13:52.300608  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 323208us
I20260709 13:13:52.365355  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:52.292272 11077 raft_consensus.cc:3055] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:52.303675  8205 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
W20260709 13:13:52.309643 11235 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:13:52.370275  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 380548us
W20260709 13:13:52.315060  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 339994us
I20260709 13:13:52.371184 11188 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:52.372117 11069 raft_consensus.cc:3055] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:52.372496 11069 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:52.373289 11069 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:52.374517 11069 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:52.346747  8062 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.022s	sys 0.073s
W20260709 13:13:52.384856  8062 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.026s	sys 0.084s
W20260709 13:13:52.318897 11200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.019s	sys 0.056s
I20260709 13:13:52.388597  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:52.389673  8204 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 4.
I20260709 13:13:52.390342 10369 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:52.388586  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:52.363655 11192 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:52.403417 11253 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:52.406040  8104 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:52.401705 11007 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.417197 11007 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:52.417352  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:52.418738  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:52.419230 11007 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:52.419940  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:52.421271  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:52.421909  8725 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:52.415298  8204 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:13:52.408782  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 409) took 1300 ms. Trace:
I20260709 13:13:52.426585  8133 rpcz_store.cc:276] 0709 13:13:51.108622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.108728 (+   106us) service_pool.cc:224] Handling call
0709 13:13:51.108974 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:51.161912 (+ 52938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.408536 (+1246624us) spinlock_profiling.cc:243] Waited 45.3 ms on lock 0x7b540036ef54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:52.408726 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104221568}
I20260709 13:13:52.431782  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:52.434520  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.436417  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:52.384601 11077 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:52.437968  8176 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:52.461175  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.472998  8129 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:52.392529 11169 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:52.407564 11005 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:52.316321 11174 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:52.483108  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:52.414666 10888 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:52.482471  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:52.490548  8204 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:52.488251  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 661) took 1018 ms. Trace:
I20260709 13:13:52.492220  8133 raft_consensus.cc:2368] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9.
I20260709 13:13:52.492220  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:52.425961 11207 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:52.431886 10973 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:52.472325 11128 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:52.517127  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.542307 11052 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:52.547334  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:52.485606 11222 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:52.485236 11253 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:52.701897  8204 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:13:52.708968 11020 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:52.574633 11222 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:52.687623 11253 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:52.694792 11022 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:13:52.777930  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.513576 11207 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:52.542944 11052 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:52.573560 10948 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:52.574908 11128 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:52.752146 11077 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:52.843820 11020 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:52.845781 11222 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:53.085305  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:52.979663  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.275795  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 666) took 1719 ms. Trace:
I20260709 13:13:53.276259  8727 rpcz_store.cc:276] 0709 13:13:51.556595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.556816 (+   221us) service_pool.cc:224] Handling call
0709 13:13:51.597950 (+ 41134us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:52.123581 (+525631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.275533 (+1151952us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b540060ead4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:53.275725 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937294720}
I20260709 13:13:53.278827  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:53.279457  8138 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:53.280658  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:53.281651 11128 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:53.282013  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:53.282179 11128 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.282593  8138 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:53.282648 11128 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:53.166764  8205 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.014s	sys 0.026s
W20260709 13:13:53.283824  8205 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.014s	sys 0.028s
W20260709 13:13:52.889446 11217 consensus_meta.cc:287] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 1.328s	user 0.013s	sys 0.010s
I20260709 13:13:53.284767 11128 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:53.285310 11128 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:53.285498 11174 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:53.245024 10948 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:53.295317  8104 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:53.296869  8104 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:53.374804  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:53.274736 10940 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.015s	sys 0.043s
I20260709 13:13:53.261963 11253 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:53.285821 11128 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.185230 11077 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:54.187507 10939 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:54.188475 10939 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.161067  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:53.564002  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:52.190659  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:54.189903 11088 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:54.196345  8727 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:13:54.141853  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2576) took 2505 ms. Trace:
I20260709 13:13:53.616226 10948 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:53.696135 10626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.184s	user 0.011s	sys 0.015s
W20260709 13:13:54.154635  8695 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.003s	sys 0.007s
I20260709 13:13:54.157874  8209 raft_consensus.cc:3055] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:54.179140  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:54.159036  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:54.191833  8722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 539) took 1756 ms (client timeout 2100 ms). Trace:
W20260709 13:13:54.038026  8026 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.219s	user 0.001s	sys 0.013s
I20260709 13:13:54.141955  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2577) took 2505 ms. Trace:
I20260709 13:13:54.042243  8027 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:54.049901  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2441) took 2533 ms. Trace:
I20260709 13:13:54.048583  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2437) took 2569 ms. Trace:
I20260709 13:13:54.104264  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:54.156505  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.051759  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 346) took 2654 ms. Trace:
I20260709 13:13:54.080647  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.177852  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 673) took 2317 ms. Trace:
W20260709 13:13:54.164958  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.143414  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1432) took 3740 ms. Trace:
I20260709 13:13:54.177464  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:54.134133  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.152354  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 441) took 2737 ms. Trace:
W20260709 13:13:54.132560  8203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1451) took 2567 ms (client timeout 2243 ms). Trace:
W20260709 13:13:54.121408  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.102753  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.144567  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:54.153741  8723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 663) took 2668 ms (client timeout 2158 ms). Trace:
W20260709 13:13:54.073771  8102 connection.cc:256] RPC call timeout handler was delayed by -1.304057739999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
I20260709 13:13:54.213725  8209 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:52.492115  8724 rpcz_store.cc:276] 0709 13:13:51.470148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.475138 (+  4990us) service_pool.cc:224] Handling call
0709 13:13:51.552469 (+ 77331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:51.637761 (+ 85292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:51.641836 (+  4075us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:51.883184 (+241348us) log.cc:844] Serialized 26 byte log entry
0709 13:13:52.301339 (+418155us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:52.301401 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.301495 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:52.488019 (+186524us) raft_consensus.cc:1685] finished
0709 13:13:52.488041 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:52.488189 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8899,"raft.queue_time_us":34350,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":52006,"thread_start_us":349507,"threads_started":1,"wal-append.queue_time_us":541757}
I20260709 13:13:54.079054  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:54.217775  8695 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.941s	user 0.003s	sys 0.012s
W20260709 13:13:53.931454 11043 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.007s	sys 0.013s
W20260709 13:13:53.610399  8693 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.988s	user 0.017s	sys 0.021s
I20260709 13:13:54.215720  8212 rpcz_store.cc:276] 0709 13:13:51.636553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.140422 (+2503869us) service_pool.cc:224] Handling call
0709 13:13:54.140944 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.141524 (+   580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.141548 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.141615 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:54.141648 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.141705 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.141798 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.198855  8207 rpcz_store.cc:276] 0709 13:13:51.636302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.140037 (+2503735us) service_pool.cc:224] Handling call
0709 13:13:54.140669 (+   632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.141219 (+   550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.141256 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.141338 (+    82us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:54.141368 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.141455 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.141593 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.216192  8058 rpcz_store.cc:276] 0709 13:13:51.516324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.533855 (+ 17531us) service_pool.cc:224] Handling call
0709 13:13:51.534144 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:51.640462 (+106318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:51.640490 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:51.640533 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:51.640572 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.049347 (+2408775us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400334d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:54.049452 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.049714 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244094208}
W20260709 13:13:53.865546 11038 consensus_meta.cc:287] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 1.561s	user 0.011s	sys 0.001s
W20260709 13:13:54.285261  8730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 529) took 1905 ms (client timeout 2247 ms). Trace:
W20260709 13:13:54.212525  8727 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 534) took 1827 ms (client timeout 2194 ms). Trace:
W20260709 13:13:54.214924  8722 rpcz_store.cc:269] 0709 13:13:52.435328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.160369 (+1725041us) service_pool.cc:224] Handling call
0709 13:13:54.191718 (+ 31349us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:54.224313  8026 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.405s	user 0.006s	sys 0.019s
I20260709 13:13:54.218811  8726 rpcz_store.cc:276] 0709 13:13:51.860472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.197420 (+336948us) service_pool.cc:224] Handling call
0709 13:13:52.216657 (+ 19237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:52.363716 (+147059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.177303 (+1813587us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540060de54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:54.177627 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336898304}
I20260709 13:13:54.216279  8055 rpcz_store.cc:276] 0709 13:13:51.479468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.479603 (+   135us) service_pool.cc:224] Handling call
0709 13:13:51.479912 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:51.597724 (+117812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.047775 (+2450051us) spinlock_profiling.cc:243] Waited 81.3 ms on lock 0x7b5400333954. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:54.048269 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":187021184}
I20260709 13:13:53.103014 11211 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:54.162678  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 865706us
W20260709 13:13:54.290556  8730 rpcz_store.cc:269] 0709 13:13:52.375444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.178540 (+1803096us) service_pool.cc:224] Handling call
0709 13:13:54.281045 (+102505us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.218986  8208 rpcz_store.cc:276] 0709 13:13:50.402676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.461613 (+ 58937us) service_pool.cc:224] Handling call
0709 13:13:50.461895 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:50.712063 (+250168us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:50.712105 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:50.712156 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:50.712201 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.142827 (+3430626us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b54000ba9d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:54.142931 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.143195 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":574833024}
I20260709 13:13:54.216653  8061 rpcz_store.cc:276] 0709 13:13:51.397633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.397766 (+   133us) service_pool.cc:224] Handling call
0709 13:13:51.397987 (+   221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:52.229259 (+831272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.051211 (+1821952us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b540009c254. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:54.051558 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1894065536}
W20260709 13:13:54.280239  8203 rpcz_store.cc:269] 0709 13:13:51.564790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.149402 (+584612us) service_pool.cc:224] Handling call
0709 13:13:54.132045 (+1982643us) spinlock_profiling.cc:243] Waited 27.5 ms on lock 0x7b54001536d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:54.132269 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63270656}
I20260709 13:13:54.293598  8729 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:54.293797 11261 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.537s	user 0.005s	sys 0.006s
I20260709 13:13:52.874840 11199 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:54.295809  8102 connection.cc:256] RPC call timeout handler was delayed by -1.0097577399999409s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
W20260709 13:13:54.290902  8727 rpcz_store.cc:269] 0709 13:13:52.382434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.388769 (+1006335us) service_pool.cc:224] Handling call
0709 13:13:54.204602 (+815833us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:54.281047  8723 rpcz_store.cc:269] 0709 13:13:51.484967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.485102 (+   135us) service_pool.cc:224] Handling call
0709 13:13:54.153176 (+2668074us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54007611d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:54.153416 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":851063168}
I20260709 13:13:53.583840  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:54.303103 11262 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.556s	user 0.003s	sys 0.008s
W20260709 13:13:54.303619 11258 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.497s	user 0.000s	sys 0.009s
W20260709 13:13:54.304198 11263 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.632s	user 0.001s	sys 0.023s
I20260709 13:13:54.304145  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:54.304486  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2580) took 2417 ms. Trace:
I20260709 13:13:54.304865 11269 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:54.305534 11249 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:53.391305 11052 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:53.150857 10888 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:53.800556 11192 consensus_meta.cc:287] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 1.375s	user 0.008s	sys 0.000s
I20260709 13:13:54.322631 11275 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.322831 10888 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.323413  8211 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:13:54.323866  8211 pending_rounds.cc:85] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 3
I20260709 13:13:54.324221  8211 pending_rounds.cc:107] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:13:54.324611  8211 raft_consensus.cc:2889] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:53.164525 11207 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:54.311167 10626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.773s	user 0.013s	sys 0.025s
I20260709 13:13:53.812265 11020 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:53.666566 11174 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:53.810636 11250 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:54.314515 10948 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.351503 10948 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:54.315056  8730 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 533) took 1933 ms (client timeout 2323 ms). Trace:
W20260709 13:13:54.352161  8730 rpcz_store.cc:269] 0709 13:13:52.381278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.303680 (+1922402us) service_pool.cc:224] Handling call
0709 13:13:54.314980 (+ 11300us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:54.315347  8693 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.673s	user 0.019s	sys 0.027s
W20260709 13:13:54.353358  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 1.731e+06us
I20260709 13:13:54.315676  8137 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:54.354647  9980 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:54.355393  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.355657  9980 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.357084  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:54.316187 11259 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.641s	user 0.015s	sys 0.008s
I20260709 13:13:54.358269  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.219226  8206 rpcz_store.cc:276] 0709 13:13:51.415192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.420238 (+  5046us) service_pool.cc:224] Handling call
0709 13:13:51.599108 (+178870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:51.599323 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:51.836978 (+237655us) log.cc:844] Serialized 14 byte log entry
0709 13:13:52.189084 (+352106us) log.cc:844] Serialized 14 byte log entry
0709 13:13:52.546117 (+357033us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:52.546178 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:52.546230 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.546303 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.144612 (+1598309us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54001540d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:54.152104 (+  7492us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":388156,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":77,"raft.queue_time_us":112647,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":29450,"spinlock_wait_cycles":410765696,"thread_start_us":924402,"threads_started":3,"wal-append.queue_time_us":389081,"wal-append.run_cpu_time_us":1533,"wal-append.run_wall_time_us":1045278}
I20260709 13:13:54.360477  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 419) took 2402 ms. Trace:
I20260709 13:13:54.361380  8206 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: [5.7-6.8]
I20260709 13:13:54.316586  8721 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:13:54.317382 10939 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:54.313881  8208 rpcz_store.cc:276] 0709 13:13:51.887189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.303266 (+2416077us) service_pool.cc:224] Handling call
0709 13:13:54.303685 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.304126 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.304157 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.304231 (+    74us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:54.304263 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.304320 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.304428 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:54.320446  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 278619us
I20260709 13:13:54.323424 11275 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.321624 11251 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:54.323161  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:54.312803 11260 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.571s	user 0.008s	sys 0.004s
I20260709 13:13:54.328876  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1445) took 2765 ms. Trace:
W20260709 13:13:54.339474 11257 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.484s	user 0.005s	sys 0.004s
W20260709 13:13:54.339838 10940 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.899s	user 0.017s	sys 0.052s
W20260709 13:13:54.330538  8204 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.001s	user 0.000s	sys 0.013s
I20260709 13:13:53.392704 11208 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:13:53.805845 11160 consensus_meta.cc:287] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 2.163s	user 0.008s	sys 0.014s
I20260709 13:13:53.464918 11206 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:54.330664  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1523) took 2443 ms. Trace:
W20260709 13:13:54.368201  8204 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.081s	user 0.000s	sys 0.013s
I20260709 13:13:54.358043  9980 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:54.330787 11043 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.012s	user 0.009s	sys 0.016s
I20260709 13:13:54.362331  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 450) took 2387 ms. Trace:
I20260709 13:13:54.378022  8206 rpcz_store.cc:276] 0709 13:13:51.974563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.360379 (+2385816us) service_pool.cc:224] Handling call
0709 13:13:54.360939 (+   560us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:54.361989 (+  1050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.362264 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.381927 10972 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:54.382664 10972 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.382892  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:54.383178 10972 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:54.384058  8722 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 504) took 1947 ms (client timeout 2307 ms). Trace:
W20260709 13:13:54.384541  8722 rpcz_store.cc:269] 0709 13:13:52.436897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.382416 (+1945519us) service_pool.cc:224] Handling call
0709 13:13:54.383985 (+  1569us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.385649 10972 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:54.388923  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 417) took 2906 ms. Trace:
I20260709 13:13:54.366211  8132 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:13:54.394232  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.385694  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:54.397264 10948 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.396673 11206 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.397957 10948 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:53.982306 11222 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.399744 11206 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:54.400496  8210 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:54.401445 11222 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:54.401590 10948 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:54.362389  8721 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 662) took 2880 ms (client timeout 1831 ms). Trace:
I20260709 13:13:54.404175  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.326133 10888 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:54.360955  8131 rpcz_store.cc:276] 0709 13:13:51.957889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.226845 (+268956us) service_pool.cc:224] Handling call
0709 13:13:52.297021 (+ 70176us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:52.297657 (+   636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.297696 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:52.318491 (+ 20795us) log.cc:844] Serialized 52 byte log entry
0709 13:13:52.501699 (+183208us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:52.501753 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.501841 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.360018 (+1858177us) raft_consensus.cc:1685] finished
0709 13:13:54.360060 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.360286 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2237,"raft.queue_time_us":1199,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":16619,"thread_start_us":85575,"threads_started":1,"wal-append.queue_time_us":1283016}
I20260709 13:13:54.386211 10972 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:13:54.403584  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 1.5849e+06us
W20260709 13:13:54.402972  8721 rpcz_store.cc:269] 0709 13:13:51.481646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.481869 (+   223us) service_pool.cc:224] Handling call
0709 13:13:54.158496 (+2676627us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:54.362301 (+203805us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6957952}
I20260709 13:13:54.422286  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.367633 10940 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:54.389575  8136 rpcz_store.cc:276] 0709 13:13:51.482742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.482889 (+   147us) service_pool.cc:224] Handling call
0709 13:13:51.483563 (+   674us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:51.484133 (+   570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:51.484171 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:52.423677 (+939506us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x7b2c00011f88. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db990fa11 00007f6db991f372 00007f6dbb051a01 00007f6dbb01643a 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2
0709 13:13:52.424392 (+   715us) log.cc:844] Serialized 582 byte log entry
0709 13:13:52.454373 (+ 29981us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:52.454437 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.454539 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.388135 (+1933596us) raft_consensus.cc:1685] finished
0709 13:13:54.388195 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.388651 (+   456us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":695,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":32444,"spinlock_wait_cycles":5984128,"thread_start_us":29555,"threads_started":1,"wal-append.queue_time_us":1244743}
I20260709 13:13:54.366392  8205 rpcz_store.cc:276] 0709 13:13:51.563065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.366699 (+803634us) service_pool.cc:224] Handling call
0709 13:13:52.367056 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.367442 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.367480 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:52.386843 (+ 19363us) log.cc:844] Serialized 26 byte log entry
0709 13:13:53.284155 (+897312us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:53.284193 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:53.284254 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.328361 (+1044107us) raft_consensus.cc:1685] finished
0709 13:13:54.328409 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.328648 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18237,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":5478,"thread_start_us":795329,"threads_started":1,"wal-append.queue_time_us":1208815}
I20260709 13:13:54.455078  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.455468  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.456024  8058 raft_consensus.cc:2393] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:54.455960  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.456460  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:54.437362  8178 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.013s	user 0.000s	sys 0.015s
W20260709 13:13:54.458007  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:54.444159  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 99798.4us
I20260709 13:13:54.457540  8722 raft_consensus.cc:2410] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:13:54.459853  8722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 536) took 2039 ms. Trace:
I20260709 13:13:54.460434  8722 rpcz_store.cc:276] 0709 13:13:52.420128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.385345 (+1965217us) service_pool.cc:224] Handling call
0709 13:13:54.459792 (+ 74447us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:13:54.462590  8178 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.187s	user 0.000s	sys 0.016s
I20260709 13:13:54.375649  8134 rpcz_store.cc:276] 0709 13:13:51.887220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.183701 (+296481us) service_pool.cc:224] Handling call
0709 13:13:52.184007 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.432028 (+248021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.432058 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:52.433450 (+  1392us) log.cc:844] Serialized 26 byte log entry
0709 13:13:52.689620 (+256170us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:52.689671 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.689765 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.330179 (+1640414us) raft_consensus.cc:1685] finished
0709 13:13:54.330223 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.330465 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6414,"raft.queue_time_us":31547,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":3398,"thread_start_us":446813,"threads_started":2,"wal-append.queue_time_us":1212150}
W20260709 13:13:54.462381  8175 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.008s	sys 0.007s
W20260709 13:13:54.469182  8175 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.009s	sys 0.007s
I20260709 13:13:54.471809  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 453) took 2287 ms. Trace:
I20260709 13:13:54.472278  8205 rpcz_store.cc:276] 0709 13:13:52.184432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.463615 (+2279183us) service_pool.cc:224] Handling call
0709 13:13:54.470959 (+  7344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.471389 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.471418 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.471488 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:54.471536 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.471603 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.471717 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.472886 11268 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:54.462086 10888 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.473881  8205 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:54.473654 11268 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.474149 10888 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.475382 11154 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.475908 11154 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:54.476594 10888 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:54.476663 11268 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:54.477649 11268 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.478485 10948 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.479085 10948 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.480171  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 675) took 2532 ms. Trace:
I20260709 13:13:54.481472 10940 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.481280 10948 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:54.482478  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2450) took 2808 ms. Trace:
I20260709 13:13:54.482568 11020 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:54.482858 10948 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.483661 10948 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.484097 10940 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:54.482900  8060 rpcz_store.cc:276] 0709 13:13:51.645837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.082022 (+2436185us) service_pool.cc:224] Handling call
0709 13:13:54.082802 (+   780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.083417 (+   615us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:54.084100 (+   683us) log.cc:844] Serialized 14 byte log entry
0709 13:13:54.311327 (+227227us) log.cc:844] Serialized 14 byte log entry
0709 13:13:54.436566 (+125239us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.436631 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:54.436692 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.436783 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.454522 (+ 17739us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106775,"prepare.queue_time_us":125449,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":7818,"raft.queue_time_us":161662,"thread_start_us":240029,"threads_started":2,"wal-append.queue_time_us":242283}
I20260709 13:13:54.485031  8054 raft_consensus.cc:2368] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9.
W20260709 13:13:54.478502  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.438633  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:54.487133  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.366696 11038 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:54.450412  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 151963us
I20260709 13:13:54.488641  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 455) took 2116 ms. Trace:
I20260709 13:13:54.489115  8205 rpcz_store.cc:276] 0709 13:13:52.369411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.473305 (+2103894us) service_pool.cc:224] Handling call
0709 13:13:54.473635 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.474320 (+   685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.486051 (+ 11731us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.454160 10972 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.492385 10521 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:54.492988 11160 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.493903 10972 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:54.487249 10888 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.494545  8062 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:54.494499  8138 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:54.494800 10888 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.495889 11160 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:54.496255  8696 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:54.496825  8722 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:13:54.496886  8131 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:13:54.497404 11030 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.498440 10682 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:13:54.511370  8722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 500) took 2225 ms. Trace:
I20260709 13:13:54.511756  8722 rpcz_store.cc:276] 0709 13:13:52.283698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.489761 (+2206063us) service_pool.cc:224] Handling call
0709 13:13:54.496375 (+  6614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.508436 (+ 12061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.508667 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.499496  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:54.499650 11038 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:54.499908  9980 consensus_queue.cc:1059] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:54.449433 11283 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.480685  8724 rpcz_store.cc:276] 0709 13:13:51.947285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.420996 (+2473711us) service_pool.cc:224] Handling call
0709 13:13:54.479272 (+ 58276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.479632 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.479678 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.479743 (+    65us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:54.479787 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.479858 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.479985 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.512624  8136 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: [7.5-8.6]
I20260709 13:13:54.513571  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 670) took 2873 ms. Trace:
W20260709 13:13:54.513722  8726 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 676) took 2248 ms (client timeout 2796 ms). Trace:
I20260709 13:13:54.515480 11211 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.516372 11128 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:54.516700 11208 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:54.523967  8726 rpcz_store.cc:269] 0709 13:13:52.265435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.499103 (+2233668us) service_pool.cc:224] Handling call
0709 13:13:54.513665 (+ 14562us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.524176  8206 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:13:54.525563  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.525887  8725 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:13:54.526628  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:54.526970 11030 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.527661 10948 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:54.528705  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1120) took 2103 ms. Trace:
I20260709 13:13:54.526973  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:54.488031 11268 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.436175 11275 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:54.538877  8730 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:13:54.543052  8730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 507) took 1153 ms. Trace:
I20260709 13:13:54.543445  8730 rpcz_store.cc:276] 0709 13:13:53.389435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.526659 (+1137224us) service_pool.cc:224] Handling call
0709 13:13:54.542988 (+ 16329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.546599 11284 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.547268 11275 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:54.547807 11275 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:54.547356 11284 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.548375 11275 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:54.541275  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 452) took 2532 ms. Trace:
I20260709 13:13:54.551534  8206 rpcz_store.cc:276] 0709 13:13:52.002401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.419724 (+2417323us) service_pool.cc:224] Handling call
0709 13:13:54.462235 (+ 42511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.530373 (+ 68138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.535115 (+  4742us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.538311  8135 rpcz_store.cc:276] 0709 13:13:52.425474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.438776 (+ 13302us) service_pool.cc:224] Handling call
0709 13:13:52.439162 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.523050 (+ 83888us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.523085 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:53.385567 (+862482us) log.cc:844] Serialized 26 byte log entry
0709 13:13:54.378031 (+992464us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:54.393388 (+ 15357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.528151 (+134763us) spinlock_profiling.cc:243] Waited 83.6 ms on lock 0x7b54000d20d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:54.528201 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.528240 (+    39us) raft_consensus.cc:1685] finished
0709 13:13:54.528257 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.528532 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192376064,"thread_start_us":184881,"threads_started":1,"wal-append.queue_time_us":239066}
I20260709 13:13:54.529382 11206 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:54.509748  8209 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.554087 11206 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.554924  8209 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:54.555378 11030 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:54.437608 11249 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:54.557693  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69047.7us
I20260709 13:13:54.540890 11208 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:54.562510  8725 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 528) took 2185 ms (client timeout 2109 ms). Trace:
I20260709 13:13:54.455457 11250 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:54.563529  8725 rpcz_store.cc:269] 0709 13:13:52.374186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.469133 (+1094947us) service_pool.cc:224] Handling call
0709 13:13:54.556240 (+1087107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.571923 11211 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:54.577366  8729 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
W20260709 13:13:54.607089  8729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 671) took 2969 ms (client timeout 2192 ms). Trace:
I20260709 13:13:54.582053 11249 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:54.587919 11268 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:54.589515 11057 raft_consensus.cc:3055] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:13:54.589746  8026 connection.cc:256] RPC call timeout handler was delayed by -1.6470508899999459s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2057s for a response
I20260709 13:13:54.485214 11269 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:54.630913  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 134886us
I20260709 13:13:54.590673  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:54.592298 11008 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:54.594745 11250 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:54.594892  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:54.595078  8137 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:54.595660  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:54.597204  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:54.585897 11154 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.600273 11128 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:13:54.601020  8101 connection.cc:256] RPC call timeout handler was delayed by -1.9390326430000187s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
I20260709 13:13:54.601533 11284 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:54.580070 11271 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:13:54.602213  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:54.607659  8729 rpcz_store.cc:269] 0709 13:13:51.638051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.650134 (+ 12083us) service_pool.cc:224] Handling call
0709 13:13:54.177054 (+2526920us) spinlock_profiling.cc:243] Waited 46.7 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:54.607008 (+429954us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107501824}
W20260709 13:13:54.623306  8693 connection.cc:256] RPC call timeout handler was delayed by -1.4001444639998226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
I20260709 13:13:54.523896  8728 rpcz_store.cc:276] 0709 13:13:51.612614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.113182 (+500568us) service_pool.cc:224] Handling call
0709 13:13:52.113575 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.115732 (+  2157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.115773 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:52.123725 (+  7952us) log.cc:844] Serialized 26 byte log entry
0709 13:13:52.416113 (+292388us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b1800349720. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:13:52.543278 (+127165us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:52.543333 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.543437 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.406774 (+1863337us) raft_consensus.cc:1685] finished
0709 13:13:54.417836 (+ 11062us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.485653 (+ 67817us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2522,"raft.queue_time_us":10656,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":7638,"spinlock_wait_cycles":68167424,"thread_start_us":126965,"threads_started":1,"wal-append.queue_time_us":1520429}
I20260709 13:13:54.627163 10948 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:54.557499 11283 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.629079  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:54.629406 10888 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:54.625876  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:54.630447  8026 connection.cc:256] RPC call timeout handler was delayed by -1.5795508900000641s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21320000000000003s for a response
I20260709 13:13:54.631438  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:54.556242 11020 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.637072  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 449) took 2657 ms. Trace:
I20260709 13:13:54.640257  8206 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:54.640655  8693 connection.cc:256] RPC call timeout handler was delayed by -1.3560444639999787s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
W20260709 13:13:54.646982  8693 connection.cc:256] RPC call timeout handler was delayed by -1.1349447070001588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
W20260709 13:13:54.647341  8693 connection.cc:256] RPC call timeout handler was delayed by -1.1349447070001588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
I20260709 13:13:54.630285 11057 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:13:54.648128  8727 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 506) took 1354 ms (client timeout 1607 ms). Trace:
I20260709 13:13:54.648602  8133 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:13:54.648646  8727 rpcz_store.cc:269] 0709 13:13:53.294214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.560842 (+1266628us) service_pool.cc:224] Handling call
0709 13:13:54.648063 (+ 87221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.636567  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:54.647640  8693 connection.cc:256] RPC call timeout handler was delayed by -1.0746447070000613s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
I20260709 13:13:54.645017  8730 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:13:54.640931  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:54.649715  8693 connection.cc:256] RPC call timeout handler was delayed by -1.0746447070000613s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
I20260709 13:13:54.649964  8729 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:13:54.650857  8054 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:54.655423  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:54.655937  8027 leader_election.cc:336] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Timed out: RequestConsensusVote RPC to 127.6.198.68:44573 timed out after 2.194s (SENT)
I20260709 13:13:54.656975  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1449) took 3092 ms. Trace:
W20260709 13:13:54.657900  8101 connection.cc:256] RPC call timeout handler was delayed by -1.6438326429999961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
W20260709 13:13:54.658289  8101 connection.cc:256] RPC call timeout handler was delayed by -1.5394326429998273s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1947s for a response
I20260709 13:13:54.658484 10972 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:54.659664  8728 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: [3.6-8.8]
I20260709 13:13:54.660146 11251 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.660425  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 537) took 2238 ms. Trace:
I20260709 13:13:54.661206 11168 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:54.661932  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 71595.5us
W20260709 13:13:54.662398  8026 connection.cc:256] RPC call timeout handler was delayed by -1.4796508899999026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2243s for a response
W20260709 13:13:54.662881  8026 connection.cc:256] RPC call timeout handler was delayed by -1.4076508900000135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2323s for a response
I20260709 13:13:54.662927 11023 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:54.664330 11040 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:54.664938 11040 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:54.665259  8104 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:54.660862 11251 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.667837 11030 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:54.669426  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.670038  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.670264  8056 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:54.670357  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.670854  8058 raft_consensus.cc:2368] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:54.671216  8055 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:54.671339  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.656853  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.671748  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.672222  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.672390 11251 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:54.672466  8059 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:54.671367  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.673983  8063 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:54.674786  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.675669  8054 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:54.675851  8205 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:54.658603  8101 connection.cc:256] RPC call timeout handler was delayed by -1.4080326429998422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
I20260709 13:13:54.661852 11168 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.677793 11022 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:54.678386 11022 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:54.679611  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:54.680677 11283 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:54.681326 10948 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.668162  9980 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:13:54.677067  8101 connection.cc:256] RPC call timeout handler was delayed by -1.349532642999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
I20260709 13:13:54.659305 10972 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:54.683678  8101 connection.cc:256] RPC call timeout handler was delayed by -1.3171326429999226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2194s for a response
W20260709 13:13:54.684036  8101 connection.cc:256] RPC call timeout handler was delayed by -1.1904334569999264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
I20260709 13:13:54.672744  8058 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:54.660887  8728 rpcz_store.cc:276] 0709 13:13:52.421627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.644658 (+2223031us) service_pool.cc:224] Handling call
0709 13:13:54.659325 (+ 14667us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:13:54.660138 (+   813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.660348 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.674119  8059 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 13:13:54.684376  8101 connection.cc:256] RPC call timeout handler was delayed by -1.1308326429998488s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24009999999999998s for a response
W20260709 13:13:54.694483  8101 connection.cc:256] RPC call timeout handler was delayed by -1.121498832999805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
W20260709 13:13:54.694869  8101 connection.cc:256] RPC call timeout handler was delayed by -1.0298213280000255s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2021s for a response
I20260709 13:13:54.666004  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:54.699509  8696 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:54.703281  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:54.702186  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 56829.3us
I20260709 13:13:54.704082  9980 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:54.646504  8203 rpcz_store.cc:276] 0709 13:13:51.973883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.315635 (+2341752us) service_pool.cc:224] Handling call
0709 13:13:54.316038 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:54.317113 (+  1075us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:13:54.393887 (+ 76774us) log.cc:844] Serialized 14 byte log entry
0709 13:13:54.544817 (+150930us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:54.545783 (+   966us) log.cc:844] Serialized 26 byte log entry
0709 13:13:54.545978 (+   195us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:13:54.546032 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.546175 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.630846 (+ 84671us) raft_consensus.cc:1685] finished
0709 13:13:54.630863 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.631467 (+   604us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":134852,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":69,"raft.queue_time_us":6327,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":7810,"thread_start_us":115588,"threads_started":2,"wal-append.queue_time_us":167137}
W20260709 13:13:54.713922  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 47573.8us
I20260709 13:13:54.715565  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.717293  8056 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:54.695076 11023 raft_consensus.cc:697] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:13:54.723492  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 79553.8us
I20260709 13:13:54.715206  8060 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:54.723009  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:54.706662 11251 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.727120  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.725076  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.666563 10682 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:13:54.709592 10939 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:54.657359  8204 rpcz_store.cc:276] 0709 13:13:51.564227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.921745 (+1357518us) service_pool.cc:224] Handling call
0709 13:13:52.922213 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.922762 (+   549us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:13:53.247983 (+325221us) log.cc:844] Serialized 14 byte log entry
0709 13:13:54.486118 (+1238135us) log.cc:844] Serialized 14 byte log entry
0709 13:13:54.507162 (+ 21044us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:54.525492 (+ 18330us) log.cc:844] Serialized 26 byte log entry
0709 13:13:54.525692 (+   200us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:54.525727 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.525821 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.656691 (+130870us) raft_consensus.cc:1685] finished
0709 13:13:54.656709 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.656910 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1105,"prepare.queue_time_us":382080,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":76,"raft.queue_time_us":2537,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":186921,"thread_start_us":1169119,"threads_started":2,"wal-append.queue_time_us":1177649}
I20260709 13:13:54.722393  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.726085  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.726512  8104 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:54.733348 11162 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:54.733167 11251 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.734422 11057 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.735661 11251 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:54.735999 11163 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.736565 11163 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.733180 10730 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:54.737394  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.737525 10730 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.738426  8130 raft_consensus.cc:2410] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:54.738515 11248 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.739553 11269 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:54.739321 11248 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.739872 10730 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:54.735670 11057 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:54.738426  8057 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:13:54.736039  8055 raft_consensus.cc:2410] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:13:54.733106 11005 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:13:54.743868  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.744844  8054 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
W20260709 13:13:54.746294  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 90401.2us
I20260709 13:13:54.746526 11188 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:54.747061  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 505) took 1875 ms. Trace:
I20260709 13:13:54.748267  8058 raft_consensus.cc:2410] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:13:54.748107  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:54.748813  8138 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:54.747445  8728 rpcz_store.cc:276] 0709 13:13:52.871657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.702284 (+1830627us) service_pool.cc:224] Handling call
0709 13:13:54.725637 (+ 23353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.746741 (+ 21104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.746764 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.746816 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:54.746851 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.746915 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.747015 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.747221 11283 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.761584  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:54.762295  8203 raft_consensus.cc:2410] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:13:54.666177 11040 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.772614  8131 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:13:54.780673  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.779151 11288 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:54.779704  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.787730  8055 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:13:54.785730  8104 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:54.742774 11208 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.792788  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.756208 11283 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.747756 10939 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.791126  9980 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:54.804529  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:54.852147  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 186325us
I20260709 13:13:54.808756  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.807017  8134 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:13:54.804034  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 530) took 2419 ms. Trace:
I20260709 13:13:54.858906  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.787068 10682 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:54.856061  8726 rpcz_store.cc:276] 0709 13:13:52.376629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.541131 (+2164502us) service_pool.cc:224] Handling call
0709 13:13:54.553638 (+ 12507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.554149 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:13:54.624393 (+ 70244us) log.cc:844] Serialized 14 byte log entry
0709 13:13:54.795626 (+171233us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.795686 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:13:54.795739 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.795805 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.795911 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32847,"prepare.queue_time_us":73558,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"thread_start_us":125248,"threads_started":2,"wal-append.queue_time_us":152452}
I20260709 13:13:54.859629  8063 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:13:54.864171  8209 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
W20260709 13:13:54.867611  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 89221.2us
I20260709 13:13:54.877897  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.864527  8209 pending_rounds.cc:85] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 5
I20260709 13:13:54.888329  8132 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:54.890405 11162 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:54.893537  8056 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:13:54.898010  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.896345  8135 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:54.893977  8056 pending_rounds.cc:85] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 8
I20260709 13:13:54.900874  8056 pending_rounds.cc:107] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:13:54.901221  8056 raft_consensus.cc:2889] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:54.902263  8209 pending_rounds.cc:107] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:13:54.902706  8209 raft_consensus.cc:2889] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:54.903635  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1452) took 3018 ms. Trace:
I20260709 13:13:54.904022  8209 rpcz_store.cc:276] 0709 13:13:51.885534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.156934 (+2271400us) service_pool.cc:224] Handling call
0709 13:13:54.157417 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.903279 (+745862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.903532 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.859228 11248 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:54.868091 11040 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:54.863399 10682 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:54.909013  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.863164 11163 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:54.912338  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 131260us
I20260709 13:13:54.886988 11038 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:13:54.907382  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.915611  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.895943 10948 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:54.914719  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 150009us
I20260709 13:13:54.907276 11154 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:54.913586 10682 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:54.926522 11202 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:13:54.926287  8138 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:13:54.927702 10939 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:54.928325 11023 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.941969  8137 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:54.909232 10972 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:54.949054  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:54.946478  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 526) took 2568 ms. Trace:
I20260709 13:13:54.978016  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:54.948589  8027 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:54.971184  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:54.980333  8135 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:54.972946  8205 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:13:54.985720 11020 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:54.992384 11302 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:13:54.995860 11028 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:13:54.916134 11005 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:13:55.001257  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.001997  8134 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:13:55.003263  8134 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:55.004244 11208 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:55.004724 10940 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.005395 10940 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:55.008131  8060 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:13:55.011286  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:54.977298  8724 rpcz_store.cc:276] 0709 13:13:52.371755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.528338 (+2156583us) service_pool.cc:224] Handling call
0709 13:13:54.528804 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:54.624813 (+ 96009us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.624862 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:54.625985 (+  1123us) log.cc:844] Serialized 26 byte log entry
0709 13:13:54.772022 (+146037us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:54.772058 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.772198 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.939517 (+167319us) raft_consensus.cc:1685] finished
0709 13:13:54.939535 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.939665 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2432,"raft.queue_time_us":95699,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":142,"thread_start_us":143869,"threads_started":2,"wal-append.queue_time_us":240419}
I20260709 13:13:55.012647  8209 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:55.014779  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:55.015864  8203 raft_consensus.cc:2410] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:13:55.016885  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:55.019222 11292 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
W20260709 13:13:55.022548 11298 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:13:54.971283  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 451) took 2996 ms. Trace:
I20260709 13:13:54.891883 11162 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:54.975296  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:55.042956  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:55.023284  8026 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:54.978870  8054 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
W20260709 13:13:55.051581  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 73948.1us
I20260709 13:13:55.059790  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.060667  8063 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:55.062184  8055 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:13:55.067158 11251 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:13:55.080389  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.080623 11288 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:55.081060 11288 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:55.093428  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:55.094568  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 154422us
W20260709 13:13:55.095645 11296 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 13:13:55.096321 11154 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:55.096868 11154 raft_consensus.cc:697] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:55.097124  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.097870 11154 consensus_queue.cc:237] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:55.099443  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.100605  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.101056  8058 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:55.101428  8055 raft_consensus.cc:2393] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:55.102484  8205 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:13:55.103073  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.103933  8063 raft_consensus.cc:2410] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 8 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:13:55.104646 10939 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:55.105611  8026 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:55.106622 10939 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:55.107144 10939 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:55.107658 10939 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:55.111357  8722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 501) took 2827 ms. Trace:
I20260709 13:13:55.111851  8722 rpcz_store.cc:276] 0709 13:13:52.284026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.514358 (+2230332us) service_pool.cc:224] Handling call
0709 13:13:54.514777 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.658878 (+144101us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:13:54.701807 (+ 42929us) log.cc:844] Serialized 14 byte log entry
0709 13:13:54.967937 (+266130us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.967987 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:54.968027 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.111120 (+143093us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54004d2e54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:55.111169 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:55.111275 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75030,"prepare.queue_time_us":46655,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":107375,"spinlock_wait_cycles":334812672,"thread_start_us":174638,"threads_started":2,"wal-append.queue_time_us":92997}
I20260709 13:13:55.115883  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.116559  8054 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:55.119009  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:55.119179  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.119920  8060 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:55.130239 11269 raft_consensus.cc:3055] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:55.130686 11269 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:55.131587 11269 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.132347  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:55.133054 11269 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:55.102447  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.134882  8055 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:55.135147  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2452) took 1050 ms. Trace:
I20260709 13:13:55.135527  8062 rpcz_store.cc:276] 0709 13:13:54.084952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.162940 (+ 77988us) service_pool.cc:224] Handling call
0709 13:13:54.163380 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.495161 (+331781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.134918 (+639757us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b5400051754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:55.135074 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":760699776}
I20260709 13:13:55.137892  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:55.146709  8722 raft_consensus.cc:2376] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9. Re-sending same reply.
I20260709 13:13:55.147428  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1132) took 1081 ms. Trace:
I20260709 13:13:55.147797  8133 rpcz_store.cc:276] 0709 13:13:54.065994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.435018 (+369024us) service_pool.cc:224] Handling call
0709 13:13:54.466512 (+ 31494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.649104 (+182592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.147200 (+498096us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b540023d2d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:55.147356 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399834880}
I20260709 13:13:55.151012 10940 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:55.151731  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 209600us
I20260709 13:13:55.151957  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1142) took 1082 ms. Trace:
I20260709 13:13:55.152487  8027 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:55.152448  8132 rpcz_store.cc:276] 0709 13:13:54.069935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.400981 (+331046us) service_pool.cc:224] Handling call
0709 13:13:54.655135 (+254154us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b58001c3e98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:55.151729 (+496594us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b5400043654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:55.151889 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":515802112}
I20260709 13:13:55.159847  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.160111  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.161023  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1129) took 1096 ms. Trace:
I20260709 13:13:55.161473  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.162142  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2578) took 3525 ms. Trace:
I20260709 13:13:55.162189 11206 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:55.162657  8211 rpcz_store.cc:276] 0709 13:13:51.636864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.143144 (+2506280us) service_pool.cc:224] Handling call
0709 13:13:54.143687 (+   543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:54.325344 (+181657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.161913 (+836569us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400208ad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:55.162081 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":412068224}
I20260709 13:13:55.163147  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.170759  8176 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:55.171753  8176 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:55.141274 11283 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:55.172732 11283 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:55.142066 11199 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:55.173377 10948 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:55.174037 10948 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.174429 10948 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:55.174765  8210 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55942 (request call id 457) took 2766 ms (client timeout 2058 ms). Trace:
W20260709 13:13:55.175243  8210 rpcz_store.cc:269] 0709 13:13:52.408379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.547025 (+138646us) service_pool.cc:224] Handling call
0709 13:13:55.174506 (+2627481us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b54002c8b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:55.174681 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4259075328}
W20260709 13:13:55.176131  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.176288  8207 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:55.177996  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2582) took 3230 ms. Trace:
I20260709 13:13:55.144274 11168 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:55.173432 11199 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:55.178311  8207 rpcz_store.cc:276] 0709 13:13:51.947749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.309667 (+2361918us) service_pool.cc:224] Handling call
0709 13:13:55.176089 (+866422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:55.176700 (+   611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.177822 (+  1122us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b54002c8b50. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:55.177942 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1806051072}
W20260709 13:13:55.180721 11174 consensus_meta.cc:287] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.727s	user 0.013s	sys 0.005s
I20260709 13:13:55.181231 11174 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:55.179685 11199 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.144160  8026 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:55.173465 11020 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:55.182740 11020 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:55.182813 11168 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:55.183220 11020 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:55.183315 11022 consensus_meta.cc:287] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.504s	user 0.017s	sys 0.025s
I20260709 13:13:55.183424  8026 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:55.183882 11022 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.184532 11188 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:55.184952 11188 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:55.185325 11188 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:55.186369 10948 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.186707  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:55.189116  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 50852.3us
I20260709 13:13:55.189352 10948 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:55.190840  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:55.192430  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:55.194702  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 48406.8us
I20260709 13:13:55.196657  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:55.170759  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 395344us
I20260709 13:13:55.161338  8129 rpcz_store.cc:276] 0709 13:13:54.064753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.348138 (+283385us) service_pool.cc:224] Handling call
0709 13:13:55.159919 (+811781us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b58001c3e98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:13:55.160961 (+  1042us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7398272}
W20260709 13:13:55.204792  8206 consensus_meta.cc:287] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.521s	user 0.013s	sys 0.009s
I20260709 13:13:55.205127  8206 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:55.149770 10939 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:55.219328  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.222328  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:55.222628  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.223605  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:55.226298  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:55.226907  8729 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:55.182274 11199 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:55.230803  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.232512  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:55.233865  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.183445 11168 raft_consensus.cc:697] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:55.239163  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.239338 11303 consensus_queue.cc:1059] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:13:55.240278 11303 raft_consensus.cc:3055] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:55.240586 11303 raft_consensus.cc:740] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:55.241406 11303 consensus_queue.cc:260] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.027292  8208 rpcz_store.cc:276] 0709 13:13:51.975260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.368696 (+2393436us) service_pool.cc:224] Handling call
0709 13:13:54.369116 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:54.379693 (+ 10577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:54.379726 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:54.484678 (+104952us) log.cc:844] Serialized 26 byte log entry
0709 13:13:54.636776 (+152098us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:54.681244 (+ 44468us) log.cc:844] Serialized 14 byte log entry
0709 13:13:54.681491 (+   247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.681559 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.971018 (+289459us) raft_consensus.cc:1685] finished
0709 13:13:54.971040 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.971211 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3259,"prepare.queue_time_us":44766,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":2198,"raft.run_cpu_time_us":438,"raft.run_wall_time_us":23780,"thread_start_us":147035,"threads_started":1,"wal-append.queue_time_us":236848}
I20260709 13:13:55.247314  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.247983  8132 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
W20260709 13:13:55.246954  7956 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34146 (request call id 257) took 21461 ms (client timeout 15000 ms). Trace:
I20260709 13:13:55.253767  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.255486  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.254555  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:55.252540  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:55.259958  8207 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:55.260746  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:55.261332  8721 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:13:55.261439 11288 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.262282  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.263928  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.262900  8056 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:55.265785  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.268507  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.269284  8059 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:55.266561  8136 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:55.241398  7961 catalog_manager.cc:5697] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:55.275038  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.275753  8133 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:55.277935  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.278651  8055 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:55.279311  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.280197  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.281761  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:55.283027  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:55.285293  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 391918us
W20260709 13:13:55.285967  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.286577  8131 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:55.288261  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.292249  7961 catalog_manager.cc:5697] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 1 to 5, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 5 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:55.294792  8060 raft_consensus.cc:2393] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:55.277372  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.295545  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.296159  8059 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:55.297178  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.285772  8057 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:55.300108  8054 raft_consensus.cc:2410] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:13:55.295360  8062 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:55.301218  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.294130  7961 catalog_manager.cc:5697] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
W20260709 13:13:55.304260  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 217694us
I20260709 13:13:55.304924  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:55.305998  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.305467  8133 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:55.310169  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.311582  8056 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:55.311836 11168 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:55.313582 10915 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:55.314985 11174 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:55.316016 11174 raft_consensus.cc:3055] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:55.316372 11174 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:55.316681  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.317463  8060 raft_consensus.cc:2368] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:55.317240 11174 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.318737  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.319356  8063 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:55.320175  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.320869  8063 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:55.316390  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:55.322275 11275 consensus_meta.cc:287] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.773s	user 0.011s	sys 0.005s
I20260709 13:13:55.322706 11275 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.324069  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:55.324970 11275 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:55.325207  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:55.288971 11303 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:55.322726  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.318300  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:55.326938  8129 raft_consensus.cc:2368] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9.
W20260709 13:13:55.252527  7956 rpcz_store.cc:269] 0709 13:13:33.782891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:33.783208 (+   317us) service_pool.cc:224] Handling call
0709 13:13:55.243543 (+21460335us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:45.193331 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:45.234425 (+ 41094us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:45.234475 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:13:45.234507 (+    32us) tablet.cc:662] Decoding operations
0709 13:13:45.325398 (+ 90891us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:45.338031 (+ 12633us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:45.340493 (+  2462us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:13:45.400088 (+ 59595us) tablet.cc:701] Row locks acquired
0709 13:13:45.400135 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:13:45.400287 (+   152us) write_op.cc:270] Start()
0709 13:13:45.400429 (+   142us) write_op.cc:276] Timestamp: P: 1783602825400224 usec, L: 0
0709 13:13:45.400452 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:13:45.401144 (+   692us) log.cc:844] Serialized 5534 byte log entry
0709 13:13:46.390258 (+989114us) op_driver.cc:464] REPLICATION: finished
0709 13:13:47.912494 (+1522236us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6db7f0c0ac 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbea4cef0
0709 13:13:48.109887 (+197393us) write_op.cc:301] APPLY: starting
0709 13:13:48.135792 (+ 25905us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:48.146082 (+ 10290us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:48.146118 (+    36us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:48.153299 (+  7181us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:48.153604 (+   305us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 13:13:48.153654 (+    50us) write_op.cc:312] APPLY: finished
0709 13:13:48.154050 (+   396us) log.cc:844] Serialized 101 byte log entry
0709 13:13:48.273685 (+119635us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:48.285815 (+ 12130us) write_op.cc:454] Released schema lock
0709 13:13:48.294301 (+  8486us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:48.319017 (+ 24716us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:48.319055 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1470680,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":1677740,"apply.run_cpu_time_us":20793,"apply.run_wall_time_us":212472,"num_ops":14,"prepare.queue_time_us":129262,"prepare.run_cpu_time_us":18682,"prepare.run_wall_time_us":302465,"raft.queue_time_us":62611,"raft.run_cpu_time_us":49940,"raft.run_wall_time_us":1712044,"replication_time_us":989595,"spinlock_wait_cycles":7319168,"thread_start_us":1788662,"threads_started":5,"wal-append.queue_time_us":186495,"wal-append.run_cpu_time_us":47101,"wal-append.run_wall_time_us":2970971}]]}
I20260709 13:13:55.326277  8723 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:55.335547  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.317489  8029 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:55.331364 11275 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:13:55.342619  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.301537 11022 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:55.343896  7948 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b578d88c1c1f4ca58b7291fed4909f09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica d9cd2999aa314b39997b423ec1ae509e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:55.347623  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:55.348062  8723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 532) took 2967 ms. Trace:
I20260709 13:13:55.348400  8207 raft_consensus.cc:2393] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:55.332235  7961 catalog_manager.cc:5697] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 4 to 5, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 5 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:55.351099 10939 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:55.325627 11176 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:55.348421  8723 rpcz_store.cc:276] 0709 13:13:52.380178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.644065 (+2263887us) service_pool.cc:224] Handling call
0709 13:13:54.644476 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:55.329059 (+684583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.347849 (+ 18790us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54004d21d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:13:55.347999 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1567657216}
I20260709 13:13:55.356443  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.354707  8729 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:13:55.358979  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.352402  7961 catalog_manager.cc:5697] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 2 to 6, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 6 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:55.360028  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:55.367794  8057 raft_consensus.cc:2393] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:55.361184  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.372516  8176 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:55.332799 10948 raft_consensus.cc:2764] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:13:55.376713  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 59340.7us
I20260709 13:13:55.379416  8133 raft_consensus.cc:2368] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:55.367012  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:55.383170  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:55.385803  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.386034  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:55.382021  8210 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:55.385638  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:55.383296  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:55.362665 11174 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:55.357993 11288 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:55.367044 10915 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.385681  7961 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 4 to 6, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 6 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:55.389292  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:55.368220 10939 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:55.390815  8212 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:55.393210  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.330967  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.391592  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:55.392905  8029 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:55.404110  7961 catalog_manager.cc:5697] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:13:55.405251  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.408655  8211 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:13:55.403927  8208 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:55.413456  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:55.414422  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:55.415076  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.387965 11206 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:55.402486 10939 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:55.422596 10939 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:55.423614  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:55.424214  8212 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:55.410589 10948 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:55.426599 10948 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.426987 10948 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:55.430475 11288 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:55.416420 11208 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:55.431707 11188 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:55.432336 11208 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.432780  8058 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:55.434453 11040 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:55.434831 11040 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:55.435196 11040 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:55.435940  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:55.437029  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:55.437517  8210 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:55.422163  8209 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: [7.8-8.9]
I20260709 13:13:55.443068  7961 catalog_manager.cc:5697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 3 to 5, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 5 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:55.444707  7961 catalog_manager.cc:5697] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 4 to 6, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 6 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:55.449851  8212 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:13:55.434065 11188 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:55.455492  8209 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:55.493189 11023 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:55.495116  8104 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:55.497975 11023 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:13:55.499819  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 112633us
I20260709 13:13:55.500594  8029 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:55.501808  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:55.505843  8129 raft_consensus.cc:2410] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:13:55.506707  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 140106us
I20260709 13:13:55.508245  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:55.508931  8210 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 5.
I20260709 13:13:55.510524 11028 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:55.511904 11028 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:55.447086 11208 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:55.403963 10915 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:55.510022  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:55.516613  8696 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:55.515816  7961 catalog_manager.cc:5697] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:55.517270  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:55.517827  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 265416us
I20260709 13:13:55.531348  8104 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:55.533593 10972 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:55.536324 11303 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:55.544814 11005 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:55.553117  7961 catalog_manager.cc:5697] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 5 to 6, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 6 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:55.556123  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:55.539520 11022 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:55.558081 11022 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.558490 11022 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:55.485733 11288 consensus_queue.cc:1059] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
W20260709 13:13:55.562397  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.565330  8027 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:55.566912 10940 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:55.537132 11020 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.568681  8205 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:55.569752 11020 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:55.572157 10915 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:55.556805 11303 raft_consensus.cc:697] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:55.573031 11040 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:55.573462 11303 consensus_queue.cc:237] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.549603 11008 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:55.576301 11008 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.576670 11008 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:55.593291  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:55.610232  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 60188.8us
W20260709 13:13:55.614537  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 104563us
W20260709 13:13:55.615607  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 115081us
W20260709 13:13:55.679540  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:55.708546  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 144571us
I20260709 13:13:55.711208  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:55.712016  8730 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:55.716785  7961 catalog_manager.cc:5697] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 2 to 6, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 6 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:55.721881  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 89932.5us
I20260709 13:13:55.725208 10939 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.731369  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:55.733670 11030 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:55.709381 11202 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:55.708207 11199 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:13:55.762275  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 153241us
I20260709 13:13:55.765902  8212 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:13:55.768314  8029 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:55.737954 11030 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:55.772365  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 60384.3us
W20260709 13:13:55.758872 10682 consensus_meta.cc:287] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.762s	user 0.012s	sys 0.004s
W20260709 13:13:55.770056  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 353322us
I20260709 13:13:55.771384  7961 catalog_manager.cc:5697] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:55.755887 11162 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:55.767726 11199 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.805205 11040 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:55.792753 11317 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:13:55.807248  7961 catalog_manager.cc:5697] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 2 to 5, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 5 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:55.809974 11199 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:55.773788 11206 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:55.793375 11162 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:55.835814  8693 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:55.818544 11283 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:55.801719 10972 consensus_queue.cc:1059] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:13:55.801592 11269 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:13:55.792346 10682 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:55.836839 11162 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:55.771476 11202 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.848539  7961 catalog_manager.cc:5697] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 2 to 7, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 7 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:55.845029 11283 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:55.851956 11290 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:55.871465  8131 raft_consensus.cc:2393] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 5: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:55.872035 11028 raft_consensus.cc:3055] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:13:55.882272  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.874711 11283 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:55.876032 11290 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.870515 10948 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.886875 11028 raft_consensus.cc:740] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:13:55.904183 11316 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:13:55.905069 11319 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:13:55.905369 11202 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:55.906318  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 513) took 1222 ms. Trace:
I20260709 13:13:55.906692  8727 rpcz_store.cc:276] 0709 13:13:54.683945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:54.705328 (+ 21383us) service_pool.cc:224] Handling call
0709 13:13:54.716831 (+ 11503us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:55.220305 (+503474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:55.220334 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:55.242506 (+ 22172us) log.cc:844] Serialized 52 byte log entry
0709 13:13:55.485770 (+243264us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:55.485829 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:55.485928 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:55.906091 (+420163us) raft_consensus.cc:1685] finished
0709 13:13:55.906120 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:55.906261 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73426,"raft.queue_time_us":66788,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":68511,"thread_start_us":665227,"threads_started":2,"wal-append.queue_time_us":515578}
I20260709 13:13:55.908056 10948 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:55.909324 10939 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:55.909452 11327 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:13:55.910411 11327 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:55.910661 10682 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:55.910768 11327 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:55.911742 11327 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.913007 11327 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:55.915202 11030 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:55.874509 11313 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:13:55.873000  7961 catalog_manager.cc:5697] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:55.923575  8730 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:13:55.893872 11290 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:55.914836 11308 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.326s	user 0.000s	sys 0.001s
I20260709 13:13:55.892982 11020 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
W20260709 13:13:55.933923  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 184926us
W20260709 13:13:55.935245  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 156160us
I20260709 13:13:55.940626  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.941524  8130 raft_consensus.cc:2393] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:55.947896  8730 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:55.947458  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.949378  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:55.949476  8136 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:13:55.891274 11206 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:55.947962  7961 catalog_manager.cc:5697] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 5 to 6, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 6 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:55.955670  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.960491  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:13:55.958328  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 209817us
I20260709 13:13:55.962545  7961 catalog_manager.cc:5697] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:55.947533 11154 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:55.969228  8104 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:55.978442 11329 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:13:55.980183 11336 consensus_queue.cc:1059] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:55.982095  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:55.963457 10939 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:55.982863 10939 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:55.984973 10939 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:55.985545  8138 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:13:55.985895  8138 pending_rounds.cc:85] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 8
I20260709 13:13:55.986106  8138 pending_rounds.cc:107] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:13:55.986320  8138 raft_consensus.cc:2889] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:55.988317  8054 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:13:55.984993  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:55.994571  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:55.995229  8135 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:55.997324 11008 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:55.998677  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:55.999392  8132 raft_consensus.cc:2393] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:56.000381 11008 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:56.006417  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 245688us
W20260709 13:13:56.011644  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 177663us
I20260709 13:13:55.963842 11168 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:56.016593  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 195427us
I20260709 13:13:56.019042  8055 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:56.020485  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:56.023120  7961 catalog_manager.cc:5697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 1 to 4, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 4 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:13:56.016763 11023 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:55.948179 11028 consensus_queue.cc:260] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:56.000972 11022 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:56.029161  8211 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:56.032222  7961 catalog_manager.cc:5697] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 4 to 5, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 5 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:56.015811 11030 consensus_queue.cc:1059] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:13:56.053177 11022 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:56.060848  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:56.062860  8029 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:56.005900 11154 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:56.074429  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:56.078017  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 53157.2us
W20260709 13:13:56.076406 11174 consensus_meta.cc:287] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.004s
I20260709 13:13:56.022454 11168 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:56.075711 11028 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:56.097935  8133 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
W20260709 13:13:56.110597  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 90264.4us
W20260709 13:13:56.111231  8693 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 2.058s (SENT) [suppressed 57 similar messages]
I20260709 13:13:56.112401  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:56.112749  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 89876.3us
I20260709 13:13:56.113409  8101 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:56.113833  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:56.082657 11335 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:56.126724  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:56.129151  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:56.130471  8029 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:56.134235 11174 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:56.102370 11023 raft_consensus.cc:697] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:56.149802 11023 consensus_queue.cc:237] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:56.076210 11290 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:56.157562  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 44215.1us
I20260709 13:13:56.158542  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:56.159718  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:56.153178 10940 raft_consensus.cc:3055] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:56.160463  8723 raft_consensus.cc:2376] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9. Re-sending same reply.
I20260709 13:13:56.087461 11333 consensus_queue.cc:1059] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:13:56.164431  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:56.166664  8722 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:13:56.166977 11168 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:56.172896  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.185209  8727 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:13:56.187556  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:56.192087  8730 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:56.193090  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:56.188493  8723 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:13:56.193785  8727 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 6: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
W20260709 13:13:56.194658  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 129622us
I20260709 13:13:56.161715 11336 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:56.172299 11283 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:56.174301 10940 raft_consensus.cc:740] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:56.199369  8027 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:56.202226  8205 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:56.199712  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:56.202934  8209 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:56.183465 11208 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:56.203778  8212 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:13:56.204020  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:56.201327  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:56.206130  8210 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:13:56.204469  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 74134.6us
I20260709 13:13:56.207224  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:56.207396 10940 consensus_queue.cc:260] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:56.208936  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:56.211467  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:56.216322  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:56.216874  8205 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:56.200284 11336 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:56.246454  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:56.247123  8210 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 5.
I20260709 13:13:56.206485 11168 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:56.258882  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:56.249837  8723 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:56.261423 11154 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:56.224670 11336 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:56.246969 11030 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:56.263257 11030 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:56.263669 11030 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:56.266671  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:56.267230  8206 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:56.269369  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:56.269904  8059 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:13:56.270705  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:56.270817  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:56.271775  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:56.271859  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:56.272228  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:56.273012 11347 log.cc:927] Time spent T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:13:56.273730  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:56.270727  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:56.274951  8063 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:56.275610  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:56.277348 11283 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:56.277913 11283 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:56.273020  8061 raft_consensus.cc:2410] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:13:56.278411 11341 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:13:56.278456 11283 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:56.279618 11337 raft_consensus.cc:3055] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:56.279992 11337 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:56.280325  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:56.281443  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:56.281808  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.282539  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:56.274906  8055 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:13:56.272902  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:56.286654  8054 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:13:56.288751  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:56.280694 11283 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:56.260421 11168 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.297192 11168 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:56.299979  8206 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:13:56.301155 11154 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:13:56.303364 11325 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.525s	user 0.004s	sys 0.000s
I20260709 13:13:56.304047  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:56.321416  8205 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:13:56.294544 11028 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:56.290742 10940 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:56.328871  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:56.331458  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:56.332167  8061 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:56.334592  8054 raft_consensus.cc:2410] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:13:56.337340  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:56.338117  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 56735us
I20260709 13:13:56.338864  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:56.340646  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:56.347872  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:56.348206  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:56.341732  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 66206.7us
I20260709 13:13:56.330440 11275 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:56.357569  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:56.326982 11333 consensus_queue.cc:1059] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:13:56.361670 11220 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:13:56.362849 11354 log.cc:927] Time spent T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:13:56.363418 11038 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:56.364675 11336 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:56.335870 11348 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.031s
W20260709 13:13:56.371222 11346 log.cc:927] Time spent T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:13:56.372661 10740 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
W20260709 13:13:56.377130 11162 consensus_meta.cc:287] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.508s	user 0.003s	sys 0.006s
W20260709 13:13:56.377813 11344 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:13:56.378235 11306 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:56.341295 10972 raft_consensus.cc:3055] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:56.384173 11290 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:56.388064 11199 consensus_meta.cc:287] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.001s
I20260709 13:13:56.394310  8029 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:56.358377  8061 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:13:56.411283  8055 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:13:56.351759 11275 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:56.414978  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:56.429988  8054 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
W20260709 13:13:56.415947  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 90972us
I20260709 13:13:56.417143  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:56.336084 10939 consensus_queue.cc:1059] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:13:56.431586  8063 raft_consensus.cc:2376] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10. Re-sending same reply.
I20260709 13:13:56.377817 11168 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:56.422653 11288 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:56.425501  8102 leader_election.cc:336] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Timed out: RequestConsensusVote RPC to 127.6.198.68:44573 timed out after 1.831s (SENT)
I20260709 13:13:56.422747 10972 raft_consensus.cc:740] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:56.423233  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 75418.4us
I20260709 13:13:56.436237  8104 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:56.412665 11353 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:13:56.440054  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:56.442235  8061 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:56.443173 11336 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:56.432952 11275 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:56.448855  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2586) took 1099 ms. Trace:
I20260709 13:13:56.448064 10972 consensus_queue.cc:260] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:56.450889 10972 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:56.453703  8104 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:56.448899 11008 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:56.423127 11162 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:56.462221 11162 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:56.462495 11348 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:13:56.464035  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 445903us
I20260709 13:13:56.464643  8176 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:56.465346  8102 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:56.422439 11038 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:56.475493  8059 raft_consensus.cc:2368] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10.
I20260709 13:13:56.476547  8061 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:13:56.476984 11348 raft_consensus.cc:697] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:56.475513 11008 raft_consensus.cc:697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:56.478719 11339 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:13:56.484302  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 90067.7us
I20260709 13:13:56.484340  8102 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:56.477819 11358 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:56.432400 11199 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:56.494144  8102 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:56.494730  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.494228  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 421257us
I20260709 13:13:56.496785  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:56.497773  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.502897  8727 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:56.504199  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:56.485875 11368 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:56.579661  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1158) took 1283 ms. Trace:
I20260709 13:13:56.580003  8131 rpcz_store.cc:276] 0709 13:13:55.296691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.330543 (+ 33852us) service_pool.cc:224] Handling call
0709 13:13:56.512002 (+1181459us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b54001a9854. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:56.579599 (+ 67597us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070380416}
W20260709 13:13:56.648305  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 152937us
W20260709 13:13:56.581516  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 87433.7us
W20260709 13:13:56.612154  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 107361us
I20260709 13:13:56.647869  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:56.611145 11206 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:56.575320 11154 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:56.512699 11327 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:56.658653 11339 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:56.662917  8102 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:56.662959  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:56.668262  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:56.673265  8055 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:56.663926  8061 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:56.492273 11336 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:13:56.590934  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.691138  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:56.692415  8129 raft_consensus.cc:2410] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:13:56.702776  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:56.702989  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:56.610530 11162 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.580261 11077 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:56.709486  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:56.710239  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:56.610272 11208 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:56.667023 11154 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:56.713155 11370 consensus_queue.cc:1059] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.227s
I20260709 13:13:56.713630  9980 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:13:56.714061  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 217300us
I20260709 13:13:56.714663  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:56.666680 11206 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.719493  8136 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:13:56.511361 11188 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.193s
W20260709 13:13:56.743152 11367 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:13:56.721306  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:56.748473 11030 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:56.722559  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:56.730760  8055 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
W20260709 13:13:56.751358  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 88460.7us
I20260709 13:13:56.702642 11040 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:56.751523  8133 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:13:56.736204  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:56.732240  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:56.742831  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:56.670600 11340 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:13:56.661212 11369 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:13:56.752609  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:56.765491  8061 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:13:56.767138  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:56.785126  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:56.785574  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:56.787045  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:56.753396  8102 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:56.753131 11206 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:56.815464  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:56.790505  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:56.574401 11358 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:56.818096 11358 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:56.791350 11370 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:56.818884 11038 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:56.819160 11370 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:56.820154 11370 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:56.820779 11154 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:56.821727 11199 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:56.786154 10948 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:56.822814 10948 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.823159 10948 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:56.750651 11168 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:56.732980 11348 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:56.667776 11339 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:56.788487  8728 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:56.791194 11030 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:56.813740 11202 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:56.813827 11040 raft_consensus.cc:697] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:56.792861 11371 log.cc:927] Time spent T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 13:13:56.835645  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 82274.8us
I20260709 13:13:56.828056 11288 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:56.838999  8175 leader_election.cc:400] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9.
I20260709 13:13:56.855654  8175 leader_election.cc:403] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:13:56.841904  7948 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b578d88c1c1f4ca58b7291fed4909f09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica d9cd2999aa314b39997b423ec1ae509e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:56.733335 11162 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:56.803471 11327 raft_consensus.cc:2764] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:13:56.858033  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 68789.3us
W20260709 13:13:56.836190 10940 consensus_meta.cc:287] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.000s
I20260709 13:13:56.838102  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:56.718844 11008 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:56.853847  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:56.788621 11365 consensus_queue.cc:1059] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.352s
I20260709 13:13:56.831079 11368 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:56.845749 11174 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:56.918483  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:56.852593 11028 consensus_meta.cc:287] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.000s
I20260709 13:13:56.859583  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:56.847175 11202 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.859572 11364 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:56.861760  8056 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.001s	sys 0.021s
W20260709 13:13:56.925422  8056 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.001s	sys 0.026s
I20260709 13:13:56.925484  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:56.871454  8132 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:56.872241 10915 raft_consensus.cc:3055] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:56.926716 10915 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:56.882879  8054 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:56.929625  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:56.928622 10915 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:56.859843 11326 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9.
I20260709 13:13:56.930438 10915 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:56.789636 11369 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:56.914968  8175 leader_election.cc:400] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:56.926309 11174 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.924319 11202 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:56.931057  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:56.918268 11359 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:56.932338  8175 leader_election.cc:403] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:13:56.870808  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.930562 11040 consensus_queue.cc:237] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:56.937906  8135 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
W20260709 13:13:56.915588  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 62505.2us
I20260709 13:13:56.945415  8693 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:56.945607  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:56.946120  8130 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:56.950263 11369 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:56.976011  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.924582 11028 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:13:56.938820  8026 leader_election.cc:336] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.831s (SENT)
I20260709 13:13:56.986652  8728 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:13:56.983642  8102 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:56.986953  8026 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:56.989293  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:56.990249  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 118917us
I20260709 13:13:56.990693  8130 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:13:56.992606  8203 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:56.992708  8027 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:13:56.994217 11378 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:13:56.926718 11364 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.000052  8723 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:13:57.000283  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 54940.8us
I20260709 13:13:57.001461  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:57.002539  8136 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:57.004097  8029 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:56.943590 11359 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:57.007268  8723 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:13:57.008105  8730 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: []
W20260709 13:13:56.994313 11384 log.cc:927] Time spent T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:13:57.022557  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.006258  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:57.024483  8211 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:57.028003  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:57.029685  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:56.998970 11369 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:57.005266 11269 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:56.941268 11174 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.005311 10972 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:57.007768  8693 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:56.983765 11275 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.038779  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:57.008881 10940 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:57.010968 11349 raft_consensus.cc:3055] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:13:57.037563  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 97463.2us
I20260709 13:13:57.005962 11389 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:57.047108  8723 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:57.033771 11154 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:57.036636 11326 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:13:57.050973  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 46919.7us
I20260709 13:13:57.052148  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:57.052140  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.053516  8728 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:57.053754  8723 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:13:57.053994  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.037050 11269 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:57.055567  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.058305  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.058806  8205 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:57.043972 11028 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:57.043941 11005 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.045169 10972 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:57.063269  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 69770.8us
I20260709 13:13:57.060187  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.043336 11349 raft_consensus.cc:740] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:57.065912  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.066520  8130 raft_consensus.cc:2376] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8. Re-sending same reply.
I20260709 13:13:57.066898  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:56.449736  8207 rpcz_store.cc:276] 0709 13:13:55.349359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.350804 (+  1445us) service_pool.cc:224] Handling call
0709 13:13:55.351075 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:55.351314 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:13:55.714193 (+362879us) log.cc:844] Serialized 14 byte log entry
0709 13:13:55.945951 (+231758us) log.cc:844] Serialized 14 byte log entry
0709 13:13:56.448412 (+502461us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b1800265c60. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ea3c0a 00007f6dc15e3219 00007f6db7f2ba40 00007f6db7f264e4 00007f6db7f26455 00007f6dbb00e060 00007f6dbb020716 00007f6dbb031287 00007f6dbb031208
0709 13:13:56.448531 (+   119us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:56.448588 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:56.448631 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.448704 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:56.448803 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":493804,"prepare.queue_time_us":67544,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":88,"raft.queue_time_us":8595,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":4496,"spinlock_wait_cycles":12063872,"thread_start_us":573314,"threads_started":2,"wal-append.queue_time_us":324363}
I20260709 13:13:57.071372  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.071897 11174 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:57.072518 11028 raft_consensus.cc:697] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:56.996599 11364 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:57.056079 10740 raft_consensus.cc:3055] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:57.089627  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.072039  8207 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:57.073491 11028 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.079212 11162 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:57.127343  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:57.084452  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.084334  8132 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:13:57.060832 11269 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.086853 11168 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:13:57.097208  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.062100 10940 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.127012  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.080874 11275 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:57.129102  8205 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:57.054421 10939 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:13:57.140976  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:57.058709 11359 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:57.167725  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.170991  8136 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: []
W20260709 13:13:57.181104 11390 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:13:57.181949  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.182592  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:57.183142  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.183557  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:57.185151  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.088129 10740 raft_consensus.cc:740] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:13:57.213135 11387 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:13:57.214417 10948 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.199741  8060 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:57.215744 11162 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:13:57.216441 10948 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:57.216965 11162 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:57.066071 11005 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.218746 11162 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:57.218851 10972 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.220203 11005 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:57.220468 11162 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.208331 11206 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.202157  8056 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:57.225232  8209 raft_consensus.cc:2368] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7.
W20260709 13:13:57.209489 11154 consensus_queue.cc:1175] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "965e59f272f143449b18f65db4e61d98" responder_term: 8 status { last_received { term: 7 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:13:57.210382  8205 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:57.226830  8203 raft_consensus.cc:2368] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8.
I20260709 13:13:57.210660 10939 raft_consensus.cc:3055] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:57.210846  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.211236 11349 consensus_queue.cc:260] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.211570  8696 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.212358  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:57.228215  8129 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:57.212929  8134 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:57.207908  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.233469  8062 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:57.209527  8211 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:57.235186  8205 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:13:57.230751  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:57.236308  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.236399  8060 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:57.237433  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:57.237496  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.243103  8059 raft_consensus.cc:2368] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:57.243378  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.243993  8134 raft_consensus.cc:2368] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:57.243101  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:57.049284 11368 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.249976  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.250443  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:57.250574  8136 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:57.251228  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 476) took 1242 ms. Trace:
I20260709 13:13:57.255208  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.258983  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:57.260313 11396 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:13:57.224083 11206 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:57.165648 11364 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:13:57.235946 11005 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:57.231325 11030 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.213833 10740 consensus_queue.cc:260] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:57.266251 11385 log.cc:927] Time spent T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:13:57.232129 10939 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:57.268144  8211 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:57.251551  8208 rpcz_store.cc:276] 0709 13:13:56.008400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.008915 (+   515us) service_pool.cc:224] Handling call
0709 13:13:56.009316 (+   401us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:56.466120 (+456804us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:13:56.466395 (+   275us) log.cc:844] Serialized 14 byte log entry
0709 13:13:57.010323 (+543928us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b18003d04e0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb623a
0709 13:13:57.020170 (+  9847us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:57.028847 (+  8677us) log.cc:844] Serialized 52 byte log entry
0709 13:13:57.028993 (+   146us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:13:57.031860 (+  2867us) log.cc:844] Serialized 14 byte log entry
0709 13:13:57.032098 (+   238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.032203 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:57.251045 (+218842us) raft_consensus.cc:1685] finished
0709 13:13:57.251067 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.251180 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":516806,"prepare.queue_time_us":1787,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":64,"raft.queue_time_us":479096,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":87284,"spinlock_wait_cycles":48938240,"thread_start_us":26933,"threads_started":1,"wal-append.queue_time_us":244960}
I20260709 13:13:57.274508  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.244632 11162 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:57.272451  8693 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:57.265725 11202 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.279502  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:57.284771  8061 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:57.285842  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:57.286701  8061 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:57.282493 11313 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:57.283798  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.263005 11349 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:57.291350  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:57.292191  8055 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:57.290524  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:57.290830  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:57.301462  8210 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
W20260709 13:13:57.289381 11392 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:13:57.316224  8207 raft_consensus.cc:2393] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:57.288957 11313 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.350415  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.351508  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.352703  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.353410  8211 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
W20260709 13:13:57.367738  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.374063  8026 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:57.375396  8026 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:13:57.376377  8026 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:57.361485 11269 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.283123 11030 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:57.381830  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.392462  8026 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:57.392844  8104 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:57.393976  8101 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.327471 11313 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.363185 11202 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:57.407058  8135 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:57.407720  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:57.410663  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.412158  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.412698  8136 raft_consensus.cc:2376] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6. Re-sending same reply.
I20260709 13:13:57.389959 11336 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:57.391319 11176 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:57.391914 11168 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:57.416035  8101 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:57.392761 10972 raft_consensus.cc:2764] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:13:57.420917  8130 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:13:57.411176 11364 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:57.402957 11377 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.410749 11382 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:13:57.429533  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6074641350001002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24380000000000002s for a response
I20260709 13:13:57.424923  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.425624  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:13:57.432509  8175 connection.cc:256] RPC call timeout handler was delayed by -1.2028745259999596s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1572s for a response
I20260709 13:13:57.420674 11168 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.419381 11176 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:57.434791  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:57.373638 10939 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.439183  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.429322 11369 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:57.429855 11382 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.444339  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:57.446303  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.431751 11364 raft_consensus.cc:697] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:57.433368 11202 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.435254 11168 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:57.447635  8136 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:57.446730 11382 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:57.429168 11337 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.474318  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:57.480178  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.457633 10939 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:57.481734  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:57.484175  8101 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.453104 11202 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.491982  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:57.488765  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 64736.2us
I20260709 13:13:57.494660  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.496093  8208 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:13:57.497265  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.481866 11206 raft_consensus.cc:2764] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:13:57.476716 11369 consensus_queue.cc:1059] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:13:57.488647 11351 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:57.507606  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.509109  8208 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:57.495100 11337 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.508217  8101 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:57.513240  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.513777  8136 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 6: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:13:57.507228 11335 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.309s	user 0.015s	sys 0.035s
I20260709 13:13:57.467397 11176 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.518841  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:57.507743 11269 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:57.520994 11339 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:13:57.495921 11389 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
W20260709 13:13:57.526870 11401 log.cc:927] Time spent T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:13:57.508773 11351 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:13:57.540241  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 48366.2us
W20260709 13:13:57.518244 11402 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
W20260709 13:13:57.520394 11335 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.320s	user 0.015s	sys 0.036s
I20260709 13:13:57.517587 11206 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:57.563053  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:57.563901  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.571095  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:57.571857  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.572588  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:57.572821  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.573323  8136 raft_consensus.cc:2376] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7. Re-sending same reply.
I20260709 13:13:57.574034  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.574704  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:57.574946  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.574685  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:57.576349  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:57.576413  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.577045 11020 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:57.577697 11327 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.577361 11375 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:57.577922 11202 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:57.578290 11375 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:57.579571  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:57.578873 11327 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:57.579963 11028 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.580211 11327 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:57.580816 11327 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:57.579969 11375 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.566517 11206 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:57.581789 11206 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.577884 11020 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:57.582508 11020 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:57.580535 11028 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.585335 11028 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:57.590826 11375 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.592883 11375 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:57.564602 11351 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:57.595356  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:57.566599 11335 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:57.600519 11020 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.515172 11364 consensus_queue.cc:237] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.609234  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.610844  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:57.612393  8205 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:57.613950 11188 consensus_queue.cc:1059] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:57.614315  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.614938 11188 raft_consensus.cc:3055] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:57.615270 11188 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:57.615399  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:57.616070 11188 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.617143 11188 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:57.617763 11040 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.618283 11336 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:57.618697 11336 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.619022 11336 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:57.617587  8029 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:57.624156 11168 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.625945 11030 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:57.626112 11168 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:57.626540 10940 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:57.626959 10940 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:57.627300 10940 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:57.627840  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.628437  8132 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:57.632611  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.605268  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.602479 11335 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:57.643049  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 47804.6us
I20260709 13:13:57.643657  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.643963  8175 leader_election.cc:400] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:57.644956  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:57.647874  8132 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:13:57.649369  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.650971  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.652575  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:57.652891  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.653789  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.656652  8136 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
W20260709 13:13:57.658658  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 150480us
I20260709 13:13:57.569227 11406 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.662452  8136 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 5. Current term is 6. Ops: []
W20260709 13:13:57.660115  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 127212us
I20260709 13:13:57.660722  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.661279  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:57.664079  8136 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:57.631842 10948 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:57.666977 11411 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.668193  8101 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:57.668692 11417 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.652232 11168 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:57.674911  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:57.674962  8132 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:57.676807  8029 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.656489 11375 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:57.657945 11162 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.676738  8693 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:13:57.705781  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 60894.1us
I20260709 13:13:57.707998 11370 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:57.708192  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:57.681571  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.708582  8138 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:57.670013 11389 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:57.670701 11028 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:57.710719  8101 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.708731 11370 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:57.669064 11206 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.678149 11283 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:57.630357 11351 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.706928 11202 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:57.709611 11030 raft_consensus.cc:2764] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:13:57.723878  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:57.713830 11040 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.679469 11410 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:57.711889 11389 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:57.717849 11375 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.718456 11028 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:57.722389 11368 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:57.711597 10940 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.679100 11336 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.721586 11202 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:57.717537 11168 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:13:57.715299 11162 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.726485 11410 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.679073 11411 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.729382 11389 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.765503  8101 leader_election.cc:400] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10.
W20260709 13:13:57.770010  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 46178.2us
I20260709 13:13:57.738013 11368 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.772279  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:57.774644  8175 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:57.762633 11202 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.778864  8175 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:57.728942 11382 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.766371 11410 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:57.783620 11426 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:57.785012  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:57.776141 11368 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:57.771400 11420 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.777626 11040 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:57.779558 10948 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:57.794891  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 50990.6us
I20260709 13:13:57.808776  8136 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:13:57.773213 11370 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:57.809959 11349 consensus_meta.cc:287] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.514s	user 0.000s	sys 0.009s
I20260709 13:13:57.808480 11020 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:57.845168  8129 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:57.843078  8026 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:57.848670  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 63708.5us
W20260709 13:13:57.847393  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 137090us
I20260709 13:13:57.808590 11327 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.850384  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.844600 10948 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.810480 11040 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.846762 11420 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:57.890503 11420 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.891021 11420 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:57.853667  8029 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:57.852262  8026 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:57.872300  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.852666 11283 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:13:57.873246 11414 log.cc:927] Time spent T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:13:57.874152  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.897645 11427 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:57.898105  8132 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:57.898101 11427 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:57.898720 11427 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:57.899560 11283 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:13:57.900369  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 48739.4us
W20260709 13:13:57.901669  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 48088us
W20260709 13:13:57.874771  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.902315  8029 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.875125  8131 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:13:57.852211 11335 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:57.884549 11327 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:57.853302 11206 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:13:57.885110 11416 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:13:57.902895  8026 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.904381 11283 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:57.904748 11335 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:57.908978 11406 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.908818 11335 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.910411  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.910683  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:13:57.910939 11418 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:13:57.911027  8134 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.911640  8137 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:57.905829  8029 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.906061  8132 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:57.906415 11008 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.835083 11028 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.914252  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.919927  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:13:57.876569 11419 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:13:57.926225 11411 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:57.926512  8061 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:13:57.938892  8061 pending_rounds.cc:85] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 7
I20260709 13:13:57.938526 11420 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.939137  8061 pending_rounds.cc:107] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:13:57.874365 11020 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:13:57.939383  8061 raft_consensus.cc:2889] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:57.940615 11420 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:57.927003 11162 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:57.926985 11421 log.cc:927] Time spent T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 13:13:57.928051 11423 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:13:57.928931 11382 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:57.929790 11351 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:57.930496 10940 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:57.932121 11040 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:57.932405  8137 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:57.934990 11410 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.949716  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.936046  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.947023  8134 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
W20260709 13:13:57.948295  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 47085.7us
I20260709 13:13:57.952986  8026 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.955405  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:57.962364  8026 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:57.947618 11313 consensus_meta.cc:287] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.000s
I20260709 13:13:57.924495 11283 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.924075 11008 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.954761 11188 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:57.966567 10940 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:57.971545 10940 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:57.972481 10940 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:57.975445  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:57.978674  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:57.957474 11336 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:57.982189  8175 leader_election.cc:400] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:57.966259 11376 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.982129  8026 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.982620  8175 leader_election.cc:403] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:13:57.983490  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69339us
W20260709 13:13:57.983623  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 264081us
I20260709 13:13:57.984601  8029 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:57.980649 11420 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:57.990505 11040 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.991021  8135 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:57.993476  8029 leader_election.cc:400] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7.
I20260709 13:13:57.994396  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:58.003835  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:58.005137 11411 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:58.006851 11411 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:13:58.008018 11326 raft_consensus.cc:3055] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:58.008468 11326 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:58.009526 11326 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.011436 10948 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:58.012468  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:57.984259 11336 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:58.011528  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:57.991814 11370 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.993315 11188 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:57.975386 11313 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:57.992735 11410 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:58.029893 11404 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.538s	user 0.001s	sys 0.000s
I20260709 13:13:58.030799 11370 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.031868 11313 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:58.033479 11370 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:58.012589 11283 consensus_queue.cc:1059] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:13:58.034998 11283 raft_consensus.cc:3055] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:58.035323 11283 raft_consensus.cc:740] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:58.036301  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1554) took 1301 ms. Trace:
I20260709 13:13:58.036051 11283 consensus_queue.cc:260] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.036648  8131 rpcz_store.cc:276] 0709 13:13:56.734534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.734702 (+   168us) service_pool.cc:224] Handling call
0709 13:13:58.036157 (+1301455us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001bb054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:58.036251 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2573711104}
I20260709 13:13:58.037412 11283 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:58.038934  8137 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:13:58.043589  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:58.012512 11020 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:58.003454 11420 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:13:57.980021  8210 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:48498
W20260709 13:13:58.022094 11394 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.822s	user 0.003s	sys 0.004s
I20260709 13:13:58.018249 11359 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.017485 11389 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.060252  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:58.061571  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:58.062088  8061 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.062451  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.063141  8060 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:58.063962  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.016173  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:58.064924  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:13:58.064939  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 48836.1us
I20260709 13:13:58.065680  8060 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.066067  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:58.061559  8728 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:58.067723  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:58.068408  8054 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:58.045058 11410 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:58.074488  8055 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:13:58.016142 11202 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.075040  8061 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:58.073621  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.075893  8057 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:58.073149  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:58.076962  8062 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:58.045214 11188 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:58.078958  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.084270  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.085074  8061 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:13:58.091279  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:58.091605  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:58.091825  8055 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.092160  8058 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:58.044261 10948 consensus_queue.cc:1059] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:13:58.072824 11206 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:58.097626  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.098774  8057 raft_consensus.cc:2393] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:58.071404 11429 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:58.083757 10682 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.057735 11368 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.116604 11368 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:58.117580 11433 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:58.118137 11433 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:58.118752  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1462) took 1121 ms. Trace:
I20260709 13:13:58.119393 11389 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:58.119032 11433 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.120675 11375 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:58.114639 10682 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.123593  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:58.123883  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2627) took 1203 ms. Trace:
I20260709 13:13:58.124118  8204 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.124665  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:58.126021 11427 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.128003 11427 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:58.128307  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:58.128770  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.129459 11358 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.129909 11358 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.131294  8204 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:58.131880 11358 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:58.132997 11335 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:58.133451 11335 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.103672 11206 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:58.134958  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:58.135699 11430 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:58.136067  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:58.137101  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:58.137848 11430 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:58.138202 11430 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:58.091161 11410 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.138659 11430 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:58.140189 11410 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:58.087714 11188 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.143476 11206 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.144017 11188 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:58.124302  8212 rpcz_store.cc:276] 0709 13:13:56.920379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.920529 (+   150us) service_pool.cc:224] Handling call
0709 13:13:56.920817 (+   288us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:13:56.934216 (+ 13399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:56.934241 (+    25us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:13:56.936159 (+  1918us) log.cc:844] Serialized 78 byte log entry
0709 13:13:57.029423 (+ 93264us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:57.041826 (+ 12403us) log.cc:844] Serialized 14 byte log entry
0709 13:13:57.042008 (+   182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.042072 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:58.021585 (+979513us) spinlock_profiling.cc:243] Waited 7.39 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:58.123607 (+102022us) raft_consensus.cc:1685] finished
0709 13:13:58.123702 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:58.123828 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":490636,"prepare.queue_time_us":12260,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":10065,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":504120,"spinlock_wait_cycles":17000320,"thread_start_us":92902,"threads_started":1,"wal-append.queue_time_us":130008}
I20260709 13:13:58.150514 11375 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:58.151369 11202 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:58.123906 10682 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:58.154664  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:58.155720 11429 raft_consensus.cc:697] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:58.156668 11429 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.124167  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:58.158824  8208 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.160535  8054 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:58.241967  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.138141  8175 leader_election.cc:400] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:58.119138  8203 rpcz_store.cc:276] 0709 13:13:56.996906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.999945 (+  3039us) service_pool.cc:224] Handling call
0709 13:13:58.118582 (+1118637us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54002bf554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:58.118701 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434567040}
I20260709 13:13:58.164340  8058 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:13:58.243814  8175 leader_election.cc:403] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:13:58.167192 11391 log.cc:927] Time spent T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.987s	user 0.005s	sys 0.000s
I20260709 13:13:58.243660  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:58.244552  8203 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:58.245173  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1558) took 1440 ms. Trace:
W20260709 13:13:58.246197  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 108162us
I20260709 13:13:58.245512  8133 rpcz_store.cc:276] 0709 13:13:56.804495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.804655 (+   160us) service_pool.cc:224] Handling call
0709 13:13:56.814581 (+  9926us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:13:56.847298 (+ 32717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:56.847346 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:13:56.855184 (+  7838us) log.cc:844] Serialized 52 byte log entry
0709 13:13:57.022933 (+167749us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:57.270567 (+247634us) log.cc:844] Serialized 14 byte log entry
0709 13:13:57.270800 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:57.270886 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:58.166592 (+895706us) spinlock_profiling.cc:243] Waited 7.55 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:13:58.244904 (+ 78312us) raft_consensus.cc:1685] finished
0709 13:13:58.244986 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:58.245104 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48010,"prepare.queue_time_us":102029,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":14782,"raft.queue_time_us":25069,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":12062,"spinlock_wait_cycles":17372416,"thread_start_us":381344,"threads_started":2,"wal-append.queue_time_us":225778}
I20260709 13:13:58.246928  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:58.247025  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:58.247539  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:58.247776 11339 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.248085  8133 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:13:58.248129  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:58.248975  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:58.249107  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:58.249496  8133 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:13:58.165683 11335 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:13:58.168440 11403 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.709s	user 0.000s	sys 0.001s
I20260709 13:13:58.250205  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:58.250696  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:58.171177  8063 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:58.251104  8133 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:58.251451  8205 raft_consensus.cc:2410] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:13:58.251842  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:58.251941  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.252200  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:58.180100  8055 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:58.252987 11429 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:58.253224  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.253120  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.181563  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:58.253799  8055 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:13:58.181361  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:58.254762  8135 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:58.182669  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:58.183619  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:58.182571  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:58.183768  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:58.189438  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:58.190714  8210 raft_consensus.cc:606] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:58.255993  8132 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.256728  8723 raft_consensus.cc:2393] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:58.180712  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:58.192909  8727 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:58.148507  8062 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:58.197850 11411 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.186111  8721 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:13:58.258482  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.258319 11411 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.255648  8134 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:13:58.259164  8062 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:13:58.260079  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.260869 11411 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:58.197001 11430 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.261135  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:58.255622  8722 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:13:58.261953  8062 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:58.209403  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:58.262840 11430 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:58.202503 11432 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:13:58.211745  8207 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.265208  8206 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:58.211982 10740 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:58.265764  8063 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 8. Ops: []
I20260709 13:13:58.209959  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:58.219141  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:58.217744  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:58.267810  8055 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.220952  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:58.223328  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:58.223938  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:58.268663  8056 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:58.268807  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 45549.7us
I20260709 13:13:58.226859 11334 raft_consensus.cc:3055] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:58.269548 11334 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:58.221922  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:58.232311 11327 raft_consensus.cc:3055] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:58.234124  8728 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:58.271189 11327 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:58.271334 11335 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:58.236012  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:58.272015 11335 raft_consensus.cc:697] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:58.272090 11327 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.273275 11327 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:58.274439  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:58.243649  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:58.252475  8133 raft_consensus.cc:2393] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:58.171782 11326 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:58.253403 11429 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:58.277663  8058 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:58.277930 11429 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.231920 11008 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:58.267293  8059 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.278857  8058 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:58.279913  8058 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:58.281018  8058 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:58.282009  8058 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.224654 11176 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:13:58.282559  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.283759 11430 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:13:58.284124 11430 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.284432 11430 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:58.284827 11429 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.235035  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:58.286834 11429 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:58.236845  8060 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:58.287755  8057 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
W20260709 13:13:58.288342  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.274891  8063 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:58.277319  8203 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:13:58.270298 11334 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.214957  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:58.291172 11334 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:58.155735 11420 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:58.268832  8055 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.276417  8056 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:58.298161 10940 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:58.299314  8056 pending_rounds.cc:85] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 4
I20260709 13:13:58.299471  8055 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.299621  8056 pending_rounds.cc:107] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:13:58.299984  8056 raft_consensus.cc:2889] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:58.300599  8055 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:58.276657  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:58.255668  8136 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:13:58.301445  8724 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:58.272918 11335 consensus_queue.cc:237] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.302106  8055 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.303444  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:58.304328  8203 raft_consensus.cc:2410] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:13:58.306404  8212 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:13:58.306351  8029 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:58.306830  8725 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:13:58.307118 11359 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:58.307605 11359 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:58.307946  8212 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.308420 11359 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.309414  8211 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:58.310007 10940 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:58.310499  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:58.310734 10940 consensus_queue.cc:1243] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:13:58.314851  8212 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:58.315850  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:58.316895  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:58.317507  8212 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:58.320689 11339 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:13:58.256593  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:58.321789  8205 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:13:58.264868  8132 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:58.322824  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:58.324057  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:58.167776 11020 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:58.325287 11359 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:58.325799 11359 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.326272 11359 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:58.190244  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:58.328598  8061 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:58.330878  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:58.331597  8205 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:58.332593  8205 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:58.333796  8205 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:13:58.339262 11359 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.340864  8058 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:58.379040  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:58.380122  8058 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:58.381134  8058 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.275264 11433 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:13:58.355887  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:58.382164  8060 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:13:58.382380 11427 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:58.382877  8026 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:58.383594 11427 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:58.384023  8026 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:58.384698 11427 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:58.386046  8058 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:13:58.386436  8058 pending_rounds.cc:85] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 4
I20260709 13:13:58.359669 11359 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:58.360744  8063 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:13:58.360803  8059 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:58.396099  8062 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:58.364765  8725 pending_rounds.cc:85] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Aborting all ops after (but not including) 5
I20260709 13:13:58.365502  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:58.397245  8725 pending_rounds.cc:107] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:13:58.370102 11432 consensus_queue.cc:1059] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:58.372476  8208 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:58.373347 11303 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:13:58.397593  8725 raft_consensus.cc:2889] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:58.362046  8211 raft_consensus.cc:2410] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:13:58.318141  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:58.408569  8212 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:13:58.409452  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:58.410171  8212 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:13:58.411147  8212 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:13:58.412354  8212 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:58.375795  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.355101  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:13:58.381166  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.385948  8026 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:58.428355  8026 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:58.429104 11427 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:58.429682 11427 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:58.428380 11364 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:13:58.430189 11417 consensus_queue.cc:1059] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:58.358832  8061 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:58.358013  8203 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.431262 11406 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:58.431663 11406 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:58.432204  8203 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.432417 11406 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.348858  8209 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:58.438026  8209 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:13:58.439143  8209 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:13:58.389703  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:58.440969  8205 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:58.396410 11359 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.442632 11359 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.444826 11359 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:58.399649  8204 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:58.447392  8204 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.400789  8029 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:58.401307  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:58.448452  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 47741.7us
I20260709 13:13:58.402350 11349 consensus_queue.cc:1059] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:58.401767 11008 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:58.449384 11154 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:58.449424  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:58.450178 11154 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:58.451121 10939 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:58.451192 11154 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.451740 10939 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:58.452780 10682 raft_consensus.cc:3055] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:13:58.452589 10939 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.453163 10682 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:58.454026 10682 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.250005  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:58.404384  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:58.456130  8206 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:13:58.404299  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:13:58.460412  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 80485.3us
I20260709 13:13:58.461321  8693 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:58.407471  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:58.407971  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:58.416072  7959 catalog_manager.cc:5697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 5 to 6, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 6 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.413451  8058 pending_rounds.cc:107] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:13:58.463851  8058 raft_consensus.cc:2889] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:58.252254  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:13:58.420794 11348 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:58.421939  8104 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:58.465958  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44086.2us
I20260709 13:13:58.466845  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:58.422459 11353 consensus_queue.cc:1059] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:58.423900 11334 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:58.424760 11174 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:13:58.425639 11208 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:58.427124  8212 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:58.430521 11364 consensus_queue.cc:1243] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:13:58.463620  8207 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:58.454977 10682 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:58.393574 11430 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.497236  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:13:58.449549 11349 raft_consensus.cc:3055] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:58.515050 11432 raft_consensus.cc:3055] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:58.515491 11432 raft_consensus.cc:740] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:58.521420  8205 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:13:58.522104  8210 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:13:58.491097 11369 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:58.498790 11425 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:58.528115  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:13:58.530134  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 277926us
W20260709 13:13:58.531464  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 62944.2us
I20260709 13:13:58.532326  8175 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:13:58.535856  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 74638.3us
I20260709 13:13:58.537596 11370 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:58.539129  8693 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:58.537705 11413 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:58.503540 11349 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:58.479620 11028 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:13:58.538199 11370 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.501463 11424 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:13:58.505244 11382 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:13:58.516338 11432 consensus_queue.cc:260] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.565472  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:13:58.574983  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 83606.2us
I20260709 13:13:58.575568  8724 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:58.565559  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:58.576160  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:58.550264 11176 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:13:58.529742 11358 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:13:58.583390  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:58.558514 11005 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.559103 11359 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:58.559430 11413 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:58.566519 11370 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:58.538837 11436 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:13:58.575155 11220 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:13:58.580933 11429 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:58.555971 11020 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:13:58.607945 11436 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.608515 11436 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.609474 11461 raft_consensus.cc:3055] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:58.609856 11461 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:58.610678 11461 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.611356 11406 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:58.611785 11461 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.616652 11348 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:58.555537 11269 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 13:13:58.619017  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.605656 11220 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:13:58.620478 11220 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:13:58.590682 11364 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:13:58.602550  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:58.643575  8104 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:58.646126  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:58.648579  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 92476.5us
I20260709 13:13:58.588531 11008 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
W20260709 13:13:58.652444  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 113761us
I20260709 13:13:58.654475  8058 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:13:58.625224 11359 raft_consensus.cc:697] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:13:58.643589 11420 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:58.496785 11206 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:58.665549  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:58.597482 11464 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:13:58.674227 11368 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:13:58.674638 11368 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:58.681272 11430 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:58.682636 11413 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.682191  7959 catalog_manager.cc:5697] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 3 to 8, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 8 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.682768 11359 consensus_queue.cc:237] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.684489  8026 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:58.684314  7959 catalog_manager.cc:5697] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 3 to 6, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 6 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.685774 11467 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:58.686647 11334 raft_consensus.cc:3055] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:58.686472  7959 catalog_manager.cc:5697] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:58.687009 11334 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:58.687702 11188 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:13:58.688750 11460 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:13:58.688829 11382 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:58.687702  7959 catalog_manager.cc:5697] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 5 to 8, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 8 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.689266 11382 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:58.659649 11375 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:58.605134 11349 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.690004 11375 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:58.690114 11382 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.649355 11465 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:13:58.690452  7959 catalog_manager.cc:5697] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.664264 11436 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:58.691496 11382 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:58.690546 11349 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:58.691982  7959 catalog_manager.cc:5697] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:13:58.658809  8104 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:58.694262  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:58.653519 11208 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:13:58.661701  7948 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b578d88c1c1f4ca58b7291fed4909f09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica d9cd2999aa314b39997b423ec1ae509e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:13:58.605773 11005 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.690510 11375 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:58.694761  8104 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:58.661397 11469 consensus_queue.cc:1059] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:13:58.694872  8724 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:58.668324 11420 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:58.714442  8104 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:58.695326  8693 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:58.658058 11364 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:13:58.702797 11464 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:13:58.696144  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 168105us
W20260709 13:13:58.699977  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.686156 11467 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:58.713202 11420 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.693468  7959 catalog_manager.cc:5697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 6 to 8, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 8 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:13:58.695214 11353 consensus_queue.cc:1175] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d9cd2999aa314b39997b423ec1ae509e" responder_term: 7 status { last_received { term: 6 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:13:58.713166 11465 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:58.721174  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.766564  8055 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:13:58.768369 11425 raft_consensus.cc:3055] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:58.768778 11425 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:58.778119  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:13:58.734380  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 198970us
I20260709 13:13:58.718716 11436 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:13:58.712386  7948 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 52943.4us
W20260709 13:13:58.764279  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 98790.9us
I20260709 13:13:58.784408  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:58.785454  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:58.786170 11020 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:58.766469 11417 consensus_queue.cc:1059] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:13:58.786690 11020 raft_consensus.cc:697] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:58.787580 11020 consensus_queue.cc:237] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.788520 11425 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.789219 11176 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:58.789690 11176 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:58.789806 11425 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.790627 11176 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.736905 11424 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:13:58.794215  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:58.795017 11430 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:58.795496 11430 raft_consensus.cc:697] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:58.796314 11430 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.797384 11370 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.719272 11303 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 13:13:58.723644 11462 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.100s	user 0.000s	sys 0.003s
I20260709 13:13:58.722460 11339 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:58.803045  8724 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:13:58.804719  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:58.803133  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:58.779922  8696 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:13:58.782951 11463 log.cc:927] Time spent T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.189s	user 0.000s	sys 0.002s
W20260709 13:13:58.821733  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 48582.1us
I20260709 13:13:58.812992  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:13:58.823172  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:58.793705 11368 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.808930  8027 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:13:58.833967  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:58.832755  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.843135  8212 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:13:58.845431  8104 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:58.848708  8054 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:13:58.850409 11326 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.852679  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:58.853268  8138 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:13:58.862811  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:58.863423  8137 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:13:58.863451  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:58.863737  8132 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:58.866271  8134 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:13:58.868642  8134 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:13:58.832015 11327 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:58.841989 11303 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:58.841746 11429 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:58.822172 11339 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:13:58.874171  8131 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:13:58.826845 11206 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.841626 11433 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:13:58.871954  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:58.861621  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:13:58.844236 11436 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:13:58.869793  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:58.861231 11275 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.885414  8134 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:58.890008  8137 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:58.891746  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:58.861027 11376 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:58.858238 11368 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:13:58.898203  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:58.907325  8129 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:58.848110 11208 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:13:58.871886 11303 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.930785  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:58.937435  8132 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:13:58.935002  8061 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:13:58.871719 11269 consensus_queue.cc:1059] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:13:58.862123 11417 consensus_queue.cc:1059] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:13:58.860947 11326 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.938169  8137 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:13:58.939518  8131 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:58.945155  8129 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:58.909246 11376 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:58.943799 11446 log.cc:927] Time spent T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:13:58.957964  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 83216.6us
I20260709 13:13:58.941077 11303 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:58.943348 11005 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:58.905167 11206 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.895802 11275 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:58.959380 11376 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:58.974844  8730 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
I20260709 13:13:58.978017 11486 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:58.978550 11486 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:58.978936 11486 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:58.939507 10740 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:13:58.874857 11467 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:58.987579  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:58.994424  8058 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:13:58.989512  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:58.990602  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:58.995940  8062 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:59.003957 11370 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:59.004508  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:59.005098  8129 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:58.967795 11410 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:13:59.007987 11467 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:59.011693  8129 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:13:59.012257  8211 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:13:59.013971 11469 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:13:59.018837 11206 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:59.020601  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:58.966027 10940 consensus_queue.cc:1059] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:13:59.029258  8062 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
W20260709 13:13:59.035429  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.021083 11420 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.014163 11375 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.032075 11433 raft_consensus.cc:697] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:59.112442 11432 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:59.059235 11220 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:59.060174 11275 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:59.061848 11326 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:59.063593  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:13:59.122300  8211 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:13:59.122424 11303 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:59.132205 11334 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.133378 11334 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:59.134176 11303 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:59.135119 11433 consensus_queue.cc:237] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.031481 11476 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:13:59.069164  8133 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:13:59.078794 11375 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:59.122931  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:59.081261  8210 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:59.085507  8209 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:59.086414  8208 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:13:59.075548 11477 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:13:59.091243  8138 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:13:59.094623 11420 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:59.098110  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:13:59.158759  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 60742.1us
I20260709 13:13:59.159154  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:59.102510 11364 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:13:59.103292 11376 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:13:59.104417  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 170020us
W20260709 13:13:59.104842  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.104902 11349 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:59.105598 11464 raft_consensus.cc:3055] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:13:59.102041  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 256808us
I20260709 13:13:59.187989  8104 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:13:59.109336  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:59.189011  8062 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:59.117954  8056 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:13:59.120057  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:13:59.198884  8059 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:13:59.122706  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:59.087498  8206 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:13:59.159385  8693 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:13:59.099958 11472 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.123162  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:59.159576  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:59.212294  8211 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:59.160107  8203 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:59.213589  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:13:59.131225  8131 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:13:59.188453  8063 raft_consensus.cc:3055] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:13:59.218349  8061 raft_consensus.cc:2393] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:59.216197  8063 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:59.224017  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 64699.4us
I20260709 13:13:59.225401  8136 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:13:59.225703  8130 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:13:59.222054  8063 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.226673  8063 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:13:59.227209  8130 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:59.230602  8130 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:13:59.129871 11220 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:59.183012 11339 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:59.217949 11303 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:59.112677 11486 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.212298 11464 raft_consensus.cc:740] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:59.255712 11349 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:13:59.256175 11364 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.256696 11364 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:59.249619 11353 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:13:59.266863 10939 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:59.267587 10939 consensus_queue.cc:1243] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:13:59.187651 11471 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:13:59.211987 11427 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:13:59.185593 11327 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:59.232385 11488 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:13:59.234951 11368 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:13:59.284294  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:59.251178 11303 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.294226  8203 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:13:59.295192  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:59.270134 11168 raft_consensus.cc:3055] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:59.296526  8203 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:59.299014  8203 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:13:59.328471  8203 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:13:59.272104 11489 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.297799 11303 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:13:59.286870 11368 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:59.303478 11492 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:13:59.300779 11425 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:13:59.339617  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:13:59.223814 11472 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:59.301141 11488 raft_consensus.cc:697] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:59.300638 11168 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:13:59.359366  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 57270.9us
I20260709 13:13:59.391014  8063 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:13:59.391495  8063 pending_rounds.cc:85] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 3
I20260709 13:13:59.391770  8063 pending_rounds.cc:107] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:13:59.392046  8063 raft_consensus.cc:2889] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:59.319978 11429 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:59.401172  8102 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:59.341485 11364 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:59.320310 11406 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:13:59.362357 11368 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:13:59.410260  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 512093us
I20260709 13:13:59.416950  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:13:59.418203  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 196103us
I20260709 13:13:59.422421  8203 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:13:59.319609 11464 consensus_queue.cc:260] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.396044 11461 consensus_queue.cc:1059] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:13:59.408174 11208 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:13:59.324580 11339 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:59.434060  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:59.436625  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:59.437542 11313 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:59.437908 11313 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.438275 11313 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:13:59.438823 11283 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
W20260709 13:13:59.441712 11481 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.498s	user 0.000s	sys 0.006s
I20260709 13:13:59.443864 11306 raft_consensus.cc:3055] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:59.444249 11306 raft_consensus.cc:740] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:13:59.445046 11306 consensus_queue.cc:260] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:59.445748 11313 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:59.448510 11313 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:59.451710 11359 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.452315 11359 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.454205  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:13:59.454890  8729 raft_consensus.cc:2393] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
W20260709 13:13:59.463320 11497 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 13:13:59.465849  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.466990  8730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 704) took 1098 ms. Trace:
I20260709 13:13:59.467399  8730 rpcz_store.cc:276] 0709 13:13:58.368221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.368313 (+    92us) service_pool.cc:224] Handling call
0709 13:13:58.974689 (+606376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:58.977191 (+  2502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.466824 (+489633us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b54005802d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:13:59.466938 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1393979776}
W20260709 13:13:59.472846  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 250907us
I20260709 13:13:59.410471 11348 consensus_queue.cc:1059] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:13:59.407918 11468 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:13:59.486008  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:59.487067 11470 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:13:59.489217  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:59.489946  8058 raft_consensus.cc:2393] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:13:59.491258 11470 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:59.493485  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:13:59.494022  8057 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:59.446259 11306 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:59.360319 11489 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.499673  8062 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:13:59.468884 11376 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:13:59.502375  8693 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:13:59.475891 11427 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:13:59.476352 11495 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:13:59.474325 11420 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:13:59.424243 11490 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 13:13:59.420137 11327 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:59.632195  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 129908us
I20260709 13:13:59.424561 11168 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:59.651645  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 147177us
W20260709 13:13:59.737875  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:59.741937  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.476099 11283 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:13:59.643357  8211 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:13:59.470597 11484 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 13:13:59.480633 11313 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
W20260709 13:13:59.739881 11477 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:13:59.652271 11162 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:13:59.651717 11470 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.131s
I20260709 13:13:59.792205 11513 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.792697 11513 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.522223 11429 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.757733 11020 raft_consensus.cc:2764] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:13:59.732452 11495 raft_consensus.cc:697] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:13:59.769493  8211 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:13:59.468015 11206 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:13:59.786712 11168 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:59.817961  8729 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11. Ops: []
I20260709 13:13:59.812613 11505 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:59.813220 11506 log.cc:927] Time spent T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:13:59.814584 11509 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:13:59.814097 11498 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
W20260709 13:13:59.761883 11154 consensus_queue.cc:1175] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8a968cada1e4446ebb2a5e167176e649" responder_term: 7 status { last_received { term: 5 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:13:59.762137 11376 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:13:59.800369 11513 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:59.522011 11488 consensus_queue.cc:237] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:59.804960 11413 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.800683 11364 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:13:59.525363 11359 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:13:59.542713 11375 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.830714 11472 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:13:59.872121  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 381984us
I20260709 13:13:59.870703 11413 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:59.852926 10939 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.945921 11327 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:59.945873 11369 raft_consensus.cc:3055] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:13:59.946890 11424 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.946782 11327 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.893342  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:59.947024 11369 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:13:59.947489 11424 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.964200  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:13:59.964843  8212 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:13:59.828528 11505 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.948352 11206 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.943145 11513 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.946260 11489 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:13:59.966190  8104 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:13:59.885443 11375 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:13:59.969626  8212 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:13:59.943054 10939 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:13:59.966403 11502 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:13:59.994590  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:13:59.986827 11493 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:13:59.989614  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:00.005196  8130 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:59.974267  8057 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:13:59.991907  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:00.008206  8137 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:13:59.981014  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 226380us
I20260709 13:13:59.987946 11348 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.996239  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:00.010131  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 66870.2us
I20260709 13:13:59.989418  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:00.011165  8138 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:00.002561  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:00.003816  8131 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:13:59.992762 11208 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:13:59.995980 11499 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.995079 11420 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:59.996337 11489 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:13:59.970783 11369 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:13:59.990854 11513 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:00.016872 11208 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.036584  8138 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:00.036242  8130 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:14:00.041224  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 462) took 1019 ms. Trace:
I20260709 13:14:00.041605 11505 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:00.041997 11501 log.cc:927] Time spent T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.472s	user 0.000s	sys 0.001s
I20260709 13:14:00.043463  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 455) took 1152 ms. Trace:
I20260709 13:14:00.043814  8134 rpcz_store.cc:276] 0709 13:13:58.890836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.891052 (+   216us) service_pool.cc:224] Handling call
0709 13:13:58.891701 (+   649us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:59.102943 (+211242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.102970 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:59.103404 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 13:13:59.271075 (+167671us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:59.271115 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.272367 (+  1252us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:00.043255 (+770888us) raft_consensus.cc:1685] finished
0709 13:14:00.043274 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:00.043387 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":210891,"raft.queue_time_us":211920,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":67,"thread_start_us":166020,"threads_started":1,"wal-append.queue_time_us":308001}
W20260709 13:14:00.041729 11508 log.cc:927] Time spent T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
W20260709 13:14:00.041935 11504 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 13:14:00.041617  8129 rpcz_store.cc:276] 0709 13:13:59.021720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.021859 (+   139us) service_pool.cc:224] Handling call
0709 13:13:59.022170 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:59.106290 (+ 84120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.106332 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:59.106861 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:13:59.346233 (+239372us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:59.346273 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.346354 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:00.041036 (+694682us) raft_consensus.cc:1685] finished
0709 13:14:00.041057 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:00.041175 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83735,"raft.queue_time_us":84900,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":7817,"thread_start_us":238953,"threads_started":1,"wal-append.queue_time_us":367940}
I20260709 13:13:59.997249 11206 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:00.050160 11327 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:00.029974 11499 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:00.060617 11499 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:00.051319 11375 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:00.051928 11515 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:13:59.960686 11495 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:00.062179 11513 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:00.057126 11528 log.cc:927] Time spent T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:14:00.057982 11206 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:00.016481 11348 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:00.039253 11420 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:00.071483  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.074313  8203 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:00.080428  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:00.081974  8203 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:14:00.047581 11208 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:00.073097  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:00.032706 11489 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:00.096390  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:00.097026  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:00.097869 11424 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:00.099699 11489 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:00.099946  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:00.100620  8203 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:00.101636  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:00.102387 11424 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:14:00.098086 11530 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:14:00.102742 11424 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.103164 11424 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:00.103729  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 505) took 1278 ms. Trace:
W20260709 13:14:00.104267 11368 consensus_meta.cc:287] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.685s	user 0.008s	sys 0.000s
I20260709 13:14:00.104656 11368 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:00.105638  8133 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:00.015599 11493 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:00.107635 11306 consensus_meta.cc:287] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.601s	user 0.008s	sys 0.000s
I20260709 13:14:00.104087  8207 rpcz_store.cc:276] 0709 13:13:58.824980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.826907 (+  1927us) service_pool.cc:224] Handling call
0709 13:13:58.854664 (+ 27757us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:59.822203 (+967539us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b180023ffe0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbb00e060 00007f6dbb00ebfb 00007f6dbb01964b
0709 13:13:59.822372 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.822423 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:59.822922 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:13:59.980363 (+157441us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:59.980413 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.980503 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:00.103486 (+122983us) raft_consensus.cc:1685] finished
0709 13:14:00.103504 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:00.103679 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91636,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":4309,"spinlock_wait_cycles":5724032,"thread_start_us":257801,"threads_started":2,"wal-append.queue_time_us":182695}
I20260709 13:14:00.109241 11413 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:00.109645 11303 consensus_meta.cc:287] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.728s	user 0.007s	sys 0.000s
I20260709 13:14:00.042217 11505 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:14:00.118146 11505 consensus_queue.cc:1243] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:14:00.118487 11303 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:00.120540 11303 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:00.070851 11420 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:00.129670  8059 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:00.129573  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:00.130280  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:00.132426  8055 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:00.133579  8724 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:14:00.133931  8724 pending_rounds.cc:85] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: Aborting all ops after (but not including) 6
W20260709 13:14:00.133687  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 54524.9us
I20260709 13:14:00.134202  8724 pending_rounds.cc:107] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:14:00.134509  8724 raft_consensus.cc:2889] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:00.134603  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:00.136173  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:00.141121  8055 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:00.143514  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:00.144250  8057 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:00.141752 11489 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:14:00.150933 11532 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.157095 11532 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:00.159348 11532 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:00.159719 11494 log.cc:927] Time spent T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.835s	user 0.001s	sys 0.001s
I20260709 13:14:00.161080  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 506) took 1335 ms. Trace:
I20260709 13:14:00.164680  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:00.161432  8204 rpcz_store.cc:276] 0709 13:13:58.825291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.842741 (+ 17450us) service_pool.cc:224] Handling call
0709 13:13:58.859353 (+ 16612us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:58.954069 (+ 94716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:58.954099 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:58.964295 (+ 10196us) log.cc:844] Serialized 26 byte log entry
0709 13:13:59.141397 (+177102us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:13:59.141433 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.141494 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:00.159056 (+1017562us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:00.160809 (+  1753us) raft_consensus.cc:1685] finished
0709 13:14:00.160830 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:00.161026 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3439,"raft.queue_time_us":90896,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":5726,"spinlock_wait_cycles":10184960,"thread_start_us":260009,"threads_started":2,"wal-append.queue_time_us":296541}
I20260709 13:14:00.148856 11348 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:00.157006 10939 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:00.156038 11489 raft_consensus.cc:3055] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:00.171229  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:00.171465  8207 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:00.177595  8057 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
W20260709 13:14:00.212080  8056 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.015s	sys 0.037s
W20260709 13:14:00.212571  8056 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.015s	sys 0.037s
I20260709 13:14:00.178527 11489 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:00.271377 10939 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.317857 11532 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.318401 11532 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:00.317106  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 287035us
I20260709 13:14:00.320469 11532 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:00.320770  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:00.321419  8057 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
W20260709 13:14:00.318149  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 304837us
I20260709 13:14:00.324859  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:00.327159  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:00.327291  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:00.329838  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:00.330394  8057 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:14:00.330534  8029 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:00.332624  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:00.333249  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:00.334309  8208 raft_consensus.cc:2410] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:14:00.337589  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:00.308310 11368 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:00.312124 10939 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:00.356539 11532 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:14:00.371125  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:00.394753  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 660023us
I20260709 13:14:00.355448 11020 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:00.346377 11206 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:00.316790 11424 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:00.317891 11208 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:00.412801  8057 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:00.415362 11524 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:00.416016 11524 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:00.417465 11531 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:14:00.419520 11531 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.420099 11531 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:00.355603 11375 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:14:00.402240 11536 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:00.424049 11500 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.947s	user 0.003s	sys 0.000s
I20260709 13:14:00.316500 11489 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:00.382716 11430 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:14:00.427655  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:00.404978 11339 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:00.426232  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:00.368584 11499 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:14:00.437247  8722 raft_consensus.cc:2410] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:14:00.438576  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:00.441291 11534 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:14:00.444615  8729 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:14:00.429903 11306 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.443220 11532 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:00.453356  8725 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:14:00.439603 11220 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:00.456983  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:00.436179 11313 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:00.460574  8729 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:00.458088 11168 consensus_meta.cc:287] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.633s	user 0.011s	sys 0.000s
I20260709 13:14:00.452241 11339 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:00.458300 11532 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.474510  8721 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
W20260709 13:14:00.483222  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 152791us
I20260709 13:14:00.477578  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:00.435120 11536 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:00.503862  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2532) took 1005 ms. Trace:
I20260709 13:14:00.457530 11517 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:14:00.501135  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:00.513717  8729 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:14:00.504343  8056 rpcz_store.cc:276] 0709 13:13:59.498158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.498302 (+   144us) service_pool.cc:224] Handling call
0709 13:13:59.498647 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.498859 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:00.216626 (+717767us) log.cc:844] Serialized 14 byte log entry
0709 13:14:00.503458 (+286832us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:00.503528 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:00.503583 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:00.503674 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:00.503800 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25950,"prepare.queue_time_us":319267,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":974174,"threads_started":3,"wal-append.queue_time_us":217664}
I20260709 13:14:00.515396  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:00.523805  8102 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:00.462729 11220 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:00.526978  8206 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.028s	sys 0.065s
I20260709 13:14:00.497028 11532 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:00.552475  8206 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.028s	sys 0.065s
I20260709 13:14:00.459944 11306 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:00.496450 11537 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:00.465147 11313 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:00.473335 11176 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:14:00.517508 11471 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:14:00.581352  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.577440 11005 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:00.578156 11499 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:00.588579 11539 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:14:00.615870  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:00.570930 11425 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:14:00.618880  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:00.621079 11313 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:00.627832 11425 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.628360 11425 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:00.615187 11368 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:00.639181  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:00.584359 11537 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:00.614203 10939 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:00.610800 11471 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:00.617717 11005 raft_consensus.cc:697] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:00.609848 11208 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:00.642216  8132 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:00.643676 11553 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:00.644691 11552 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:14:00.645074 11551 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:00.645565 11531 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:00.648198 11537 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:00.655818  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:00.653855 11306 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:00.664791  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 178891us
I20260709 13:14:00.670258  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:00.618848 11499 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:00.677490  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:00.639640 11368 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:00.687098  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.691780  8130 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:00.675024  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:00.611575 11339 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:00.653612 11471 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:00.691859  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:00.763984  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:00.700830  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:00.765618  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1521) took 1341 ms. Trace:
I20260709 13:14:00.844676  8725 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:14:00.846282 11368 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:00.782866 11529 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.878s	user 0.000s	sys 0.008s
W20260709 13:14:00.759368 11511 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.935s	user 0.004s	sys 0.000s
I20260709 13:14:00.845245 11358 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:00.706768 11551 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:00.846017  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 154257us
I20260709 13:14:00.761914  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:00.858795  8056 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:14:00.859617  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:00.860167  8056 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:14:00.861518  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:00.844408  8206 rpcz_store.cc:276] 0709 13:13:59.424480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.470917 (+ 46437us) service_pool.cc:224] Handling call
0709 13:13:59.471210 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:59.777577 (+306367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.777620 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:59.786725 (+  9105us) log.cc:844] Serialized 26 byte log entry
0709 13:14:00.570487 (+783762us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:00.570524 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:00.570585 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:00.765401 (+194816us) raft_consensus.cc:1685] finished
0709 13:14:00.765428 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:00.765567 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":52916,"raft.run_cpu_time_us":4460,"raft.run_wall_time_us":183202,"thread_start_us":1055972,"threads_started":2,"wal-append.queue_time_us":885188}
I20260709 13:14:00.863193  8131 raft_consensus.cc:2368] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:14:00.863857  8693 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:00.864560  8056 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:14:00.864881 11436 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:00.865267 11436 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.865536 11551 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:00.691951 11553 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:00.854131  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:00.695267 11208 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:00.865612 11436 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:00.868643 11208 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:00.869122  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:00.878119  8132 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:00.725122 11550 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.870852 11532 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:00.727846 11382 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:00.874387 11306 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:00.722107 11430 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.852124 11499 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:00.879194  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:00.869534 11553 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:00.705981  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:00.670215 11543 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:00.882311  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.882525  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:00.883208  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:00.883765  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:00.660472 10939 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:00.889532  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:00.892388  8211 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
W20260709 13:14:00.741737 11510 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.858s	user 0.002s	sys 0.006s
I20260709 13:14:00.896473  8207 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:00.896564  8204 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:14:00.904336  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:00.721958 11375 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.694656 11425 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:00.722134 11005 consensus_queue.cc:237] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:00.901746  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:00.938153 10939 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.089186  8062 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:01.090909 10939 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.091470 10939 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:00.936131 11430 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.092221 11532 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.092783 11375 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:00.899019 11531 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:00.965099  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 259206us
I20260709 13:14:00.898787 11537 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.161151 11531 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.161410 11537 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.161758  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:01.162516 11206 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:01.162932  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:01.163190 11531 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.163712 11349 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.163820 11206 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:01.164212 11349 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.094913  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:01.165141  8056 raft_consensus.cc:2368] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:14:01.094924 11553 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:01.165920 11553 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.167991 11553 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.168548 11495 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:00.935091 10740 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:14:00.962507  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 63215.4us
I20260709 13:14:00.994930 11020 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:00.935775  8061 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:14:01.076387 11551 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:01.171639  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 575) took 1721 ms. Trace:
I20260709 13:14:01.171463 11551 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.164716  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:01.172004  8724 rpcz_store.cc:276] 0709 13:13:59.449861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.452345 (+  2484us) service_pool.cc:224] Handling call
0709 13:13:59.452609 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:00.134920 (+682311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.171433 (+1036513us) spinlock_profiling.cc:243] Waited 675 ms on lock 0x7b54004d21d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:01.171588 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1553024384}
I20260709 13:14:01.172806 11020 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:01.165994 11375 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:01.160696 11496 log.cc:927] Time spent T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 1.794s	user 0.007s	sys 0.000s
I20260709 13:14:00.998361 11472 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:01.164502 11206 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:00.827477 11548 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.124346  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:00.886543 11327 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:00.963233 11566 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 13:14:00.962728 11498 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:14:00.886580 11382 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:01.177950  8175 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:01.191948  8175 leader_election.cc:400] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:14:01.192307  8175 leader_election.cc:403] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:01.192929 11206 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:00.985930 11555 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:01.039340 11510 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:14:01.166116 11349 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.000424 11410 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:01.187565  8062 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:14:01.065320 11560 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.063674 11561 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.891443 11436 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:01.013876 11562 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:01.067886 11220 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:01.160195 11425 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.193754 11551 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.180094 11375 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:14:01.193611 11206 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.193569 11486 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:01.188269 11472 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:01.007545 11208 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:01.140702 11543 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.173808 10939 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.210984 11349 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.981312 11565 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:01.190582 11327 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:01.192966 11548 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.232520 11206 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:00.938903 11550 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:01.232335 11425 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.245080 11486 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:14:01.211144 11410 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.277755  8059 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:01.224921 11382 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.245195 11349 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.286574 11382 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:01.288053 11349 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.253736 11208 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:01.291535 11327 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:01.292248 11327 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:01.292712 11327 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.297705 11550 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:01.298233 11537 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.299572 11327 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:01.303131 11472 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.305282 11425 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.254716 10939 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:01.320475  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.199054 11555 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.325206  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.330468  8129 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:01.335803  8129 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:14:01.340235  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.306267 11430 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:01.341039  8132 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:14:01.324148 11425 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:01.329393 11537 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:01.238011 11562 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.354380  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2533) took 1852 ms. Trace:
I20260709 13:14:01.356704 11537 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.365384 11537 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.351320 11532 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:01.347895  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:01.386420  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:01.387012  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.388816  8130 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:01.253412 11565 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.352123 11368 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:01.355880 11425 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.408947  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.236665 11561 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.385468 11532 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.411751  8132 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:14:01.409137 11436 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.413832  8130 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:01.232788 11560 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.412942 11486 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:14:01.444659 11425 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.448663 11548 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.463101  8134 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:14:01.466706  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:01.412954 11555 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.464339 11174 raft_consensus.cc:3055] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:01.474197 11382 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:01.499030  8694 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.023s	sys 0.016s
W20260709 13:14:01.499874  8694 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.023s	sys 0.016s
I20260709 13:14:01.473908 11537 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:01.490530  8104 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.007s	sys 0.029s
W20260709 13:14:01.500955  8104 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.007s	sys 0.029s
I20260709 13:14:01.501086  8101 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:01.510111  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:01.510208  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.511134  8137 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:01.511164  8136 raft_consensus.cc:2393] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:01.512995  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:01.513234  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.477804 11555 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:01.515148  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.482353 11174 raft_consensus.cc:740] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:01.487268 11425 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.531169  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:01.532388  8136 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:01.518164  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.480340 11208 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:01.519389  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:01.485725 11543 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:01.512418  8206 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.009s	sys 0.009s
I20260709 13:14:01.536433  8208 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:01.536273  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:01.514693 11573 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:01.537878 11208 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:01.544201  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:01.547083  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:01.557400  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:01.552340  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:01.524022  8729 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.013s	sys 0.007s
W20260709 13:14:01.553037  8206 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.009s	sys 0.009s
I20260709 13:14:01.514434 11410 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.554642  8211 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
W20260709 13:14:01.558198  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.560765  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:01.599612  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:01.515517 11537 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.582389  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:01.509617 11382 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.612845  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:01.611963  8729 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.014s	sys 0.008s
W20260709 13:14:01.612708  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 51831.6us
I20260709 13:14:01.615769  8175 leader_election.cc:400] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8.
I20260709 13:14:01.545538 11573 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:01.572407  8721 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.030s	sys 0.045s
W20260709 13:14:01.630282  8721 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.034s	sys 0.046s
I20260709 13:14:01.598168  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:01.599720 11358 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.015s	sys 0.014s
I20260709 13:14:01.617847 11561 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.530056 11555 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.646220  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:01.647004  8026 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:01.646919  8056 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
W20260709 13:14:01.639568 11358 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.015s	sys 0.014s
I20260709 13:14:01.653405  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:01.653882 11570 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 13:14:01.653405  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:01.654070  8137 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:01.654721  8061 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:01.655049  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.656282  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.657141 11565 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.657508  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.657737 11565 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:01.658300 11565 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:01.658955 11572 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:14:01.660454 11565 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:01.662393  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:01.663123  8135 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:01.663379  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:01.663638 11555 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.663991  8061 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:14:01.664322 11586 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.664836  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:01.664781 11586 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.665427  8061 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:01.665802  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:01.537544 11425 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.666357  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:01.608175 11486 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:01.664013  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.680030  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.680644  8135 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:01.677347  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:01.598978 11551 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:01.608570 11543 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:01.676108  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:01.680786  8101 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:01.558665 11220 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.642395  8208 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:01.646158 11573 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:01.676604  8061 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:14:01.688643  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:01.692340  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45441.7us
I20260709 13:14:01.681711  8211 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:01.666860 11579 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:02.022917  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1533) took 1156 ms. Trace:
I20260709 13:14:02.050374  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.970613  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:02.036201  8104 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.67:35745 timed out after 1.830s (SENT) [suppressed 18 similar messages]
I20260709 13:14:01.946846  8729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 544) took 1310 ms. Trace:
W20260709 13:14:02.012810  8057 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2539) took 1666 ms (client timeout 1830 ms). Trace:
I20260709 13:14:01.952427  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.787807  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.021070  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:01.734879  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 118333us
I20260709 13:14:01.788074  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.028908  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1534) took 1160 ms. Trace:
I20260709 13:14:01.734284  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.054884  8132 raft_consensus.cc:2410] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:14:01.867376  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:02.051469  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.020689  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1209) took 1383 ms. Trace:
I20260709 13:14:02.021272  8101 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:02.051877  8729 rpcz_store.cc:276] 0709 13:14:00.636942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.637690 (+   748us) service_pool.cc:224] Handling call
0709 13:14:00.880310 (+242620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:00.880684 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:00.880708 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:00.888906 (+  8198us) log.cc:844] Serialized 26 byte log entry
0709 13:14:01.615161 (+726255us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:01.615199 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.615255 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:01.946596 (+331341us) raft_consensus.cc:1685] finished
0709 13:14:01.946623 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:01.946778 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":10687,"thread_start_us":610404,"threads_started":1,"wal-append.queue_time_us":773108}
I20260709 13:14:01.674947 11578 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.057797  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:02.057112  8134 rpcz_store.cc:276] 0709 13:14:00.637147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.637538 (+   391us) service_pool.cc:224] Handling call
0709 13:14:00.637848 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:01.463641 (+825793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.020414 (+556773us) spinlock_profiling.cc:243] Waited 825 ms on lock 0x7b5400062054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:02.020622 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1897233280}
I20260709 13:14:02.054250  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:02.054764  8212 rpcz_store.cc:276] 0709 13:14:00.868861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.868973 (+   112us) service_pool.cc:224] Handling call
0709 13:14:00.869308 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:00.885286 (+ 15978us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:00.885326 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:00.885787 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:14:01.949937 (+1064150us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b5400149c20. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf80090 00007f6dbaf8d6b8 00007f6dbb01692b 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c
0709 13:14:01.950023 (+    86us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:01.950083 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.950174 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:01.950230 (+    56us) raft_consensus.cc:1685] finished
0709 13:14:01.950249 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.028835 (+ 78586us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15604,"raft.queue_time_us":15714,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"spinlock_wait_cycles":36062976,"thread_start_us":452425,"threads_started":1,"wal-append.queue_time_us":450330}
I20260709 13:14:02.050542  8206 rpcz_store.cc:276] 0709 13:14:00.866963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.867122 (+   159us) service_pool.cc:224] Handling call
0709 13:14:00.867376 (+   254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:00.889661 (+ 22285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:00.889691 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:00.897338 (+  7647us) log.cc:844] Serialized 26 byte log entry
0709 13:14:01.613446 (+716108us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:01.613770 (+   324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.614372 (+   602us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.022715 (+408343us) raft_consensus.cc:1685] finished
0709 13:14:02.022742 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.022868 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21858,"raft.queue_time_us":22010,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":1863,"thread_start_us":607510,"threads_started":1,"wal-append.queue_time_us":1050910}
I20260709 13:14:02.061556  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:02.049628  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.043867  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:02.078423  8061 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:02.023485  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 711) took 1583 ms. Trace:
W20260709 13:14:02.052505  8057 rpcz_store.cc:269] 0709 13:14:00.332300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.332421 (+   121us) service_pool.cc:224] Handling call
0709 13:14:01.989477 (+1657056us) spinlock_profiling.cc:243] Waited 79.4 ms on lock 0x7b5400051754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:01.998178 (+  8701us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182725504}
I20260709 13:14:02.052961  8131 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:02.059365  8175 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:02.056936  8135 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
W20260709 13:14:02.122749  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 44353.2us
I20260709 13:14:02.051684  8026 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:02.129338  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:02.129905  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 78287.8us
I20260709 13:14:01.767518 11376 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:02.127216  8728 rpcz_store.cc:276] 0709 13:14:00.440006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.440231 (+   225us) service_pool.cc:224] Handling call
0709 13:14:00.440512 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:00.582137 (+141625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:00.582164 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:00.582203 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:00.582231 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.023219 (+1440988us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5400198f54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:02.023277 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.023409 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":325284096}
I20260709 13:14:02.131207  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.064527  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:02.131866  8135 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:14:02.128165  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:02.132678 11349 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:02.132870  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:02.133060 11349 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.133464 11349 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:01.975710 11327 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.156s
I20260709 13:14:02.134645  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.134124  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:02.132936  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:02.024447 11512 log.cc:927] Time spent T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 2.143s	user 0.006s	sys 0.012s
I20260709 13:14:02.150523  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:02.152086  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.787497 11413 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:14:02.135926  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 76654.2us
I20260709 13:14:02.135980  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.187212  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:02.187898  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.127391 11348 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:14:01.788925 11206 consensus_meta.cc:287] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.534s	user 0.004s	sys 0.003s
I20260709 13:14:01.768165 11560 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:02.152685  8132 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:14:01.707562 11580 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:01.903040 11168 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.020404 11562 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:02.155019  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:02.179205  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:01.767426 11417 raft_consensus.cc:3055] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:14:01.951936 11538 consensus_queue.cc:1175] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d9cd2999aa314b39997b423ec1ae509e" responder_term: 7 status { last_received { term: 7 index: 6 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:01.688329 11486 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:01.938735 11424 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:02.079625 11489 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:01.693565 11551 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.179884 11577 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:01.693934 11174 consensus_queue.cc:260] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.818674 11220 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.022639 11425 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.987413 11536 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.131067 11376 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:01.997334 11575 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.188359 11596 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.182695  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:01.969743 11588 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:01.955120 11493 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.217571 11420 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:01.865173 10939 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.939181 11436 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:01.734921 11472 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.734615 11553 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.957125 11586 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:01.706887 11561 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:02.135823  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:02.234930 11588 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:02.206413 11580 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.128348 11578 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.137302 11585 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:02.212320  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2659) took 2919 ms. Trace:
I20260709 13:14:02.290167  8210 rpcz_store.cc:276] 0709 13:13:59.293260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.294886 (+  1626us) service_pool.cc:224] Handling call
0709 13:13:59.295157 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:59.302188 (+  7031us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.302214 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:59.325880 (+ 23666us) log.cc:844] Serialized 26 byte log entry
0709 13:13:59.830568 (+504688us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:13:59.830604 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.830688 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.023857 (+2193169us) spinlock_profiling.cc:243] Waited 63.7 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:02.212020 (+188163us) raft_consensus.cc:1685] finished
0709 13:14:02.212042 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.212260 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6426,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":18812,"spinlock_wait_cycles":146462592,"thread_start_us":425318,"threads_started":1,"wal-append.queue_time_us":493025}
I20260709 13:14:01.954190 11518 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.229s
I20260709 13:14:02.219098  8057 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:02.137300 11507 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.222631  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:02.187999 11368 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:01.706607 11543 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.300858 11543 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:01.992920 11555 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:02.192413 11348 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:02.079881 11364 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:02.232110  8101 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:02.218807 11417 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:02.301692 11348 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:02.301762 11364 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.302445 11364 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:02.225916 11174 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:02.202798 11168 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.193759 11206 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:02.308228  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 76179.1us
I20260709 13:14:02.195478  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:02.243883 11493 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:02.063726 11579 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.274727 11562 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:02.293114  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 257024us
I20260709 13:14:02.242082 11598 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:02.223217 11516 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:14:02.303731  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 116617us
I20260709 13:14:02.276530 11472 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.226536 11575 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.284514 11588 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.275360 11436 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.245749 11420 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.278900 11553 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:02.315165  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 77812.4us
W20260709 13:14:02.307930 11495 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.013s	sys 0.008s
I20260709 13:14:02.319674  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:02.303812  8060 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:02.301641 11555 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.322172 11579 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.312836 11368 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:02.322486 11425 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.355664  8058 rpcz_store.cc:276] 0709 13:13:59.498505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.499377 (+   872us) service_pool.cc:224] Handling call
0709 13:13:59.499756 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.499985 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:00.894681 (+1394696us) spinlock_profiling.cc:243] Waited 6.75 ms on lock 0x7b1800461b60. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf
0709 13:14:00.894800 (+   119us) log.cc:844] Serialized 14 byte log entry
0709 13:14:01.350073 (+455273us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:01.350119 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:01.350167 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.350229 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:01.351297 (+  1068us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24940,"prepare.queue_time_us":338829,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":3712,"raft.queue_time_us":1404,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":2421,"spinlock_wait_cycles":15518464,"thread_start_us":681872,"threads_started":2}
W20260709 13:14:02.341034 11495 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.014s	sys 0.009s
I20260709 13:14:02.276401 10939 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.332278  8057 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:02.235289 11596 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:02.333608  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 253706us
I20260709 13:14:02.439023  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:02.336411  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:02.439889  8724 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:02.340481  8101 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:02.325625 11516 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.342708  8135 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
W20260709 13:14:02.440817  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 100407us
I20260709 13:14:02.365674  8056 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
W20260709 13:14:02.325734  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.370596  8137 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:02.372359 11580 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:02.372977 11595 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:14:02.378077 11486 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.325621 11562 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.304836 11507 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.324954 11598 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.347316 11368 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.342382 11600 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.426955 11335 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:02.302435 11417 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.424469  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.373407 11327 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:14:02.374444  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1609) took 3150 ms. Trace:
I20260709 13:14:02.373275 11607 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:14:02.368834 11610 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:02.427860 11538 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:02.438764 11594 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.204s	user 0.001s	sys 0.004s
I20260709 13:14:02.443779 11516 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:02.481531 11616 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.454947 11424 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.424310 11603 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:02.486174 11551 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.502763 11578 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:02.484159  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45250.7us
I20260709 13:14:02.485400  8133 rpcz_store.cc:276] 0709 13:13:59.224488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.225344 (+   856us) service_pool.cc:224] Handling call
0709 13:13:59.248559 (+ 23215us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:00.105953 (+857394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.374198 (+2268245us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b54001aa754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:02.374391 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1970612608}
I20260709 13:14:02.487385 11586 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.455624 11486 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:02.510128  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.487649  8061 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:02.449900 11597 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:02.397243 11425 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.455668 11413 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:02.467259 11368 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.464968 11353 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.513679 11597 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.521852 11597 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.499524 11596 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.486985 11417 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:02.482486 11220 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:02.500190  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:02.454833 11608 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:14:02.412676 11376 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:02.514197  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 172118us
I20260709 13:14:02.532882  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:02.533613 11562 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.521341 11353 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.536149 11353 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.509871 11598 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.451100 11349 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.525400  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.539434  8133 raft_consensus.cc:2410] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:02.529485  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1220) took 1087 ms. Trace:
I20260709 13:14:02.488116 11335 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.542594  8130 rpcz_store.cc:276] 0709 13:14:01.442745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.442891 (+   146us) service_pool.cc:224] Handling call
0709 13:14:01.443289 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:01.465452 (+ 22163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.465495 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:01.465994 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:14:01.674803 (+208809us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:01.674846 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.674925 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.529244 (+854319us) raft_consensus.cc:1685] finished
0709 13:14:02.529264 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.529416 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21696,"raft.queue_time_us":24251,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":8861,"thread_start_us":183720,"threads_started":1,"wal-append.queue_time_us":725700}
I20260709 13:14:02.495548 11538 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.517061 11168 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:02.522006 11464 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.484117 11600 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.520046 11206 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.560002 11600 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:02.563282  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1626) took 1051 ms. Trace:
I20260709 13:14:02.531214 11436 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:02.560822  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:59846 (request call id 81) took 1676 ms. Trace:
I20260709 13:14:02.563756  8130 rpcz_store.cc:276] 0709 13:14:01.512221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.562444 (+1050223us) service_pool.cc:224] Handling call
0709 13:14:02.562788 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:02.562974 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:02.562998 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:02.563041 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:02.563073 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.563127 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.563226 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:02.503541 11424 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.564251  8203 rpcz_store.cc:276] 0709 13:14:00.884863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.894849 (+  9986us) service_pool.cc:224] Handling call
0709 13:14:02.560502 (+1665653us) spinlock_profiling.cc:243] Waited 41.7 ms on lock 0x7b540019e6d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01c735 00007f6dbb0149fa 00007f6dbffdc4c9 00007f6db9558c54 00007f6db9558be2 00007f6db9558b11 00007f6db9558a8c 00007f6db9557d42 00007f6db935148c 00007f6db93508c6
0709 13:14:02.560729 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95956864}
I20260709 13:14:02.563491  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:02.571550  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.574025  8130 raft_consensus.cc:1217] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 2.5->[3.6-10.9]   Dedup: 7.7->[8.8-10.9]
I20260709 13:14:02.574803  8130 pending_rounds.cc:85] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 7
I20260709 13:14:02.575147  8130 pending_rounds.cc:107] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:14:02.575488  8130 raft_consensus.cc:2889] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:02.503355 11616 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.537883 11348 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.588625  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1625) took 1076 ms. Trace:
I20260709 13:14:02.589118  8133 rpcz_store.cc:276] 0709 13:14:01.511963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.551199 (+1039236us) service_pool.cc:224] Handling call
0709 13:14:02.587998 (+ 36799us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:02.588213 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:02.588250 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:02.588307 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:02.588349 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.588417 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.588559 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:02.590194  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:02.548128 11220 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:02.516772 11486 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:02.593116  7948 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b578d88c1c1f4ca58b7291fed4909f09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica d9cd2999aa314b39997b423ec1ae509e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:02.593945 10939 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.594111 11600 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:02.594573 11600 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:02.594959 11600 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.515790 11586 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.597998  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:02.596524  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1622) took 1085 ms. Trace:
W20260709 13:14:02.601922  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.616564  8131 rpcz_store.cc:276] 0709 13:14:01.511190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.286662 (+775472us) service_pool.cc:224] Handling call
0709 13:14:02.294751 (+  8089us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:02.295005 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:02.295041 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:02.310970 (+ 15929us) log.cc:844] Serialized 26 byte log entry
0709 13:14:02.491010 (+180040us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:02.491047 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.491109 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.596254 (+105145us) raft_consensus.cc:1685] finished
0709 13:14:02.596282 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.596448 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":150113,"threads_started":1,"wal-append.queue_time_us":204572}
I20260709 13:14:02.619208 11507 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:02.585326 11598 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:02.528575 11413 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.572723 11335 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.509035 11603 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:02.627512  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 122927us
I20260709 13:14:02.597244 11220 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.628548  8131 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:14:02.635107 11615 consensus_queue.cc:1059] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:02.635893 11555 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.624223 11598 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.596153 11495 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:02.622514 11472 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:02.620628 11206 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:02.651206  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1637) took 1136 ms. Trace:
I20260709 13:14:02.621970 11607 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:02.653648  8721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 583) took 1666 ms. Trace:
I20260709 13:14:02.649765  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:02.656755  8131 rpcz_store.cc:276] 0709 13:14:01.515091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.618232 (+1103141us) service_pool.cc:224] Handling call
0709 13:14:02.628261 (+ 10029us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:02.650764 (+ 22503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.651091 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:02.588398 11616 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.637899 11353 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.663849  8693 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:02.642891 11220 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:02.660216  8721 rpcz_store.cc:276] 0709 13:14:00.987498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.987655 (+   157us) service_pool.cc:224] Handling call
0709 13:14:01.014756 (+ 27101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:01.015004 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:14:01.015291 (+   287us) log.cc:844] Serialized 14 byte log entry
0709 13:14:01.994177 (+978886us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b180041faa0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf 00007f6dbb030038
0709 13:14:02.290360 (+296183us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.653350 (+362990us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:02.653399 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:02.653441 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.653499 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.653598 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5862,"prepare.queue_time_us":644084,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":80,"raft.queue_time_us":241553,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":31985,"spinlock_wait_cycles":10697472,"thread_start_us":1267757,"threads_started":3,"wal-append.queue_time_us":1013979}
I20260709 13:14:02.655911 11598 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:02.656265 11005 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:02.678802  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:02.659162 11607 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.679983  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1393) took 1012 ms. Trace:
I20260709 13:14:02.680429 11348 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:02.685767  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:02.686656  8131 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:02.689014 11603 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:02.689740  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:02.697177  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:02.690310  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:02.697973  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:02.701610  8722 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:02.691039  8058 rpcz_store.cc:276] 0709 13:14:01.667164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.366334 (+699170us) service_pool.cc:224] Handling call
0709 13:14:02.367161 (+   827us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:02.367648 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:02.367689 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:02.368249 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 13:14:02.566962 (+198713us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:02.567002 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.567067 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.679704 (+112637us) raft_consensus.cc:1685] finished
0709 13:14:02.679722 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.679899 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":12313,"thread_start_us":123627,"threads_started":1,"wal-append.queue_time_us":201174}
I20260709 13:14:02.664908 11472 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:02.666674 11206 raft_consensus.cc:697] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:02.698923  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:02.696362  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:02.702474  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:02.705063  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:02.706077  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:02.706667  8058 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:02.693351 11607 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:02.659991 11495 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.714381  8058 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:02.696072 11538 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.716375  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1396) took 1048 ms. Trace:
I20260709 13:14:02.693939 11425 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.714408  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:02.682019 11353 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:02.736572  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.710297  8693 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:02.738757  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2546) took 1125 ms. Trace:
W20260709 13:14:02.764909  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:02.764933  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 54869.9us
I20260709 13:14:02.716795  8058 rpcz_store.cc:276] 0709 13:14:01.667981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.713077 (+1045096us) service_pool.cc:224] Handling call
0709 13:14:02.713357 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:02.716079 (+  2722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.716325 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:02.764864  8062 rpcz_store.cc:276] 0709 13:14:01.612978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.613092 (+   114us) service_pool.cc:224] Handling call
0709 13:14:01.613334 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:02.226920 (+613586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:02.226954 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:02.226994 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:02.227023 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.738544 (+511521us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b540025f8d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:02.738588 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.738700 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1410685184}
I20260709 13:14:02.694012 11621 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:02.784799  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:02.722898 11600 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.720072 11553 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:02.712258 11586 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.824738  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1223) took 1285 ms. Trace:
I20260709 13:14:02.825316  8137 rpcz_store.cc:276] 0709 13:14:01.538939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.680207 (+141268us) service_pool.cc:224] Handling call
0709 13:14:02.824523 (+1144316us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b540015b154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:02.824662 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724984576}
I20260709 13:14:02.801363 11600 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:02.829599 11624 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.791673  8722 raft_consensus.cc:2410] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:14:02.766811 11349 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:02.719153 11493 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:02.830119 11624 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.774673 11603 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:02.732007 11588 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.802767 11486 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.779407 11614 raft_consensus.cc:3055] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:02.835980 11555 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:02.842880 11585 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:02.843096 11621 raft_consensus.cc:697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:02.864511  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.788642 11424 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.856729 11413 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.866531  8131 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:02.822659 11220 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.867388  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:02.868211  8137 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 7: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
W20260709 13:14:02.871390 11628 log.cc:927] Time spent T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:14:02.720307 11206 consensus_queue.cc:237] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.855791 11603 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.881093 11603 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:02.881521  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.882138  8137 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:02.882911  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.883402  8137 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:14:02.884141  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.884629  8137 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:02.885391  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.885852  8137 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:14:02.886020 11603 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.888928  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.887624 11524 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:02.893220  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1629) took 1380 ms. Trace:
I20260709 13:14:02.864475 11614 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:02.893599  8130 rpcz_store.cc:276] 0709 13:14:01.513092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.573209 (+1060117us) service_pool.cc:224] Handling call
0709 13:14:02.573720 (+   511us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:14:02.576362 (+  2642us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:14:02.597705 (+ 21343us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.632762 (+ 35057us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.632845 (+    83us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:02.633477 (+   632us) log.cc:844] Serialized 52 byte log entry
0709 13:14:02.633599 (+   122us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:02.633761 (+   162us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.634364 (+   603us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.634523 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.634591 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.892986 (+258395us) raft_consensus.cc:1685] finished
0709 13:14:02.893009 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.893164 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1618,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":9992,"spinlock_wait_cycles":42121856,"thread_start_us":34603,"threads_started":1,"wal-append.queue_time_us":67279}
I20260709 13:14:02.896485  8724 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:02.862751 11624 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:02.904978  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.908264  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.908840  8137 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:02.909799  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:02.910440  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:02.911273  8175 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:02.911877 11493 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:02.912340 11493 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:02.913369 11493 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:02.907312 11489 consensus_meta.cc:287] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.685s	user 0.010s	sys 0.001s
I20260709 13:14:02.924307  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.926105  8131 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:02.867810 11555 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.928639  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.931187  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.932612  8130 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:14:02.908597 11610 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.904984 11486 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:02.933706  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:02.926829 11573 consensus_meta.cc:287] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.831s	user 0.008s	sys 0.004s
I20260709 13:14:02.888377 11585 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.924556 11622 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:02.929883 11624 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:02.935420 11375 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.950245  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.949394 11413 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:02.955775 11413 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.956140 11598 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.958061 11598 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:02.939028 11614 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:02.959899 11588 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:02.957720 11413 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:02.962332  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.943400 11486 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.941414 11610 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.944160 11353 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:02.969986  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 409) took 2206 ms. Trace:
W20260709 13:14:02.971799  8056 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1398) took 1303 ms (client timeout 1709 ms). Trace:
W20260709 13:14:03.004328  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:02.968133  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:02.973541  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 407) took 2210 ms. Trace:
I20260709 13:14:03.010792  8055 rpcz_store.cc:276] 0709 13:14:00.762780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.762907 (+   127us) service_pool.cc:224] Handling call
0709 13:14:00.763210 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:00.858845 (+ 95635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:00.858876 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:00.858914 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:00.858942 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.973345 (+2114403us) spinlock_profiling.cc:243] Waited 95.5 ms on lock 0x7b540009bad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:02.973387 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.973494 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219558656}
W20260709 13:14:03.009305  8056 rpcz_store.cc:269] 0709 13:14:01.668459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.690522 (+ 22063us) service_pool.cc:224] Handling call
0709 13:14:02.971643 (+1281121us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b540043ce54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:02.971745 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":692731648}
I20260709 13:14:03.013980  8056 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:14:02.945717 11622 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:03.009121  8063 rpcz_store.cc:276] 0709 13:14:00.763942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.764112 (+   170us) service_pool.cc:224] Handling call
0709 13:14:00.783516 (+ 19404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:00.867083 (+ 83567us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:00.867111 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:00.867147 (+    36us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:00.867174 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.969705 (+2102531us) spinlock_profiling.cc:243] Waited 83.4 ms on lock 0x7b5400151154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:02.969766 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.969918 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191788928}
W20260709 13:14:02.970639 11618 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 13:14:02.971141 11526 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 3.090s	user 0.003s	sys 0.009s
I20260709 13:14:02.945010 11573 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:02.946066 11621 consensus_queue.cc:237] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:03.028299  8056 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:02.966007 11495 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:02.972744 11382 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:03.035395  8138 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:03.035526  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:03.037289  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:03.047080  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:03.047210  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2663) took 3752 ms. Trace:
I20260709 13:14:03.014389 11614 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:03.014674 11208 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:03.036068 11176 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:03.016597 11598 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:03.016264 11588 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:03.051702  8130 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:03.051697  8205 rpcz_store.cc:276] 0709 13:13:59.294300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.322436 (+ 28136us) service_pool.cc:224] Handling call
0709 13:13:59.322735 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:59.728254 (+405519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.728281 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:59.779062 (+ 50781us) log.cc:844] Serialized 26 byte log entry
0709 13:13:59.854936 (+ 75874us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:59.854988 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.730871 (+2875883us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b540019dcd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:02.730932 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.970615 (+239683us) spinlock_profiling.cc:243] Waited 64.4 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:03.046989 (+ 76374us) raft_consensus.cc:1685] finished
0709 13:14:03.047012 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:03.047152 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":85410,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":9214,"spinlock_wait_cycles":685122048,"thread_start_us":63261,"threads_started":1,"wal-append.queue_time_us":89473}
I20260709 13:14:03.061079  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:03.061606  8205 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:14:03.062404  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:03.063071  8205 raft_consensus.cc:2410] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:14:03.063692  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:03.064164  8205 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:14:03.064857  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:03.022872 11622 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:03.051375  8212 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:03.025911 11489 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:03.058305  8132 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:03.051785 11559 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.065321  8205 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:14:02.954389 11375 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:03.080291  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:03.081348  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:03.082713  8205 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:03.048043 11633 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:03.059664 11527 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.736s	user 0.036s	sys 0.048s
I20260709 13:14:03.088043  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:03.091439  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:03.092407  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:03.095695  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:03.096248  8205 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:14:03.097003  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:03.098425  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:03.099464  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:03.092679 11527 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.769s	user 0.036s	sys 0.049s
I20260709 13:14:03.089033 11495 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:03.068526 11598 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:03.068717 11588 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:03.108368 11632 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:14:03.116806 11597 consensus_meta.cc:287] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.571s	user 0.006s	sys 0.003s
I20260709 13:14:03.118119  8130 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:03.142506  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:03.143515  8130 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:14:03.128643  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.128756  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:03.129222  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:03.133370  8212 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:14:03.092226 11603 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:03.135792  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1394) took 1468 ms. Trace:
I20260709 13:14:03.068058 11176 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:03.088174 11489 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:03.147492  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:03.153893  8057 rpcz_store.cc:276] 0709 13:14:01.667466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.444533 (+777067us) service_pool.cc:224] Handling call
0709 13:14:02.444873 (+   340us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:02.454583 (+  9710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:02.454616 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:02.455212 (+   596us) log.cc:844] Serialized 52 byte log entry
0709 13:14:02.586708 (+131496us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:02.586761 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.586928 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:03.135573 (+548645us) raft_consensus.cc:1685] finished
0709 13:14:03.135592 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:03.135733 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8525,"raft.queue_time_us":5147,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":13816,"thread_start_us":94732,"threads_started":1,"wal-append.queue_time_us":205343}
I20260709 13:14:03.173524  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:03.155159 11597 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:03.174451 11489 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:03.083586 11559 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:03.189664 11532 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:03.177479  8205 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:03.191715 11364 consensus_meta.cc:287] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.879s	user 0.005s	sys 0.006s
I20260709 13:14:03.192102 11364 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:03.181062 11610 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:03.181697 11486 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:03.183952 11549 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 2.630s	user 0.006s	sys 0.009s
I20260709 13:14:03.185251 11598 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:03.186885  8027 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:03.187054 11588 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:03.135892 11495 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:03.188344  8212 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:14:03.203461  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:03.205412  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:03.262075  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:03.138739 11555 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:03.256366  8134 raft_consensus.cc:2393] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:14:03.256610  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 69793.1us
I20260709 13:14:03.094322 11633 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:03.175988 11174 consensus_meta.cc:287] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.868s	user 0.010s	sys 0.002s
I20260709 13:14:03.274610 11585 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:03.276947 11368 consensus_meta.cc:287] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.748s	user 0.008s	sys 0.000s
I20260709 13:14:03.277328 11368 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:03.278436  8137 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:03.279556 11176 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:03.286928 11417 consensus_meta.cc:287] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.745s	user 0.006s	sys 0.003s
I20260709 13:14:03.287678  8131 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
W20260709 13:14:03.289729 11516 consensus_meta.cc:287] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.790s	user 0.008s	sys 0.001s
I20260709 13:14:03.290156 11516 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:03.291697  8211 raft_consensus.cc:2393] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 8: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:03.293720  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:03.303081  8207 raft_consensus.cc:2393] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:03.315784  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:04.277765  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1660) took 1074 ms. Trace:
I20260709 13:14:04.278146  8130 rpcz_store.cc:276] 0709 13:14:03.203068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.262022 (+ 58954us) service_pool.cc:224] Handling call
0709 13:14:04.277712 (+1015690us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:03.342573  8207 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1553) took 1674 ms (client timeout 1789 ms). Trace:
W20260709 13:14:04.279585  8207 rpcz_store.cc:269] 0709 13:14:01.668528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.668658 (+   130us) service_pool.cc:224] Handling call
0709 13:14:01.997158 (+328500us) spinlock_profiling.cc:243] Waited 8.4 ms on lock 0x7b5800168398. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:14:03.293550 (+1296392us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b5800168398. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1e9c 00007f6db7ef1a5b 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:14:03.342497 (+ 48947us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30961408}
W20260709 13:14:03.383455  8138 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1631) took 1869 ms (client timeout 2215 ms). Trace:
W20260709 13:14:04.282614  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:03.939777  8175 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.027s	sys 0.050s
W20260709 13:14:04.285029  8175 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.207s	user 0.029s	sys 0.064s
W20260709 13:14:04.285426  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 1.20794e+06us
I20260709 13:14:04.280335  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2554) took 2618 ms. Trace:
I20260709 13:14:04.276729  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.287111  8061 rpcz_store.cc:276] 0709 13:14:01.662324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.285140 (+622816us) service_pool.cc:224] Handling call
0709 13:14:02.285465 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:02.513708 (+228243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.280123 (+1766415us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b540033efd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:04.280283 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":464495872}
I20260709 13:14:04.276741  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1635) took 2762 ms. Trace:
I20260709 13:14:04.288266  8133 rpcz_store.cc:276] 0709 13:14:01.514646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.590214 (+1075568us) service_pool.cc:224] Handling call
0709 13:14:02.623936 (+ 33722us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:02.624679 (+   743us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:02.624720 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:02.625263 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 13:14:02.895401 (+270138us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:02.895444 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.895512 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:03.191551 (+296039us) raft_consensus.cc:1685] finished
0709 13:14:03.191581 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.276682 (+1085101us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":227,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":11303,"thread_start_us":232474,"threads_started":1,"wal-append.queue_time_us":295801,"wal-append.run_cpu_time_us":2600,"wal-append.run_wall_time_us":1308734}
I20260709 13:14:03.261124 11561 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:04.287496  8212 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:14:04.293777  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2690) took 1111 ms. Trace:
W20260709 13:14:04.281361  8138 rpcz_store.cc:269] 0709 13:14:01.513561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.952170 (+438609us) service_pool.cc:224] Handling call
0709 13:14:03.383247 (+1431077us) spinlock_profiling.cc:243] Waited 983 ms on lock 0x7b5400042754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:03.383348 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2259946752}
I20260709 13:14:04.294106  8212 rpcz_store.cc:276] 0709 13:14:03.182027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.262940 (+ 80913us) service_pool.cc:224] Handling call
0709 13:14:04.293722 (+1030782us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:04.311527  8205 consensus_meta.cc:287] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 1.121s	user 0.019s	sys 0.032s
I20260709 13:14:04.312453  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.312597  8205 raft_consensus.cc:2410] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:04.313220  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2685) took 1132 ms. Trace:
I20260709 13:14:04.312768  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.313455  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:04.313575  8212 rpcz_store.cc:276] 0709 13:14:03.180482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.312196 (+1131714us) service_pool.cc:224] Handling call
0709 13:14:04.313158 (+   962us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.314196  8138 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:04.314002  8207 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:04.314635  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1652) took 1583 ms. Trace:
I20260709 13:14:04.314841  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2692) took 1050 ms. Trace:
I20260709 13:14:04.315019  8138 rpcz_store.cc:276] 0709 13:14:02.731287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.313234 (+1581947us) service_pool.cc:224] Handling call
0709 13:14:04.314582 (+  1348us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.315593  8207 rpcz_store.cc:276] 0709 13:14:03.265204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.312557 (+1047353us) service_pool.cc:224] Handling call
0709 13:14:04.314771 (+  2214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.316217  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:04.316946  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:03.944904 11206 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.010s	sys 0.025s
W20260709 13:14:03.357286 11611 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.779s	user 0.000s	sys 0.003s
W20260709 13:14:03.679713 11527 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.015s	sys 0.022s
W20260709 13:14:03.410573 11607 consensus_meta.cc:287] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.684s	user 0.004s	sys 0.007s
W20260709 13:14:04.300328 11635 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 1.211s	user 0.006s	sys 0.014s
W20260709 13:14:03.650913 11614 consensus_meta.cc:287] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.585s	user 0.003s	sys 0.007s
I20260709 13:14:04.322727  8138 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 13:14:04.324075  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.328163  8212 raft_consensus.cc:2410] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:14:04.324920  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:04.329104  8130 raft_consensus.cc:2410] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:14:04.329815  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1224) took 2781 ms. Trace:
I20260709 13:14:04.330261  8138 rpcz_store.cc:276] 0709 13:14:01.548205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.321933 (+2773728us) service_pool.cc:224] Handling call
0709 13:14:04.322407 (+   474us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:04.329484 (+  7077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.329750 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:03.375769 11420 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:03.397508 11633 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:04.336153  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:04.339908  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2686) took 1159 ms. Trace:
I20260709 13:14:04.340405  8212 rpcz_store.cc:276] 0709 13:14:03.180834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.314282 (+1133448us) service_pool.cc:224] Handling call
0709 13:14:04.339837 (+ 25555us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.301918 11561 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:04.319906 11206 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.139s	user 0.017s	sys 0.035s
W20260709 13:14:04.335717 11637 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 1.121s	user 0.009s	sys 0.030s
I20260709 13:14:03.376663 11622 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:04.335686 11588 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:04.353719  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 479) took 2219 ms. Trace:
I20260709 13:14:04.354574  8133 rpcz_store.cc:276] 0709 13:14:02.134141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.341820 (+2207679us) service_pool.cc:224] Handling call
0709 13:14:04.353128 (+ 11308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.353336 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:04.353372 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:04.353427 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:04.353471 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.353538 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.353655 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.360417  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 478) took 2226 ms. Trace:
I20260709 13:14:04.361001  8130 rpcz_store.cc:276] 0709 13:14:02.133762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.344785 (+2211023us) service_pool.cc:224] Handling call
0709 13:14:04.359806 (+ 15021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.360032 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:04.360068 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:04.360121 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:04.360166 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.360235 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.360353 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.362601  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.346829 11561 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:04.368115  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 515) took 1334 ms. Trace:
I20260709 13:14:04.371016  8204 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:04.371898  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2697) took 1002 ms. Trace:
I20260709 13:14:04.372452  8212 rpcz_store.cc:276] 0709 13:14:03.369202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.347406 (+978204us) service_pool.cc:224] Handling call
0709 13:14:04.371827 (+ 24421us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.369766  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 417) took 2376 ms. Trace:
I20260709 13:14:04.374243  8133 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:14:04.368654  8203 rpcz_store.cc:276] 0709 13:14:03.033611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.033730 (+   119us) service_pool.cc:224] Handling call
0709 13:14:03.034072 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:03.036276 (+  2204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:03.036319 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:03.036745 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 13:14:03.152702 (+115957us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:03.152746 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:03.152819 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:04.367835 (+1215016us) raft_consensus.cc:1685] finished
0709 13:14:04.367863 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.368045 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1838,"raft.queue_time_us":7403,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":105767,"threads_started":1,"wal-append.queue_time_us":155755}
I20260709 13:14:04.375556  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 481) took 2223 ms. Trace:
I20260709 13:14:04.376744  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:04.376977  8133 rpcz_store.cc:276] 0709 13:14:02.152363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.355744 (+2203381us) service_pool.cc:224] Handling call
0709 13:14:04.373918 (+ 18174us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:04.375222 (+  1304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.375492 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.374464  8058 rpcz_store.cc:276] 0709 13:14:01.993819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.766491 (+772672us) service_pool.cc:224] Handling call
0709 13:14:02.780725 (+ 14234us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:02.833320 (+ 52595us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:14:02.833350 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:02.833777 (+   427us) log.cc:844] Serialized 26 byte log entry
0709 13:14:02.938480 (+104703us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:02.938518 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.938583 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:03.296174 (+357591us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:04.365371 (+1069197us) raft_consensus.cc:1685] finished
0709 13:14:04.365401 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.369693 (+  4292us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55170,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":5582,"spinlock_wait_cycles":6828928,"thread_start_us":104413,"threads_started":1,"wal-append.queue_time_us":217690}
I20260709 13:14:04.339196 11607 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:04.348656 11420 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:14:04.387708  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1646) took 2156 ms. Trace:
I20260709 13:14:04.388302  8130 rpcz_store.cc:276] 0709 13:14:02.231501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.362265 (+2130764us) service_pool.cc:224] Handling call
0709 13:14:04.387097 (+ 24832us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.387321 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:04.387357 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:04.387413 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:04.387456 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.387524 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.387644 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.392524  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 476) took 2259 ms. Trace:
I20260709 13:14:04.373929 10940 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:03.395027 11495 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:04.363821 11588 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:04.400198  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:04.398906  8138 rpcz_store.cc:276] 0709 13:14:02.133246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.391566 (+2258320us) service_pool.cc:224] Handling call
0709 13:14:04.391914 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.392144 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:04.392180 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:04.392234 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:04.392275 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.392343 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.392462 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.402062  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.401646 10940 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:04.403612 11537 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:04.384677 11516 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:04.404452 11573 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:04.426754  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:04.427817  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:04.429045  8138 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:14:04.429517  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 483) took 2187 ms. Trace:
I20260709 13:14:04.429791  8138 rpcz_store.cc:276] 0709 13:14:02.242080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.428551 (+2186471us) service_pool.cc:224] Handling call
0709 13:14:04.428783 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.429310 (+   527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.429453 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.430945  8138 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:04.431396  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1227) took 1796 ms. Trace:
I20260709 13:14:04.431686  8138 rpcz_store.cc:276] 0709 13:14:02.634730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.430455 (+1795725us) service_pool.cc:224] Handling call
0709 13:14:04.430696 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.431207 (+   511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.431349 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.422577  8203 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:04.401091  8207 raft_consensus.cc:2393] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:04.424129  8212 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 6.
I20260709 13:14:04.423686  8133 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:14:04.434348  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2694) took 1080 ms. Trace:
I20260709 13:14:04.434621  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2688) took 1253 ms. Trace:
I20260709 13:14:04.436527  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:04.434741  8203 rpcz_store.cc:276] 0709 13:14:03.353649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.376471 (+1022822us) service_pool.cc:224] Handling call
0709 13:14:04.434291 (+ 57820us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.437110  8057 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:04.425009  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 942231us
I20260709 13:14:04.436565  8133 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:04.435434  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:04.437774  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.438055  8061 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:14:04.438459  8203 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:04.435163  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:04.439051  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2693) took 1088 ms. Trace:
I20260709 13:14:04.439359  8058 raft_consensus.cc:2410] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:14:04.436764  8212 rpcz_store.cc:276] 0709 13:14:03.181449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.398428 (+1216979us) service_pool.cc:224] Handling call
0709 13:14:04.434572 (+ 36144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.439352  8203 rpcz_store.cc:276] 0709 13:14:03.351074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.437564 (+1086490us) service_pool.cc:224] Handling call
0709 13:14:04.439008 (+  1444us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.438069  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1230) took 1802 ms. Trace:
I20260709 13:14:04.437901  8696 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:04.441695  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.440902  8133 rpcz_store.cc:276] 0709 13:14:02.635504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.434834 (+1799330us) service_pool.cc:224] Handling call
0709 13:14:04.436368 (+  1534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.437848 (+  1480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.438023 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.442572  8212 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:04.442993  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:04.443152  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2696) took 1074 ms. Trace:
I20260709 13:14:04.443809  8203 raft_consensus.cc:2410] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:14:04.443568  8212 rpcz_store.cc:276] 0709 13:14:03.368846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.440510 (+1071664us) service_pool.cc:224] Handling call
0709 13:14:04.443085 (+  2575us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.445041  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2689) took 1263 ms. Trace:
I20260709 13:14:04.445454  8203 rpcz_store.cc:276] 0709 13:14:03.181742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.442769 (+1261027us) service_pool.cc:224] Handling call
0709 13:14:04.444996 (+  2227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.446209  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.447759 10940 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:04.448882  8061 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:04.452478  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.453051  8203 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 5.
I20260709 13:14:04.455278  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.456507  8203 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
W20260709 13:14:04.458186 11639 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:14:04.458273  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.461227  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.461738  8203 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:04.464037  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.466971  8203 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:04.467511  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2687) took 1286 ms. Trace:
I20260709 13:14:04.467851  8203 rpcz_store.cc:276] 0709 13:14:03.181149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.463837 (+1282688us) service_pool.cc:224] Handling call
0709 13:14:04.467452 (+  3615us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.471616 11588 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:04.444435 11643 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:04.474386  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.474968  8212 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:04.472528 11643 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:04.476307  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.477540  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.478803  8203 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:14:04.478863  8212 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
W20260709 13:14:04.469568 11640 log.cc:927] Time spent T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:14:04.480392  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.481165  8212 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:04.482288  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.483034  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2683) took 1328 ms. Trace:
I20260709 13:14:04.483402  8212 rpcz_store.cc:276] 0709 13:14:03.154541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.482088 (+1327547us) service_pool.cc:224] Handling call
0709 13:14:04.482997 (+   909us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.487356  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.487888  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.482553 11643 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:04.493213  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 55382.6us
I20260709 13:14:04.495609  8696 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:04.494179 11625 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 1.821s	user 0.008s	sys 0.016s
I20260709 13:14:04.500357  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 522) took 1463 ms. Trace:
I20260709 13:14:04.501642  8203 rpcz_store.cc:276] 0709 13:14:03.037211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.499455 (+1462244us) service_pool.cc:224] Handling call
0709 13:14:04.499791 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.499986 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:04.500022 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:04.500075 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:04.500120 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.500187 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.500298 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.505939  8203 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:14:04.488915  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.506837  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 524) took 1469 ms. Trace:
I20260709 13:14:04.507124  8203 rpcz_store.cc:276] 0709 13:14:03.037664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.504565 (+1466901us) service_pool.cc:224] Handling call
0709 13:14:04.504876 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.506608 (+  1732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.506785 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.508327  8212 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:04.508783  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 533) took 1468 ms. Trace:
I20260709 13:14:04.509706  8212 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:04.509251  8203 rpcz_store.cc:276] 0709 13:14:03.039925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.507871 (+1467946us) service_pool.cc:224] Handling call
0709 13:14:04.508125 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.508288 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:04.508320 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:04.508366 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:04.508399 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.508454 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.508738 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.510586  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 534) took 1470 ms. Trace:
I20260709 13:14:04.508843 11607 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:04.511394  8203 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:04.512305  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2681) took 1416 ms. Trace:
I20260709 13:14:04.512822 11375 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:04.512670  8203 rpcz_store.cc:276] 0709 13:14:03.096105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.510856 (+1414751us) service_pool.cc:224] Handling call
0709 13:14:04.511177 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.512052 (+   875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.512256 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.515004 11643 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:04.514444 11561 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:04.515401 11643 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:04.515811 11643 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:04.515892 11364 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:04.517441 11561 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:04.514427  8203 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:04.520226 11306 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:04.522404 11575 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:04.524923  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.528388  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.528482 11643 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:04.529034  8722 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:14:04.537321  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.538220  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.538355  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.539191  8725 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:14:04.539340  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.540324 11649 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.540839 11649 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:04.541836  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.542670  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:04.542886 11649 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:04.543424 11649 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:04.538496 11643 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:04.543900 11649 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:04.544713 11622 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:04.545279 11643 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:04.526191  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.545837 11649 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:04.545776 11643 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:04.546398  8724 raft_consensus.cc:2368] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12.
I20260709 13:14:04.547852 11643 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:04.545840  8730 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: []
W20260709 13:14:04.555828 11582 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 2.608s	user 0.007s	sys 0.010s
W20260709 13:14:04.556258  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 136483us
I20260709 13:14:04.558835 11647 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:04.570837 11005 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:04.571283 11005 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:04.575351  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.577323  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.577872 11220 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:04.578303  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.579892  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.581008  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:04.559629 11306 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:04.558982  8693 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:04.582523 11306 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:04.583025 11306 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:04.583205  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.584762  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.585322 11369 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:04.578068  8722 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:04.579895 11597 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:04.594136  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.597862 11368 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:04.604326 11306 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:04.618922 11559 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:04.622926 11493 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.487s	user 0.018s	sys 0.043s
W20260709 13:14:04.623365 11493 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.517s	user 0.018s	sys 0.043s
I20260709 13:14:04.626212 11465 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.626756 11465 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:01.997892  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:14:04.673398  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 205151us
W20260709 13:14:04.678555  8026 leader_election.cc:336] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.749s (SENT)
I20260709 13:14:04.679486  8026 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:04.681249  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 406) took 3919 ms. Trace:
I20260709 13:14:04.681788  8054 rpcz_store.cc:276] 0709 13:14:00.761813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.761942 (+   129us) service_pool.cc:224] Handling call
0709 13:14:00.762318 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:01.294858 (+532540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.294890 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:01.295325 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:14:01.553472 (+258147us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:01.553527 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.954260 (+400733us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:04.680295 (+2726035us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b540009c254. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:04.680416 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:04.680571 (+   155us) raft_consensus.cc:1685] finished
0709 13:14:04.680602 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.680979 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1022,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"spinlock_wait_cycles":1240260224,"thread_start_us":257751,"threads_started":1,"wal-append.queue_time_us":648825,"wal-append.run_cpu_time_us":4306,"wal-append.run_wall_time_us":1068164}
I20260709 13:14:04.665992  7959 catalog_manager.cc:5697] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 3 to 6, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 6 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:04.476532  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1235) took 1816 ms. Trace:
I20260709 13:14:04.717770  8131 rpcz_store.cc:276] 0709 13:14:02.659793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.940161 (+280368us) service_pool.cc:224] Handling call
0709 13:14:04.476391 (+1536230us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400174154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:04.476482 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774519552}
I20260709 13:14:04.511029  8212 rpcz_store.cc:276] 0709 13:14:03.040142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.509144 (+1469002us) service_pool.cc:224] Handling call
0709 13:14:04.509407 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.510333 (+   926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.510537 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.718725 11306 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:04.720307  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:04.720968  8212 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:14:04.720090 11306 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:04.722105  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 421) took 2677 ms. Trace:
I20260709 13:14:04.736114  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 482) took 2500 ms. Trace:
I20260709 13:14:04.736596  8130 rpcz_store.cc:276] 0709 13:14:02.235409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.389551 (+2154142us) service_pool.cc:224] Handling call
0709 13:14:04.421361 (+ 31810us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.735457 (+314096us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:14:04.735497 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:04.735557 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:04.735586 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.735698 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.735897 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20862,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":19307,"thread_start_us":313337,"threads_started":1}
W20260709 13:14:04.740810  8132 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1650) took 2249 ms (client timeout 2612 ms). Trace:
W20260709 13:14:04.741374  8132 rpcz_store.cc:269] 0709 13:14:02.491005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.545997 (+ 54992us) service_pool.cc:224] Handling call
0709 13:14:04.740232 (+2194235us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b54003c96d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:04.740539 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1116697984}
W20260709 13:14:04.743642  8134 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1620) took 3232 ms (client timeout 3028 ms). Trace:
W20260709 13:14:04.744179  8134 rpcz_store.cc:269] 0709 13:14:01.510736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.134422 (+623686us) service_pool.cc:224] Handling call
0709 13:14:04.743094 (+2608672us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001a9d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:04.743399 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2397139712}
I20260709 13:14:04.745625  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:04.748651  8138 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: []
W20260709 13:14:04.747437  8135 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 480) took 2611 ms (client timeout 1736 ms). Trace:
W20260709 13:14:04.754211  8135 rpcz_store.cc:269] 0709 13:14:02.135554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.187612 (+ 52058us) service_pool.cc:224] Handling call
0709 13:14:04.746981 (+2559369us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54001ab154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:04.747212 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354320640}
I20260709 13:14:04.755722  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:04.757711  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:04.758926  8135 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:04.759934  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:04.760267  8137 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1657) took 1888 ms (client timeout 1940 ms). Trace:
W20260709 13:14:04.765120  8137 rpcz_store.cc:269] 0709 13:14:02.871970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.909447 (+ 37477us) service_pool.cc:224] Handling call
0709 13:14:04.759836 (+1850389us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b54004092d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:04.760086 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":802364160}
I20260709 13:14:04.766649  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 517) took 1731 ms. Trace:
I20260709 13:14:04.766965  8137 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:04.767055  8204 rpcz_store.cc:276] 0709 13:14:03.034674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.034777 (+   103us) service_pool.cc:224] Handling call
0709 13:14:03.035008 (+   231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.371508 (+1336500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.766090 (+394582us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540018a454. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:04.766446 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3072056064}
I20260709 13:14:04.767629  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1238) took 1861 ms. Trace:
I20260709 13:14:04.768299 11493 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.768913  8132 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:04.769158 11493 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:04.761066  8134 raft_consensus.cc:2410] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:14:04.773360  8207 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2691) took 1590 ms (client timeout 2003 ms). Trace:
W20260709 13:14:04.773874  8207 rpcz_store.cc:269] 0709 13:14:03.182320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.316667 (+1134347us) service_pool.cc:224] Handling call
0709 13:14:04.772840 (+456173us) spinlock_profiling.cc:243] Waited 83 ms on lock 0x7b5400189054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:04.773088 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190922624}
I20260709 13:14:04.776008  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 520) took 1739 ms. Trace:
I20260709 13:14:04.781641  8208 rpcz_store.cc:276] 0709 13:14:03.036248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.036353 (+   105us) service_pool.cc:224] Handling call
0709 13:14:03.036591 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:03.283747 (+247156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:03.283782 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:03.283825 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:03.283862 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.775499 (+1491637us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54002c9554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:04.775581 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.775824 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567968512}
I20260709 13:14:04.767977  8137 rpcz_store.cc:276] 0709 13:14:02.906575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.766114 (+1859539us) service_pool.cc:224] Handling call
0709 13:14:04.766651 (+   537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.767384 (+   733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.767550 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.777375  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 516) took 1743 ms. Trace:
I20260709 13:14:04.784577  8210 rpcz_store.cc:276] 0709 13:14:03.033924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.034048 (+   124us) service_pool.cc:224] Handling call
0709 13:14:03.034284 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:03.165753 (+131469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:03.165800 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:03.165857 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:03.165904 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.776801 (+1610897us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54002be654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:04.776876 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.777150 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301825920}
W20260709 13:14:04.778666  8211 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55942 (request call id 521) took 1741 ms (client timeout 1807 ms). Trace:
W20260709 13:14:04.786918  8211 rpcz_store.cc:269] 0709 13:14:03.036935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.037049 (+   114us) service_pool.cc:224] Handling call
0709 13:14:04.778131 (+1741082us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54002413d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:04.778429 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551804288}
I20260709 13:14:04.788060 11465 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:04.779524  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2677) took 1738 ms. Trace:
I20260709 13:14:04.788777  8205 rpcz_store.cc:276] 0709 13:14:03.040594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.099277 (+ 58683us) service_pool.cc:224] Handling call
0709 13:14:04.779064 (+1679787us) spinlock_profiling.cc:243] Waited 77.4 ms on lock 0x7b54000ba4d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:04.779321 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178010496}
I20260709 13:14:04.792922  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:04.794404  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:04.795149  8054 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:04.795315  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1229) took 2160 ms. Trace:
I20260709 13:14:04.795881  8138 rpcz_store.cc:276] 0709 13:14:02.635277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.432298 (+1797021us) service_pool.cc:224] Handling call
0709 13:14:04.748019 (+315721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.749191 (+  1172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.795028 (+ 45837us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54001a9d50. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:04.795242 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725229440}
I20260709 13:14:04.796942  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:04.799256  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:04.802525  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:04.734086  7959 catalog_manager.cc:5697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:04.810940  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:04.810022  7959 catalog_manager.cc:5697] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:04.806538  8129 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:14:04.807294  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:04.725857  8062 rpcz_store.cc:276] 0709 13:14:02.044943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.844916 (+799973us) service_pool.cc:224] Handling call
0709 13:14:02.845285 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:02.967367 (+122082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:02.967395 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:02.967442 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:02.967485 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.721528 (+1754043us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400219b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:04.721594 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.721858 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280340224}
I20260709 13:14:04.824151  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1216) took 3436 ms. Trace:
I20260709 13:14:04.762890  8135 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:14:04.824494  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:04.727802  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1392) took 3060 ms. Trace:
W20260709 13:14:04.726634  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 402419us
I20260709 13:14:04.831746  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1234) took 2171 ms. Trace:
I20260709 13:14:04.832135  8135 rpcz_store.cc:276] 0709 13:14:02.656664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.761892 (+2105228us) service_pool.cc:224] Handling call
0709 13:14:04.762443 (+   551us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:04.828279 (+ 65836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.828499 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.826258  8129 rpcz_store.cc:276] 0709 13:14:01.387973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.388117 (+   144us) service_pool.cc:224] Handling call
0709 13:14:01.388427 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.813192 (+3424765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.814707 (+  1515us) spinlock_profiling.cc:243] Waited 3.32 s on lock 0x7b5400174654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:04.824060 (+  9353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7640755584}
I20260709 13:14:04.827253  8060 rpcz_store.cc:276] 0709 13:14:01.666880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.343415 (+676535us) service_pool.cc:224] Handling call
0709 13:14:02.343721 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:02.344042 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:02.344070 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:02.344471 (+   401us) log.cc:844] Serialized 26 byte log entry
0709 13:14:02.591242 (+246771us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:02.591279 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.592872 (+  1593us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:04.435760 (+1842888us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:04.727098 (+291338us) raft_consensus.cc:1685] finished
0709 13:14:04.727135 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.727540 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":18512,"spinlock_wait_cycles":8789760,"thread_start_us":246470,"threads_started":1,"wal-append.queue_time_us":296528}
I20260709 13:14:04.844163 11306 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:04.846050  7959 catalog_manager.cc:5697] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:04.844192  8060 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:04.848905  7959 catalog_manager.cc:5697] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 2 to 4, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 4 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:04.852947  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1386) took 3952 ms. Trace:
I20260709 13:14:04.852377 11005 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:04.853436  8059 rpcz_store.cc:276] 0709 13:14:00.900713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.900866 (+   153us) service_pool.cc:224] Handling call
0709 13:14:00.901189 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:01.278280 (+377091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.852316 (+3574036us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b5400052654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:04.852682 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865178880}
W20260709 13:14:04.854712  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 425495us
W20260709 13:14:04.853227  8102 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.030s	sys 0.030s
W20260709 13:14:04.858853  8102 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.030s	sys 0.030s
W20260709 13:14:04.859082  8026 leader_election.cc:336] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.595s (SENT)
W20260709 13:14:04.859375  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 541232us
I20260709 13:14:04.860638  8026 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:04.862319  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.866909  8206 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:14:04.867951  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2668) took 5361 ms. Trace:
I20260709 13:14:04.874754  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:04.875478  8724 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:04.814558 11410 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:04.879206  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 153849us
I20260709 13:14:04.884035  8131 raft_consensus.cc:2410] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:14:04.888613  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2695) took 1482 ms. Trace:
I20260709 13:14:04.889137  8203 rpcz_store.cc:276] 0709 13:14:03.354525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.514003 (+1159478us) service_pool.cc:224] Handling call
0709 13:14:04.514253 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.518902 (+  4649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.834146 (+315244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:04.870820  8209 rpcz_store.cc:276] 0709 13:13:59.506699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.115007 (+608308us) service_pool.cc:224] Handling call
0709 13:14:00.115295 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:00.140981 (+ 25686us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:14:00.141023 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:00.142426 (+  1403us) log.cc:844] Serialized 26 byte log entry
0709 13:14:00.460601 (+318175us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:00.460638 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:00.460700 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:03.183416 (+2722716us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:04.549159 (+1365743us) spinlock_profiling.cc:243] Waited 7.75 ms on lock 0x7b240023b538. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7daf9c4 00007f6db7dadacb 00007f6db7f68dc9 00007f6db7f68a55 00007f6db7f67fb0 00007f6dbabde488 00007f6dbabdebe8 00007f6dbabb6398 00007f6dbabb176a 00007f6dbabb0bb6
0709 13:14:04.824869 (+275710us) raft_consensus.cc:1685] finished
0709 13:14:04.828408 (+  3539us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:04.867726 (+ 39318us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25268,"raft.queue_time_us":25933,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":5092,"spinlock_wait_cycles":60443264,"thread_start_us":310856,"threads_started":1,"wal-append.queue_time_us":403929}
I20260709 13:14:04.898483  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.900520  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.902074  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.904021  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:04.905913  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.906991  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:04.909700  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 518) took 1874 ms. Trace:
I20260709 13:14:04.910157  8206 rpcz_store.cc:276] 0709 13:14:03.034938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:03.035175 (+   237us) service_pool.cc:224] Handling call
0709 13:14:03.035446 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:04.867343 (+1831897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.909333 (+ 41990us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54002beb54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:04.909631 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3851156352}
W20260709 13:14:04.911819  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 51472.5us
I20260709 13:14:04.912163 11659 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:04.912750 11659 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:04.913203 11659 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:04.915377  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:04.916208  8060 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:04.912999  8026 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:04.921301  7961 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34134 (request call id 264) took 19439 ms (client timeout 15000 ms). Trace:
W20260709 13:14:04.921746  7961 rpcz_store.cc:269] 0709 13:13:45.677623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:45.723375 (+ 45752us) service_pool.cc:224] Handling call
0709 13:14:04.920984 (+19197609us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:56.443594 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:56.444317 (+   723us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:56.444367 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:13:56.444387 (+    20us) tablet.cc:662] Decoding operations
0709 13:13:56.447564 (+  3177us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:56.447642 (+    78us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:56.447672 (+    30us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:13:56.722323 (+274651us) tablet.cc:701] Row locks acquired
0709 13:13:56.754327 (+ 32004us) write_op.cc:260] PREPARE: finished
0709 13:13:56.790432 (+ 36105us) write_op.cc:270] Start()
0709 13:13:56.790608 (+   176us) write_op.cc:276] Timestamp: P: 1783602836783347 usec, L: 0
0709 13:13:56.790634 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:13:56.791378 (+   744us) log.cc:844] Serialized 7820 byte log entry
0709 13:13:57.631063 (+839685us) op_driver.cc:464] REPLICATION: finished
0709 13:13:57.885533 (+254470us) write_op.cc:301] APPLY: starting
0709 13:13:57.885650 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:57.886175 (+   525us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:57.886314 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:57.889558 (+  3244us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:57.890025 (+   467us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=20
0709 13:13:57.890089 (+    64us) write_op.cc:312] APPLY: finished
0709 13:13:57.894817 (+  4728us) log.cc:844] Serialized 139 byte log entry
0709 13:13:57.895044 (+   227us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:57.895339 (+   295us) write_op.cc:454] Released schema lock
0709 13:13:57.895413 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:57.895815 (+   402us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:57.895852 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":598336,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":254149,"apply.run_cpu_time_us":5577,"apply.run_wall_time_us":10266,"num_ops":20,"prepare.queue_time_us":106801,"prepare.run_cpu_time_us":24388,"prepare.run_wall_time_us":485135,"raft.queue_time_us":141136,"raft.run_cpu_time_us":22021,"raft.run_wall_time_us":281235,"replication_time_us":840265,"thread_start_us":331554,"threads_started":4,"wal-append.queue_time_us":256386,"wal-append.run_cpu_time_us":38817,"wal-append.run_wall_time_us":1844304}]]}
I20260709 13:14:04.925252 11493 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:04.928021  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:04.930306  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 44676.7us
I20260709 13:14:04.911906  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:04.939342  8728 raft_consensus.cc:3055] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:04.939743  8728 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:04.939966  8696 leader_election.cc:336] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Timed out: RequestConsensusVote RPC to 127.6.198.67:35745 timed out after 1.736s (SENT)
I20260709 13:14:04.940943  8728 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:04.942615  8029 connection.cc:256] RPC call timeout handler was delayed by -1.84939955200025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18000000000000002s for a response
W20260709 13:14:04.943084  8029 connection.cc:256] RPC call timeout handler was delayed by -1.7233995520000462s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.194s for a response
I20260709 13:14:04.943207 11005 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:04.943678  8029 connection.cc:256] RPC call timeout handler was delayed by -1.4456032660000346s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
W20260709 13:14:04.944061  8029 connection.cc:256] RPC call timeout handler was delayed by -1.3934032659999502s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
W20260709 13:14:04.944378  8029 connection.cc:256] RPC call timeout handler was delayed by -1.3853032659999371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1595s for a response
W20260709 13:14:04.944672  8029 connection.cc:256] RPC call timeout handler was delayed by -1.2971032660000219s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1693s for a response
W20260709 13:14:04.944967  8029 connection.cc:256] RPC call timeout handler was delayed by -1.2575767610001094s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
W20260709 13:14:04.945307  8029 connection.cc:256] RPC call timeout handler was delayed by -1.2107032659998822s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
W20260709 13:14:04.945595  8029 connection.cc:256] RPC call timeout handler was delayed by -1.1765767609999784s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1613s for a response
W20260709 13:14:04.945878  8029 connection.cc:256] RPC call timeout handler was delayed by -1.1468767610001578s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
W20260709 13:14:04.946897  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 87942.7us
I20260709 13:14:04.942301  8728 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:04.946146  8029 connection.cc:256] RPC call timeout handler was delayed by -1.0712767610000355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17300000000000001s for a response
W20260709 13:14:04.959578  8029 connection.cc:256] RPC call timeout handler was delayed by -1.0028767610001523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1806s for a response
W20260709 13:14:04.964601  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 51878.7us
I20260709 13:14:04.953852 11410 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:04.983027  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:04.983372  8061 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:04.984673  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:04.984813  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:04.985376  8061 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:04.986020  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:04.986646  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:04.987390  8209 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:04.987406  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.007490  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.008160  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:04.999915  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 1.7782e+06us
I20260709 13:14:05.010563  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:05.011850  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1241) took 2100 ms. Trace:
I20260709 13:14:05.012161 11420 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.013078 11420 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:05.012334  8134 rpcz_store.cc:276] 0709 13:14:02.910037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.770404 (+1860367us) service_pool.cc:224] Handling call
0709 13:14:04.770866 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:04.771373 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:04.824846 (+ 53473us) log.cc:844] Serialized 14 byte log entry
0709 13:14:05.009806 (+184960us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:05.009862 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:05.009912 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.009986 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:05.010110 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4036,"prepare.queue_time_us":265,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"thread_start_us":156035,"threads_started":1}
I20260709 13:14:05.016512 11410 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:05.007284  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:05.017948  8207 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:05.018640 11410 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:05.020423  8176 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:05.006956  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:05.023046  8210 raft_consensus.cc:2393] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 8: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:05.006575  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:05.024376  8211 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:14:05.012005  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.034741  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:05.035477  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.036003  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:05.036592  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.037742  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.043838 10940 consensus_queue.cc:1059] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:14:05.045428  8057 consensus_meta.cc:287] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.608s	user 0.005s	sys 0.012s
I20260709 13:14:05.045873  8057 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
W20260709 13:14:05.046938  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.048637  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.048967  8203 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:05.050185  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:05.053295  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 46744.5us
I20260709 13:14:05.007788 11533 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:05.055060  8027 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:05.057536  8211 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
W20260709 13:14:05.012058  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.062969  8207 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:05.051869  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.046968  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.007251  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:05.067977  8724 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:05.074546 11306 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:05.085893  8728 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:14:05.086416  8728 pending_rounds.cc:85] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Aborting all ops after (but not including) 9
I20260709 13:14:05.094774  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:05.104436  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.170157  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:05.171216  8134 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
W20260709 13:14:05.174897  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 154575us
I20260709 13:14:05.179358  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1233) took 2525 ms. Trace:
I20260709 13:14:05.178023 11610 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:05.059978 11659 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:05.239274  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:05.226835  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 186952us
I20260709 13:14:05.179901  8133 rpcz_store.cc:276] 0709 13:14:02.653656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.442543 (+1788887us) service_pool.cc:224] Handling call
0709 13:14:04.750611 (+308068us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:04.751523 (+   912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:04.751560 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:04.752413 (+   853us) log.cc:844] Serialized 26 byte log entry
0709 13:14:04.894823 (+142410us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b4800002dd8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:14:04.919102 (+ 24279us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:04.947500 (+ 28398us) log.cc:844] Serialized 14 byte log entry
0709 13:14:04.949030 (+  1530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:04.959712 (+ 10682us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:05.040392 (+ 80680us) raft_consensus.cc:1685] finished
0709 13:14:05.040495 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:05.178989 (+138494us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b54001ab150. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:05.179124 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28848,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":2539,"raft.run_cpu_time_us":4762,"raft.run_wall_time_us":141474,"spinlock_wait_cycles":781149824,"thread_start_us":166139,"threads_started":1,"wal-append.queue_time_us":206778}
I20260709 13:14:05.195649  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:05.119083  8728 pending_rounds.cc:107] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:14:05.236577  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1240) took 2327 ms. Trace:
I20260709 13:14:05.190582  8058 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:14:05.298976  8728 raft_consensus.cc:2889] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:05.300925  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 244853us
I20260709 13:14:05.301558  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:05.176772 11660 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:05.305585  8206 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:14:05.306028  8054 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:05.307487  8209 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:05.307833  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.236356 10940 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:05.305635 11660 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:05.197263  8208 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:14:05.311988  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:05.314270  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 136989us
I20260709 13:14:05.315032  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:05.182714 11420 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:05.315138  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:05.321152  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:05.314684  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.297011  8132 rpcz_store.cc:276] 0709 13:14:02.909347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.769718 (+1860371us) service_pool.cc:224] Handling call
0709 13:14:04.842676 (+ 72958us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:04.866786 (+ 24110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:04.866824 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:04.867490 (+   666us) log.cc:844] Serialized 52 byte log entry
0709 13:14:04.976416 (+108926us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:05.069643 (+ 93227us) log.cc:844] Serialized 14 byte log entry
0709 13:14:05.083740 (+ 14097us) log.cc:844] Serialized 14 byte log entry
0709 13:14:05.172608 (+ 88868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.172693 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:05.236281 (+ 63588us) raft_consensus.cc:1685] finished
0709 13:14:05.236313 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:05.236493 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111895,"prepare.queue_time_us":103257,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":27965,"raft.queue_time_us":220167,"raft.run_cpu_time_us":395,"raft.run_wall_time_us":396,"thread_start_us":108420,"threads_started":1,"wal-append.queue_time_us":324313}
I20260709 13:14:05.320827  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:05.327683  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.329278  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.330834  8061 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:05.327525  8059 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
W20260709 13:14:05.331672  8026 leader_election.cc:336] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.940s (SENT)
I20260709 13:14:05.331908  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:05.332211  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.332952  8059 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:14:05.333814  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:05.333901  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:05.313529 10940 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:05.337637  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:05.337977  8058 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:05.331681  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 149076us
I20260709 13:14:05.334810  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.331697  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:05.335500  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:05.337016  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.343024  8056 raft_consensus.cc:2393] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 8: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:05.331763  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.344714  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.351922  8061 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:05.353799  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.340564 10940 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:05.357203  8129 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:05.358918  8062 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:05.362277  8134 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:14:05.363077  8131 raft_consensus.cc:2393] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:05.366567  8027 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:05.366956  8054 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:14:05.367353  8054 pending_rounds.cc:85] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 3
I20260709 13:14:05.367610  8054 pending_rounds.cc:107] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:14:05.367857  8054 raft_consensus.cc:2889] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:05.369300 11576 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.369884 11576 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:05.370640  8063 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:05.371340  8130 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:05.372316 11576 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:05.377203  8054 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:14:05.378449  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:05.376588  8133 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:14:05.386798  8727 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:05.387053  8724 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:05.387951  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 353633us
I20260709 13:14:05.402880  8059 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:05.394537  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:05.502866  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:05.504117  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:05.526485  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:05.397862  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:05.415056 11677 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:05.419050 11659 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:05.549980  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:05.551930  8058 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:05.379882 11420 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.152s
W20260709 13:14:05.554276  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 156508us
I20260709 13:14:05.558486  8724 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:05.559072 11610 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:05.579634  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:05.579839 11610 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:05.580519 10940 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:05.587154  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:05.587986  8722 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
W20260709 13:14:05.593601  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 43724.4us
I20260709 13:14:05.558507 11677 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:14:05.600761  8104 connection.cc:256] RPC call timeout handler was delayed by -1.9442768569999771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1915s for a response
I20260709 13:14:05.605963  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:05.606695  8133 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:05.559013 11659 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
W20260709 13:14:05.609076  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 203907us
I20260709 13:14:05.610751 10940 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:05.407018 11660 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:05.631595  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:05.632215  8138 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:05.607554 11493 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:14:05.637282  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:05.580987 11610 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:05.642107  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 306901us
I20260709 13:14:05.649495  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:05.650513  8138 raft_consensus.cc:2410] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:14:05.651152 11602 consensus_queue.cc:1059] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:14:05.652217 11602 raft_consensus.cc:3055] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:05.652618 11602 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:05.653875 11005 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:05.657612 11677 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:05.664707  8133 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:05.645254 11410 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:05.658521 11660 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:05.673882  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.685000  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.653716 11602 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:05.688241  8056 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:14:05.689013  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:05.689720  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:05.694684  8055 raft_consensus.cc:2393] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:05.691295 11665 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:14:05.691557 11621 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:14:05.696398 11665 raft_consensus.cc:3055] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:05.696802 11665 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:05.698501 11306 consensus_queue.cc:1059] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:05.665887 11420 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:05.691574  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:05.671329 11659 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:14:05.691972  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 75419.6us
W20260709 13:14:05.708483  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 110217us
I20260709 13:14:05.673792 11410 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:14:05.707278  8026 leader_election.cc:336] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.693s (SENT)
W20260709 13:14:05.706475  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 90177.5us
I20260709 13:14:05.714406  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:05.712956  8104 connection.cc:256] RPC call timeout handler was delayed by -1.8650768570000764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
W20260709 13:14:05.715276  8104 connection.cc:256] RPC call timeout handler was delayed by -1.7642768570001408s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21150000000000002s for a response
I20260709 13:14:05.716089  8723 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:05.717839  8130 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:05.717837  8104 connection.cc:256] RPC call timeout handler was delayed by -1.7178348800000549s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3104s for a response
W20260709 13:14:05.718606  8104 connection.cc:256] RPC call timeout handler was delayed by -1.7035995620001358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1645s for a response
W20260709 13:14:05.718891  8104 connection.cc:256] RPC call timeout handler was delayed by -1.6531995620000544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17010000000000003s for a response
W20260709 13:14:05.719911  8104 connection.cc:256] RPC call timeout handler was delayed by -1.3813995620000696s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
I20260709 13:14:05.697794 11665 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:05.708451 11659 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:05.720275  8104 connection.cc:256] RPC call timeout handler was delayed by -1.3705995620000522s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2015s for a response
I20260709 13:14:05.724852  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.711680 11420 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:05.728348  8102 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:05.729130 11576 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:05.729451 11420 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:05.728355  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.714274 11602 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:05.726523  8104 connection.cc:256] RPC call timeout handler was delayed by -1.2733995620001224s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21230000000000004s for a response
W20260709 13:14:05.731889  8104 connection.cc:256] RPC call timeout handler was delayed by -1.0308049840000422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18300000000000002s for a response
W20260709 13:14:05.734566  8026 leader_election.cc:336] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.789s (SENT)
I20260709 13:14:05.722720  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:05.738617  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 108080us
I20260709 13:14:05.742386 11665 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:05.756946  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.730225 11465 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:14:05.760865  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 51697.9us
I20260709 13:14:05.759847  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:05.733428 11659 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:05.765841 11493 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:05.766424 11493 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:05.766871 11493 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:05.766945  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:05.767717  8129 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:05.770630  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:05.760480 11465 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:05.771595 11465 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:05.772714  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:05.773255  8132 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:05.778808  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:05.807560  8723 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
W20260709 13:14:05.809082  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:05.809911  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 92591us
I20260709 13:14:05.822440  8027 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:05.827255 11533 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:05.786945 11410 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:05.838596 11576 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:14:05.875145  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 104623us
I20260709 13:14:05.881129  8029 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:05.882313 11684 raft_consensus.cc:2764] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:14:05.887709 11665 consensus_queue.cc:1059] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:05.888826 11420 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:05.890377 11665 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:05.898545  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:05.901849  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 141368us
I20260709 13:14:05.865381 11683 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:05.861146 11533 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:05.892786 11410 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:05.878746 11576 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:05.861649 11465 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:05.861933 11610 consensus_queue.cc:1059] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:14:05.912253 11533 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:05.948132  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 59632.2us
I20260709 13:14:05.931299  8130 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:05.914000 11683 raft_consensus.cc:697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:05.917132 11693 raft_consensus.cc:3055] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:14:05.943140 11700 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:14:05.943507 11703 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
W20260709 13:14:05.944766 11698 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:14:05.946460  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:05.947213 11704 log.cc:927] Time spent T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:14:05.870275 11685 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:14:05.958928  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.961952  8061 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:14:05.963307  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:05.963838  8104 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:05.964274 11683 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:05.961267 11693 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:05.966894  8055 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:05.931645 11420 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:14:05.971979  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 78387.3us
I20260709 13:14:05.983786  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:05.971442 11420 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:05.988878  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.988844 11707 raft_consensus.cc:2764] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
W20260709 13:14:05.990557 11705 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:14:05.976703 11621 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:14:05.992955  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:05.994238  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:05.995280  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:05.996407  8132 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:14:05.990890  8134 raft_consensus.cc:2376] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8. Re-sending same reply.
I20260709 13:14:05.999586  8208 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:06.000229  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:06.004700  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:06.005425  8206 raft_consensus.cc:2393] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:06.007354 11493 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.009502 11493 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:06.003360  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:06.014873  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:06.014935  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:06.014328  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:06.014874  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:06.029973  8210 raft_consensus.cc:2410] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:06.030921  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:06.029615  8205 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:06.040609 11465 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:05.979189 11576 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:06.042536  8207 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:14:06.043607  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:06.044260  8208 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:06.046432  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:06.047009  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58856.1us
I20260709 13:14:06.029053 11711 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:06.047825  8696 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:06.047988 11711 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.048669  8696 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:06.049592  8696 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:06.052814  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.053504  8129 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:06.054813  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.051821  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:06.057605 11713 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:14:06.057933  8209 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:06.058771 11420 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.059629 11649 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:06.059940  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:06.060369 11420 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:06.060858  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:06.059281  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:06.060046 11649 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:06.058045  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:06.064327  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 94004.3us
I20260709 13:14:06.080248  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:06.080883  8207 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:14:06.081655  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:06.087141  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:06.088160  8210 raft_consensus.cc:2410] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:14:06.090914  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:06.092078 11610 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:06.092586 11610 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:06.092970 11610 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:06.095748  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:06.110122  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:06.118216  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:06.123739  8208 raft_consensus.cc:2410] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:06.124822  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:06.126565  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 243686us
I20260709 13:14:06.125566  8205 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:06.002969 11693 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.128245 11701 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:06.128713 11701 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:06.129050 11701 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:06.129133 11693 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:06.129613  8029 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:06.130980  8029 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:06.132156  8210 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:06.126147 11711 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:06.151607  8026 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:06.151778  8029 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:06.151925 11715 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:06.152525 11717 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:06.152807 11493 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:06.148131  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 98601us
I20260709 13:14:06.154911  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:06.157140  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:06.157825  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:06.158113 11685 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:06.158524 11719 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:06.149669 11621 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:06.158547  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:06.150760  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:06.170735  8055 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:06.158846  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:06.169922  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 60494.4us
I20260709 13:14:06.163174 11717 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:06.208779  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:06.216290  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.179986  8061 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:06.214867  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 63170.3us
I20260709 13:14:06.215430  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:06.234102  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:06.180403 11685 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:06.219141  8062 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:06.171296 11493 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.238945 11711 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:14:06.251976  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 118226us
I20260709 13:14:06.223704  8056 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:06.253330  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:06.216856 11677 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:06.259325  8059 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:06.218262 11717 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:06.252727 11715 raft_consensus.cc:697] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:06.268873  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:06.274466 11665 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:06.274621 11722 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:06.274945 11665 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:06.275172 11722 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:06.275442 11665 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:06.275630 11722 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:06.268898  8026 leader_election.cc:336] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.859s (SENT)
I20260709 13:14:06.278096  8026 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:06.278581  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:06.279129  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.279211  8059 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:14:06.279866 11677 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:06.280248  8026 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:06.282469 11465 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.281687  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.281008  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.282789  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.172180 11649 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:06.259797 11711 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:06.283473  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.284222  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.282473  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.297051  8132 raft_consensus.cc:2393] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:06.282694 11708 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.278491  8029 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:06.298029  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 46348.3us
I20260709 13:14:06.292361  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:06.285776  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:06.294904  8696 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:06.300177  8722 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:14:06.301416  8131 raft_consensus.cc:2368] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10.
I20260709 13:14:06.303993  8056 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:06.305042  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.305596  8130 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:06.306787  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1405) took 1325 ms. Trace:
I20260709 13:14:06.307133  8060 rpcz_store.cc:276] 0709 13:14:04.981136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.000285 (+ 19149us) service_pool.cc:224] Handling call
0709 13:14:05.021112 (+ 20827us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:05.605678 (+584566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:05.605721 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:05.631481 (+ 25760us) log.cc:844] Serialized 26 byte log entry
0709 13:14:05.757016 (+125535us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:05.757048 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.306426 (+549378us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b54001e87d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:06.306472 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:06.306516 (+    44us) raft_consensus.cc:1685] finished
0709 13:14:06.306529 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:06.306732 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8935,"raft.queue_time_us":200280,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":20098,"spinlock_wait_cycles":91367680,"thread_start_us":553293,"threads_started":2,"wal-append.queue_time_us":236204}
W20260709 13:14:06.309253 11659 consensus_meta.cc:287] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.504s	user 0.000s	sys 0.007s
I20260709 13:14:06.309639 11659 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:06.310155 11715 consensus_queue.cc:237] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:06.312853  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:06.314136  8059 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:06.314579  8057 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:06.319801  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:06.320123  8029 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:06.301033 11711 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:06.322718  8061 raft_consensus.cc:2410] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:14:06.324298  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.325064  8132 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:06.326537  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:06.328210  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 54774.7us
I20260709 13:14:06.319041 11410 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:06.318297 11621 raft_consensus.cc:697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:06.326057 11683 raft_consensus.cc:2764] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:14:06.327957 11677 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:06.341866  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.332788  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.336402 11709 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:06.315656 11708 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.324632 11719 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:06.337785  8055 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:06.301822 11465 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.338562 11718 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:06.330047  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:06.340233 11493 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:06.319259 11659 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:06.344821  8131 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:14:06.352384  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 69541.3us
W20260709 13:14:06.348692  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 74556.3us
I20260709 13:14:06.347286 11719 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:06.353216 11732 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:06.357352  8101 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:06.349761 11660 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:06.360318 11686 log.cc:927] Time spent T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.669s	user 0.004s	sys 0.000s
I20260709 13:14:06.358481 11739 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:06.353698 11602 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:06.381245 11719 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:06.386052 11699 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:06.386616 11602 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:06.387041 11602 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:06.385324 11730 raft_consensus.cc:2955] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:06.387703 11621 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:06.379554  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:06.344538 11709 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:06.309654 11701 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:06.388984  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:06.391942  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:06.380934 11732 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:06.407038  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:06.406945 11659 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:06.411775  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:06.418145  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 75667.8us
I20260709 13:14:06.411222 11732 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:06.423346 11735 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:06.423959 11735 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:06.381395 11685 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.429805 11712 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:06.424804 11735 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:06.430485  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:06.431586  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:06.432345 11731 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:06.432714 11731 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:06.433058 11731 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:06.428129 11722 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.434702 11736 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:06.435163  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.435787 11722 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:06.435947  8130 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:06.436414 11722 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:06.436878 11722 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:06.437597 11722 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:06.441172  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.421048 11702 raft_consensus.cc:2764] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:14:06.451614 11737 log.cc:927] Time spent T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:14:06.452232 11720 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:06.455322  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 43615.6us
W20260709 13:14:06.456655  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 75496.9us
I20260709 13:14:06.416553 11725 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.132s
W20260709 13:14:06.464941  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 47689.7us
I20260709 13:14:06.474895  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:06.475996  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:06.456128 11465 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:06.486054  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.487672  8132 raft_consensus.cc:2393] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:06.492542 11682 raft_consensus.cc:3055] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:06.492877 11701 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:06.492942 11682 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:06.493974 11682 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.495534 11682 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:06.496522  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:06.496852  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.497062  8134 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:06.497990  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.498584  8131 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:06.446327 11533 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.501334  8063 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:06.502302 11533 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:06.503023  8135 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:06.504264  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:06.504820  8132 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:06.505589  8211 raft_consensus.cc:2410] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:06.506735  8206 raft_consensus.cc:2410] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:14:06.510280  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 546496us
I20260709 13:14:06.517163  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:06.517438 11602 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.500195 11659 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:06.518481 11707 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:06.518903 11707 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:06.519520 11707 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:06.520958 11711 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:06.522912 11713 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:14:06.523875 11602 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:06.525095 11711 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:06.528829 11610 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:06.529217 11707 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.514628  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:06.531307  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:06.532717 11659 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:06.534955 11679 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:06.535914 11649 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:06.537201  8175 connection.cc:256] RPC call timeout handler was delayed by -1.4897362809999777s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2625s for a response
I20260709 13:14:06.532469  8029 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:06.549474  8104 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:06.551573  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:06.547796  8063 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:06.554663  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:06.605985  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.444177 11709 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.551743 11683 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:06.548789 11659 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:06.613677  8205 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:06.512769 11708 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:06.618975  8212 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:14:06.628162  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 78751.6us
W20260709 13:14:06.638110  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 173341us
W20260709 13:14:06.639907  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 88440.9us
I20260709 13:14:06.617774 11725 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:06.546308 11665 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.616935 11420 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:06.646631  8134 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
W20260709 13:14:06.649994  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.651327  8104 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:06.654448  8026 leader_election.cc:336] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.730s (SENT)
I20260709 13:14:06.623029 11683 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:06.655017  8026 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:06.679804  8026 leader_election.cc:336] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.744s (SENT)
I20260709 13:14:06.680079  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:06.680310  8026 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:06.681352  8026 leader_election.cc:336] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.795s (SENT)
I20260709 13:14:06.681600  8029 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:06.682575  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:06.683703  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:06.684794  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:14:06.682327  8026 leader_election.cc:336] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.806s (SENT)
I20260709 13:14:06.687019  8026 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:06.688627 11410 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:06.689033 11410 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:06.689390 11410 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:06.689262  8101 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:06.694633 11689 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:06.695096 11689 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:06.695456 11689 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:06.698630 11714 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:06.699038 11714 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:06.699379 11714 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:06.703943 11748 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:14:06.707916 11677 raft_consensus.cc:2764] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:14:06.708568 11677 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:06.708953 11749 consensus_queue.cc:1059] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:06.710530 11703 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:06.710990 11703 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:06.711844 11703 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:06.713032 11703 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:06.647439 11725 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:06.663545 11533 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:06.617203 11717 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.685626 11684 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:06.766738  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 110334us
I20260709 13:14:06.659567 11683 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:06.659137 11610 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:06.783584  8724 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:06.785073  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:06.785609  8723 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:06.652637 11708 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 13:14:06.694674 11690 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:06.768175 11725 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:06.797003  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 149186us
I20260709 13:14:06.710141 11749 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:14:06.805585 11752 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:06.807410  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:06.806936  8063 raft_consensus.cc:2955] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:06.808733  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:06.663900 11420 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.816280 11720 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:06.662524 11732 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:06.798197 11690 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:06.784582 11750 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:06.856130 11747 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.856871 11747 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:06.778064 11715 consensus_queue.cc:1059] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:14:06.802000 11707 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:06.858233 11715 raft_consensus.cc:3055] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:06.906217  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:06.919416  8722 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: []
W20260709 13:14:06.920279  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 237954us
I20260709 13:14:06.859804 11690 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:06.928154  8723 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
W20260709 13:14:06.930821  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 160568us
I20260709 13:14:06.907517 11750 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:14:06.924062 11730 consensus_meta.cc:287] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.534s	user 0.010s	sys 0.000s
I20260709 13:14:06.907832 11715 raft_consensus.cc:740] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:06.905380 11720 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:06.943140  8101 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:06.951001 11763 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:06.926412 11410 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:06.954316 11410 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:06.955108 11717 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:06.955857  8101 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:06.956859 11763 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:14:06.959384  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 128058us
I20260709 13:14:06.961197  8102 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:06.966511  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 428827us
W20260709 13:14:06.968505  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 159875us
I20260709 13:14:06.973203  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:06.930851 11739 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:06.939332 11750 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:06.989666 11719 consensus_meta.cc:287] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.603s	user 0.009s	sys 0.000s
I20260709 13:14:06.974241 11707 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:14:07.005470  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 320723us
I20260709 13:14:07.006006  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:14:06.994107 11731 consensus_meta.cc:287] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.561s	user 0.010s	sys 0.000s
I20260709 13:14:06.992995 11750 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:06.970316 11763 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:14:07.005111 11747 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:07.042601 11420 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:07.044100 11770 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:07.044703 11770 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:07.045154 11770 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:07.046506  8104 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:07.055307 11755 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:14:07.057219 11465 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:07.057734 11465 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:07.058122 11465 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:07.007233 11719 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:07.010576 11715 consensus_queue.cc:260] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:07.091387 11714 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:07.092443 11724 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.854s	user 0.006s	sys 0.008s
I20260709 13:14:07.093809 11714 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:07.094700  8026 leader_election.cc:336] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.523s (SENT)
I20260709 13:14:07.097232  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:07.099293  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2717) took 1052 ms. Trace:
I20260709 13:14:07.099670  8206 rpcz_store.cc:276] 0709 13:14:06.047064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.059708 (+ 12644us) service_pool.cc:224] Handling call
0709 13:14:07.099094 (+1039386us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54001536d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:07.099237 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1025506048}
W20260709 13:14:07.111881  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.118592  8175 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.998s (SENT) [suppressed 61 similar messages]
I20260709 13:14:07.123059  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:07.123704  8136 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:07.029976 11665 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:07.136185  8026 leader_election.cc:336] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 2.134s (SENT)
I20260709 13:14:07.138376  8026 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:07.141673  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.146718  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 155995us
I20260709 13:14:07.012528 11731 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:07.157229  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.157817  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 130765us
W20260709 13:14:07.160653  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.163616  8130 raft_consensus.cc:3055] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:07.164181  8130 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:07.139578 11682 consensus_meta.cc:287] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.644s	user 0.001s	sys 0.009s
W20260709 13:14:07.182192  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.167760  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:07.180943  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:07.183274  8131 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:07.138697  8135 raft_consensus.cc:2955] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:07.138254 11717 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:07.154501 11754 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
W20260709 13:14:07.186178  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50034us
I20260709 13:14:07.187987  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:07.188107  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:07.191083  8132 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:07.165000  8130 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:07.191579  8026 leader_election.cc:336] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 2.171s (SENT)
W20260709 13:14:07.192322  8026 leader_election.cc:336] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 2.310s (SENT)
I20260709 13:14:07.192677  8029 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:07.194296  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:07.026995 11720 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:07.196024  8130 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:07.196373  8137 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:07.160204 11760 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:07.199894  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 172112us
I20260709 13:14:07.193209 11707 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:07.204344  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:07.159976 11465 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:07.206918 11776 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:07.207757 11689 consensus_meta.cc:287] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.000s
I20260709 13:14:07.207845 11465 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:07.208261 11689 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:07.214996  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:07.215544  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:07.149783 11683 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:07.233814  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:07.201025 11682 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:07.240902 11465 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:07.216908 11689 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:07.266896  8132 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
W20260709 13:14:07.267604  8175 leader_election.cc:336] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.693s (SENT)
I20260709 13:14:07.196249 11717 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:07.279078  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.270181 11729 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 1.030s	user 0.007s	sys 0.003s
I20260709 13:14:07.298043  8723 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:14:07.270658 11770 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:07.272883 11731 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:07.275156 11465 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:07.278581 11690 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:07.279073 11719 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:07.290386 11715 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:07.279713  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.324175  8056 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:07.324234  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 56666.7us
I20260709 13:14:07.280908 11683 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:07.283387 11703 consensus_meta.cc:287] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.570s	user 0.012s	sys 0.001s
I20260709 13:14:07.285935 11725 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:07.287540 11732 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:07.289216  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:07.289654  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 84361.9us
W20260709 13:14:07.293180  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 75762.7us
W20260709 13:14:07.302349  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 149038us
I20260709 13:14:07.306097  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:07.307359  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:07.309513  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:07.344419  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:07.354044  8209 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:07.314970  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:07.355218  8130 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:07.315730  8061 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:07.321205  8136 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:07.277906 11707 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:07.323096  8060 raft_consensus.cc:1240] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:07.299198 11746 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.369063  8101 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:07.371694 11770 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:07.373805  8131 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:07.342473 11577 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:07.351248 11679 consensus_queue.cc:1059] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:14:07.377337 11750 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:07.327484 11760 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:07.356479 11702 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:07.401531  8101 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:07.407557  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:07.409189  8693 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:07.367197 11719 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:14:07.408576 11722 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.413610 11722 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:07.414887 11781 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:14:07.408423 11778 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:07.415830 11722 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:07.417196 11679 raft_consensus.cc:3055] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:14:07.417457  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 52134us
I20260709 13:14:07.417596 11679 raft_consensus.cc:740] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:07.418547 11679 consensus_queue.cc:260] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:07.419885 11679 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:07.420017  8104 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:07.427853  8101 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:07.400040  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 67677.5us
I20260709 13:14:07.431806  8136 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:07.401253 11770 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:07.437934  8104 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:07.438725  8101 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:07.432811 11465 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:07.350993 11709 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:07.447912  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:07.445513  8130 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:07.450462  8208 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:07.379489 11746 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:07.429423 11778 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:07.455366  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.455899  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.458122  8209 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:07.468948 11714 raft_consensus.cc:3055] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:07.467595  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:07.446499 11005 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:07.463721  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:07.466352 11750 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:07.446354 11780 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:07.462812  8104 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:07.468912 11746 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:07.506750  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:07.507359  8057 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:07.466324  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:07.452775 11465 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:07.512053  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:07.454382 11763 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:07.469906 11725 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:07.451514 11752 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:07.438552 11770 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:07.495407  8101 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:07.398729 11577 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:07.504264  8056 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:14:07.539650  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:07.504359  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:07.505990 11780 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:07.454487  8204 raft_consensus.cc:2955] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:07.513188  8728 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:07.514914  8054 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: []
W20260709 13:14:07.515100  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 52407us
I20260709 13:14:07.542465  8209 raft_consensus.cc:2410] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:07.515287 11420 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:07.515686 11665 consensus_queue.cc:1059] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:07.494159 11714 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:07.516512 11747 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:07.547477  8212 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:07.547959  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:07.518823 11576 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:07.519845 11717 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:07.520514 11690 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:07.521436  8057 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:07.522055 11659 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.525821 11465 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:07.525925  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:07.427565 11760 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:07.527097  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:07.527575  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:07.527724  8205 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:07.528066  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:07.528687  8203 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:07.528965  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:07.529568  8059 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:07.481710  8208 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:07.527734  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:07.532449  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.521996  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:07.543581  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:07.543915  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 90380.3us
I20260709 13:14:07.520526 11702 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:07.534747 11005 raft_consensus.cc:697] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:07.620891  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 86726.4us
I20260709 13:14:07.621325  8062 raft_consensus.cc:2410] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:07.549718  8104 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:07.622124  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 87632.7us
I20260709 13:14:07.622640  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:07.571730  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:07.618666  8063 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:07.536062 11770 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:07.617770  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:07.630805  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:07.543210 11780 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:07.630020  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:07.633004  8056 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:07.633814  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:07.639638  8057 raft_consensus.cc:2376] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11. Re-sending same reply.
I20260709 13:14:07.620028 11659 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:07.641983  8063 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:07.642323 11659 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:07.643893  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:07.634359  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:07.645411  8059 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:07.634140 11717 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:07.549503 11747 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:07.646394 11747 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:07.647177 11715 consensus_queue.cc:1059] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:07.648195 11715 raft_consensus.cc:3055] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:07.648511 11715 raft_consensus.cc:740] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:07.649317 11715 consensus_queue.cc:260] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:07.644222  8026 leader_election.cc:336] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.990s (SENT)
I20260709 13:14:07.634490  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:07.650267  8026 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:07.644691  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:07.653127  8059 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:07.643983  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:07.549254 11707 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:07.654655  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:07.684965  8061 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
W20260709 13:14:07.683701  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 64505.3us
I20260709 13:14:07.543242 11420 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:07.682979  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:07.657991  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:07.696761  8209 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:07.657701 11747 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:07.623173 11759 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:14:07.699043  8203 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:07.556315 11752 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:07.709517  8026 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:07.693624  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:07.706526  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:07.710682  8059 raft_consensus.cc:2393] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:07.707329  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:07.711164  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 168440us
I20260709 13:14:07.711862  8104 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:07.631850 11746 raft_consensus.cc:3055] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:07.693025  8212 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:07.655776 11659 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:07.695663  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:07.691628 11715 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:07.715648  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:07.716866  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.720409  8061 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:07.699133 11420 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:07.724450  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:07.725831  8135 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:14:07.690439 11722 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.718977  8206 raft_consensus.cc:2368] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10.
W20260709 13:14:07.719376  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 76002us
I20260709 13:14:07.684891 11763 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:07.721796  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:07.713745 11746 raft_consensus.cc:740] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:07.728865  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:07.732450  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:07.732838  8135 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:07.685055 11621 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:14:07.710177 11719 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:07.729559  8062 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
W20260709 13:14:07.730739  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 47881.6us
I20260709 13:14:07.739853  8029 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:07.694802 11717 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:07.748550  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:07.712513 11707 raft_consensus.cc:697] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:07.737777  8209 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:14:07.738389  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:07.740979  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:07.732023  8054 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:07.741909  8130 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:07.654583 11714 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:07.729363  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 212088us
I20260709 13:14:07.829773  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:07.812548  8203 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: []
W20260709 13:14:07.830720  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 103399us
W20260709 13:14:07.758559  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.816006  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 85426.9us
I20260709 13:14:07.813776  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:07.741998  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:07.723956 11649 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 13:14:07.835475  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:07.727385 11690 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:07.731839 11777 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:07.820518  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:07.846073  8206 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:07.837651  8207 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:07.838843  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 737) took 1048 ms. Trace:
I20260709 13:14:07.839947  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:07.812368 10940 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:07.840533  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:07.843997  8104 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:07.847312  8728 rpcz_store.cc:276] 0709 13:14:06.781354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.781457 (+   103us) service_pool.cc:224] Handling call
0709 13:14:06.783093 (+  1636us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.541777 (+758684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.828560 (+286783us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54004d1554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:07.829398 (+   838us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1678678784}
I20260709 13:14:07.856242  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:07.733166 11722 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:07.880285  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50604.4us
W20260709 13:14:07.860402  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.861763  8101 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:07.830567 11679 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:07.888883  8132 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:07.882613  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:07.887792  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:07.901463  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 160561us
I20260709 13:14:07.903545  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:07.904126  8130 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:07.904798 11750 consensus_queue.cc:1059] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:07.905743 11750 raft_consensus.cc:3055] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:07.906040 11750 raft_consensus.cc:740] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:07.908164  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 64271.5us
I20260709 13:14:07.910449  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:07.892324 11763 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:07.913205  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:07.913498  8207 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:07.829082 11005 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:07.878908 11683 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:14:07.911520 11736 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:07.909215 11665 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:07.907850 11708 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:07.933440  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:07.922482 10940 consensus_queue.cc:1059] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:14:07.841334 11752 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:07.906773 11750 consensus_queue.cc:260] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:07.993080  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 74360.3us
W20260709 13:14:07.993939  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 98026.6us
I20260709 13:14:08.010263  8696 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:07.926270 11690 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:14:07.988852 11783 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.442s	user 0.004s	sys 0.000s
I20260709 13:14:07.900739 11768 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
W20260709 13:14:07.986687  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 105731us
W20260709 13:14:08.012072  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:08.013972  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 88306.1us
I20260709 13:14:07.926175 11689 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:08.017028  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:07.931322 11725 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:14:08.036320  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:08.036926  8206 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:08.000993 11714 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:07.992229 11809 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.998214 11708 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.041718  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:07.998438 11747 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:07.998865 11577 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:08.011375 11794 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.326s	user 0.001s	sys 0.001s
I20260709 13:14:07.900991 11746 consensus_queue.cc:260] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:07.946130 11659 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:14:07.910225 11776 consensus_queue.cc:1059] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 13:14:08.008245 11736 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:08.018553 11775 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:14:08.058401  8203 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:14:08.016815 11693 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.054471 11809 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:08.054611 11750 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:08.126647 11665 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:08.052436 11708 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:08.051991 11807 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:07.997911 11707 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.091246 11804 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.132020 11717 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:08.131531 11693 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:08.154889 11793 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.316s	user 0.001s	sys 0.000s
I20260709 13:14:08.155653 11693 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:08.162472 11804 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:08.135905 11746 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:08.124296 11799 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:08.136586 11770 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.174014 11770 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:08.136437 11720 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:08.174721 11720 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:08.175079 11720 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:08.178772  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.090807 11731 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
W20260709 13:14:08.152460 11783 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:14:08.093146 11465 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.196382  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:08.195919  8206 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
W20260709 13:14:08.199331  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:08.200829  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 164839us
I20260709 13:14:08.205119  8029 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:08.210589  8104 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:08.174122 11420 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.185555 11807 raft_consensus.cc:697] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:08.218163  8029 leader_election.cc:400] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10.
W20260709 13:14:08.204710 11576 consensus_meta.cc:287] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.637s	user 0.008s	sys 0.001s
I20260709 13:14:08.220466  8029 leader_election.cc:403] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:14:08.221454 11792 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
I20260709 13:14:08.222114 11810 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.090s
W20260709 13:14:08.222743 11795 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.316s	user 0.000s	sys 0.001s
W20260709 13:14:08.223650 11812 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:14:08.212137 11708 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.237275  8129 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:08.205305 11722 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:08.240477  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1270) took 1093 ms. Trace:
I20260709 13:14:08.243040  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:08.242362  8134 rpcz_store.cc:276] 0709 13:14:07.142404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.142567 (+   163us) service_pool.cc:224] Handling call
0709 13:14:07.142975 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:07.143200 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.143236 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:07.155857 (+ 12621us) log.cc:844] Serialized 26 byte log entry
0709 13:14:07.305752 (+149895us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:07.306205 (+   453us) log.cc:844] Serialized 14 byte log entry
0709 13:14:07.306378 (+   173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.306443 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.825382 (+518939us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:08.232861 (+407479us) raft_consensus.cc:1685] finished
0709 13:14:08.235282 (+  2421us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.235423 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9587,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":32171,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":48,"spinlock_wait_cycles":5565312,"thread_start_us":149565,"threads_started":1,"wal-append.queue_time_us":345173}
I20260709 13:14:08.244166  8029 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:08.218992 11776 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:08.220901 11709 consensus_meta.cc:287] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.765s	user 0.006s	sys 0.004s
I20260709 13:14:08.257362  8211 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:08.242063 11683 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:14:08.260869  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 243975us
I20260709 13:14:08.262252  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:08.266395 11719 consensus_queue.cc:1059] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:14:08.266875 11803 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.212s	user 0.000s	sys 0.000s
I20260709 13:14:08.267557 11818 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:08.268919 11760 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:08.270818 11805 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.271296 11805 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.271456 11720 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:08.280437  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69973.1us
I20260709 13:14:08.281090  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:08.266533  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:08.282895  8724 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:08.283967  8176 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:08.260303 11780 consensus_meta.cc:287] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.551s	user 0.010s	sys 0.000s
I20260709 13:14:08.286298  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:08.319200  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:08.254846 11804 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:08.324491  8104 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:08.317327 11682 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:08.327308  8724 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:08.259606 11815 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:08.325004 11708 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:08.330912  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 45543.8us
W20260709 13:14:08.321290 11792 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:14:08.203501 11736 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:08.240243 11665 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:08.334791  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 89678.2us
I20260709 13:14:08.316984 11722 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:08.325278 11813 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:08.322124 11750 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:08.336223  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:08.325021  7946 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b578d88c1c1f4ca58b7291fed4909f09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:14:08.347880  7946 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 68359.4us
I20260709 13:14:08.337247 11683 raft_consensus.cc:697] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:08.337806 11818 raft_consensus.cc:697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:08.326134 11780 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.340327 11722 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.357928 11818 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:08.355751 11802 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:14:08.352099  8206 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:08.362250 11817 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:08.363591 11683 consensus_queue.cc:237] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.364215 11465 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:08.364672 11720 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:08.336488 11770 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:14:08.368299 11410 raft_consensus.cc:3055] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:08.368710 11410 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:08.369655  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:08.369591 11410 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:08.378583  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 54171.6us
I20260709 13:14:08.342167 11819 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:14:08.346032 11702 consensus_meta.cc:287] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.709s	user 0.008s	sys 0.001s
I20260709 13:14:08.378186  8726 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:08.384666 11816 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:14:08.385488 11821 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:14:08.388553  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.389833  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:08.391376 11821 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:08.392861 11804 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:08.392696 11821 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.394112  8102 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:08.393394 11804 raft_consensus.cc:697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:08.394851 11720 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:08.395354 11720 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:08.395547 11804 consensus_queue.cc:237] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.396194 11720 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:08.397378  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.397758  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.400045  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:08.357311 11690 raft_consensus.cc:3055] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:08.402854 11690 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:08.403290  8135 raft_consensus.cc:2368] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10.
I20260709 13:14:08.403661 11690 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:08.404302 11780 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:08.389104 11702 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:08.349694 11809 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:08.407519  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.348080 11805 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:08.383360  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:08.411453  8131 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:14:08.373558 11722 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:08.417845  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.419916 11825 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:14:08.423866  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.429198  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.429600  8055 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:08.429791  8056 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:08.419951  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.431720  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.330458 11807 consensus_queue.cc:237] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:08.437588  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 54345.8us
W20260709 13:14:08.444061  8175 leader_election.cc:336] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.709s (SENT)
I20260709 13:14:08.431666 11714 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:08.411010 11770 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:14:08.431901 11420 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:08.457491  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.457401 11818 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.462779 11818 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:08.463898 11715 consensus_meta.cc:287] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.741s	user 0.006s	sys 0.003s
I20260709 13:14:08.464766 11818 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:08.466128 11768 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.466437 11815 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10.
I20260709 13:14:08.466683 11768 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:08.469592 11768 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:08.469961 11751 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:08.470393 11751 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:08.470746 11778 raft_consensus.cc:3055] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:08.471175 11778 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:08.471344 11751 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:08.471879 11778 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.472656 11751 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:08.472890 11778 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:08.473639 11760 raft_consensus.cc:3055] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:08.474035 11760 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:08.475133  8104 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:08.452395 11714 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:08.477332 11714 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:08.478055 11679 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.478560 11679 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.480885  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:08.470207 11689 raft_consensus.cc:3055] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:08.480475 11679 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:08.482276 11689 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:08.483115 11689 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.484242 11689 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:08.486773  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:08.487332  8206 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:14:08.488448  8696 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:08.489269  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.489218 11735 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:08.489751 11735 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.490072 11735 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:08.491798  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:08.492376  8134 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:08.494055 11752 raft_consensus.cc:3055] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:08.494470 11752 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:08.494649  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:08.495253  8137 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
W20260709 13:14:08.477015  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.495272 11752 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.498719  8104 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:08.500101  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.500669  8057 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:08.500757 11819 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:08.500344  8104 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:08.498805 11752 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:08.514245  8137 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: []
W20260709 13:14:08.586905  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 86620.7us
I20260709 13:14:08.515529 11725 raft_consensus.cc:3055] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:08.516547  8133 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:08.517359  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:08.531651  8129 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1674) took 2250 ms (client timeout 1795 ms). Trace:
I20260709 13:14:08.532641  8138 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:14:08.534298  8136 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:08.542132  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1570) took 1076 ms. Trace:
I20260709 13:14:08.590525  8211 rpcz_store.cc:276] 0709 13:14:07.465839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.466109 (+   270us) service_pool.cc:224] Handling call
0709 13:14:08.541923 (+1075814us) spinlock_profiling.cc:243] Waited 749 ms on lock 0x7b54001545d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:08.542058 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1722710400}
I20260709 13:14:08.546602 11828 raft_consensus.cc:3055] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:08.546885  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:08.562371 11679 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:08.589218  8129 rpcz_store.cc:269] 0709 13:14:06.280944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.282565 (+  1621us) service_pool.cc:224] Handling call
0709 13:14:08.531388 (+2248823us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b54003c91d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:08.531543 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4467100416}
I20260709 13:14:08.562790  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:08.593829  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:08.594398  8135 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:08.563519 11410 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:08.568583 11683 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.569672  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:08.570080  8134 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
W20260709 13:14:08.570567  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.474843 11760 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.581027  8130 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:14:08.581699  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.501475 11819 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:08.611243  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.611537  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:08.611877  8058 raft_consensus.cc:2376] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8. Re-sending same reply.
I20260709 13:14:08.612752  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:08.613831  8135 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
W20260709 13:14:08.613786  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 169753us
W20260709 13:14:08.514825 11775 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.278s	user 0.002s	sys 0.000s
I20260709 13:14:08.622846  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:08.623626  8130 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
W20260709 13:14:08.623952  8175 leader_election.cc:336] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.806s (SENT)
I20260709 13:14:08.624410  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:08.624527  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:08.625104  8130 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:08.625041  8693 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:08.626325 11773 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:08.626511 11709 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:08.628039  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.628594  8058 raft_consensus.cc:2376] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9. Re-sending same reply.
I20260709 13:14:08.629103 11722 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:08.630044  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.630636  8058 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:08.630650  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.631069  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.631531  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.631727  8056 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:08.596118  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:08.420022  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:08.614239  8063 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:08.545499 11690 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:08.597538  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:08.645018  8137 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: []
W20260709 13:14:08.584715  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.645530  8133 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:08.595314 11828 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:08.606858 11777 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:08.621272 11735 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:08.622438 11683 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.663188 11735 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:08.663467 11683 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:08.588634 11776 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.664489 11776 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.667346 11776 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:08.629427 11719 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.668218 11719 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.630239  8208 raft_consensus.cc:2393] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:08.632803  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:08.631804  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.615141  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:08.671589  8135 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:08.672014  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:08.630731  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:08.673741  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:08.638679  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.641799  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:08.642918  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:08.675307 11719 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:08.675820 11719 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:08.631312  8054 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:08.676265 11719 raft_consensus.cc:697] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:08.676487  8057 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:08.632105  8058 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:08.644083  8206 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:08.678185  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:08.678390  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:08.678740  8058 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:08.679364  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:08.676849  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:08.680039  8206 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:08.680339  8054 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:08.680734  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:08.616425 11679 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.676173  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:08.682245  8203 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:08.683319 11679 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:08.683804  8029 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:08.684607 11679 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:08.684964 11679 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:08.685322 11679 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:08.686996  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:08.689429  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:08.677160 11719 consensus_queue.cc:237] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.696156  8137 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:08.697444  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:08.590747  8026 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:08.654452  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.649066  8133 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:08.705065  8058 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:08.709126  8054 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
W20260709 13:14:08.715842  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:08.701922  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 111219us
I20260709 13:14:08.713065  8057 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:08.589197 11725 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:08.730506  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:08.731933  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:08.716212  8062 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:08.734149  8026 leader_election.cc:400] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10.
I20260709 13:14:08.735845  8210 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:08.737215  8212 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:08.738296  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:08.743039  8727 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:08.743701  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:08.744048  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:08.744747  8205 raft_consensus.cc:2410] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:14:08.744802  8207 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:08.745710  8136 raft_consensus.cc:1240] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:08.751112  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:08.751878  8026 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:08.758728  8026 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:08.644600 11815 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.760131  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:08.762203  8063 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 5. Current term is 6. Ops: []
W20260709 13:14:08.763589  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.588775 11714 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:08.769575  8026 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:08.770033  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:08.768692  8204 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:08.767983  8212 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:08.776265  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:08.777007  8210 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:08.783659  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:08.784559  8212 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:08.785184  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:08.785887  8063 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:08.788013  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:08.790153  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:08.792596  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.798035  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:08.799715  8101 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:08.800436  8101 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:08.806200  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.807085  8135 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:08.808976  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:08.810910  7959 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37774 (request call id 139) took 17397 ms (client timeout 15000 ms). Trace:
I20260709 13:14:08.812435  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:08.812867  8137 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:08.813541  8101 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:08.779069 11816 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:08.758786 11659 consensus_queue.cc:1059] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:14:08.819011  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:08.821085  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 195700us
I20260709 13:14:08.813643  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:08.763252 11825 consensus_queue.cc:1059] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:14:08.832190  8210 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:14:08.815300  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.757738 11818 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:14:08.825517  8137 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:08.828305  8104 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:08.788013 11707 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:08.839627  8724 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:08.803287 11776 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:08.807582 11819 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:14:08.854399  8175 leader_election.cc:336] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.972s (SENT)
I20260709 13:14:08.813623 11702 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:08.813956 11770 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:08.854913  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:08.861060  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.779201 11815 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:08.811429  7959 rpcz_store.cc:269] 0709 13:13:51.413628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.413870 (+   242us) service_pool.cc:224] Handling call
0709 13:13:52.541595 (+1127725us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b5400002210. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef20b2 00007f6dc0e6ba89 00007f6dc0e03da6 00007f6dc0dd5088 00007f6dc0ed337e 00007f6dbf3942a4 00007f6dbf394232 00007f6dbf394161 00007f6dbf3940dc
0709 13:13:54.122181 (+1580586us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b5400002210. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1e9c 00007f6db7ef20f6 00007f6dc0e6ba89 00007f6dc0e03da6 00007f6dc0dd5088 00007f6dc0ed337e 00007f6dbf3942a4 00007f6dbf394232 00007f6dbf394161 00007f6dbf3940dc
0709 13:14:08.810821 (+14688640us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:05.397069 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:05.397808 (+   739us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:05.397859 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:14:05.397878 (+    19us) tablet.cc:662] Decoding operations
0709 13:14:05.399147 (+  1269us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:05.399256 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:05.399282 (+    26us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:14:05.400651 (+  1369us) tablet.cc:701] Row locks acquired
0709 13:14:05.400709 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:14:05.400879 (+   170us) write_op.cc:270] Start()
0709 13:14:05.401013 (+   134us) write_op.cc:276] Timestamp: P: 1783602845400817 usec, L: 0
0709 13:14:05.401033 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:14:05.401904 (+   871us) log.cc:844] Serialized 5218 byte log entry
0709 13:14:05.973117 (+571213us) op_driver.cc:464] REPLICATION: finished
0709 13:14:06.159121 (+186004us) write_op.cc:301] APPLY: starting
0709 13:14:06.159235 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:06.159551 (+   316us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:06.159681 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:06.381479 (+221798us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:06.417945 (+ 36466us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:14:06.421289 (+  3344us) write_op.cc:312] APPLY: finished
0709 13:14:06.508113 (+ 86824us) log.cc:844] Serialized 95 byte log entry
0709 13:14:06.508294 (+   181us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:06.508431 (+   137us) write_op.cc:454] Released schema lock
0709 13:14:06.508509 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:06.508919 (+   410us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:06.508957 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6915325,"reported_tablets":28,"spinlock_wait_cycles":72823680,"child_traces":[["op",{"apply.queue_time_us":151120,"apply.run_cpu_time_us":5498,"apply.run_wall_time_us":357903,"num_ops":13,"prepare.queue_time_us":370069,"prepare.run_cpu_time_us":113106,"prepare.run_wall_time_us":473538,"raft.queue_time_us":71653,"raft.run_cpu_time_us":5162,"raft.run_wall_time_us":122917,"replication_time_us":559826,"thread_start_us":544169,"threads_started":4,"wal-append.queue_time_us":484052,"wal-append.run_cpu_time_us":46182,"wal-append.run_wall_time_us":1640464}]]}
I20260709 13:14:08.835391 11465 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:08.927537  8055 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:08.930218 11822 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.143s
I20260709 13:14:08.930967 11805 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:08.930461  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 87267.1us
I20260709 13:14:08.931715 11827 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:08.932471 11826 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:08.932891 11826 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.933239 11826 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:08.933915 11733 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:08.934331 11733 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.934669 11733 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:08.934814 11714 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:08.843253 11708 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:08.843422 11750 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:08.956856  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.835670 11682 consensus_queue.cc:1059] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 13:14:08.913968 11683 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:08.843804 11679 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.972947  8056 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:08.997610  8063 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
W20260709 13:14:09.027015  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 209258us
I20260709 13:14:08.848977 11707 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.974759 11714 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:14:09.042984  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 64204.4us
I20260709 13:14:09.040647  8210 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
W20260709 13:14:09.043628  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.810021 11828 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:08.974503 11702 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:09.053110  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:08.848790 11725 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:09.054987 11736 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:14:09.053359 11849 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:14:09.048821 11837 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:14:09.049338 11689 consensus_meta.cc:287] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.565s	user 0.007s	sys 0.004s
I20260709 13:14:09.066167  8056 raft_consensus.cc:2410] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:09.042096 11810 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:14:09.058467 11846 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:14:08.965117 11750 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.079516  8206 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:14:09.077524  8059 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:14:09.058022 11722 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:09.061753 11649 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:09.086817 11851 log.cc:927] Time spent T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
W20260709 13:14:09.087635 11841 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:14:09.057586 10940 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:14:09.080495 11725 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:09.108456  8137 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:14:09.113757  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:09.102219  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 273997us
I20260709 13:14:09.115259  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:09.116357  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:09.117445  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:09.067211 11736 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.118948  8696 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:09.110121 11770 raft_consensus.cc:697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:09.124185  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:09.136747  8132 raft_consensus.cc:2410] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:14:09.126524  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:09.128177 11659 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:09.128890 11732 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:09.130134 11736 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:09.105593 11649 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.130605  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:09.133039  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:09.133845 11306 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:09.134910 11750 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:09.135880  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:09.180547  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44744.1us
W20260709 13:14:09.154836 11843 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:14:09.155000 11816 raft_consensus.cc:3055] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:09.159639 11861 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:14:09.161283  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.192655 11816 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:09.162917  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:09.119827 11679 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:09.199091  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:09.199155  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.200273  8175 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:09.167176  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.121789 11465 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:09.172150  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:09.172942  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44722.7us
I20260709 13:14:09.177155 11665 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:09.177817 11814 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:09.080647 11810 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.178788  8133 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:09.179169 11746 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:09.190897 11649 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:09.191911  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:09.194187  8132 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:14:09.185017 11794 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:14:09.239287 11465 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:09.167316 11659 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:09.211411  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:09.212986  8212 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:09.214133  8211 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:09.136145 11702 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:09.221697  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:09.192482 11776 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:09.223774  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:09.224562  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:09.251382  8208 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:09.252468  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:09.224586  8204 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:14:09.232127 11861 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:09.232520 11750 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:09.209545 11804 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:09.255182 11804 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:09.233599 11707 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:09.235409 11746 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.256386 11746 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:09.236068 11770 consensus_queue.cc:237] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:09.237217 11816 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:09.210081 11306 raft_consensus.cc:697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:09.233767 11810 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:09.238631 11818 raft_consensus.cc:3055] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:09.216634 11722 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:09.259629 11818 raft_consensus.cc:740] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:09.260255 11722 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:09.260841 11818 consensus_queue.cc:260] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:09.262463  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.213132 11679 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:09.263331 11679 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.259651 11306 consensus_queue.cc:237] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.267297 11679 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:09.267452  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:09.255218  8203 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 6.
I20260709 13:14:09.262070 11722 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:09.268932  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:09.276125  8207 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
W20260709 13:14:09.248154 11752 consensus_meta.cc:287] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.653s	user 0.007s	sys 0.001s
I20260709 13:14:09.272161 11746 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:09.271370 11818 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:09.284368  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:09.286617 11776 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.287395 11750 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.287811 11708 raft_consensus.cc:3055] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:09.250257 11819 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:09.277244 11825 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:14:09.301586 11826 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:09.304405  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:09.305028  8135 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:09.307011 11826 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:09.279119 11679 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:09.316262  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:09.316816  8209 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:09.317984 11819 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:09.318742 11778 consensus_meta.cc:287] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.845s	user 0.010s	sys 0.000s
I20260709 13:14:09.320609 11659 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:09.324560  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:09.324856  8727 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:14:09.325313  8132 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:09.282724 11857 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:14:09.291863 11708 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:09.331626  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:09.310810 11733 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.336767 11733 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:09.337518  8726 raft_consensus.cc:2393] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:09.321878 11679 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:09.343969 11850 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
I20260709 13:14:09.344619 11679 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:09.296438 11750 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.345585  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:09.346773 11750 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:09.347682 11690 consensus_meta.cc:287] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.690s	user 0.006s	sys 0.003s
I20260709 13:14:09.291635 11776 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:09.349182  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:09.331475 11778 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:09.352046  8060 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:09.353426  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:09.354518  8204 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:09.356024  8133 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:14:09.358937  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:09.363279  8209 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:09.370464  8135 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:09.371743  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:09.375397  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:09.375949  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:09.376151  8133 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:09.372437  8135 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:09.379698  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:09.380506  8210 raft_consensus.cc:2393] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:09.380553  8696 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:09.350098 11776 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:09.358379 11751 consensus_meta.cc:287] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.879s	user 0.005s	sys 0.005s
I20260709 13:14:09.382778  8136 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:09.411450  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:09.412217  8132 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:09.412266  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:09.347849 11708 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:09.447785  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:09.577646  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.617113  8210 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:09.614352  8209 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:14:09.618923  8209 pending_rounds.cc:85] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 3
I20260709 13:14:09.619266  8209 pending_rounds.cc:107] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:14:09.619604  8209 raft_consensus.cc:2889] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:09.623679  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:09.626782  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.502887  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:09.485951  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 117231us
I20260709 13:14:09.429965  8133 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
W20260709 13:14:09.644380  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 206232us
I20260709 13:14:09.614249 11708 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:09.470912 11872 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:14:09.435591 11306 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:09.438266 11857 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:09.527127 11818 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.648306  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:09.650633  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.428225 11649 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.218s
I20260709 13:14:09.435474 11732 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.217s
I20260709 13:14:09.646692 11306 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.670542 11805 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:09.670928 11760 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:09.656220 11857 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.675000 11857 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:09.655093 11818 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.678390 11818 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:09.679033  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:09.681798 11815 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:09.682576  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:09.683147  8210 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:09.686692  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:09.687819  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:09.688133 11712 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:09.689100 11712 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:09.470486 11864 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.217s
I20260709 13:14:09.692297  8211 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:14:09.692772 11306 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:09.700248  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:09.702418  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:09.702682  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 54444.5us
I20260709 13:14:09.703855  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:09.705165  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:09.706550 11804 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:09.717460 11882 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.699272 11703 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:14:09.717955 11882 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.721349  8206 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:14:09.723502 11882 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:09.724597  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:09.728417 11821 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:09.728812 11821 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:09.729175 11821 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:09.698107 11712 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:09.738217 11712 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:09.740871  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:09.741671 11804 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:09.745369 11683 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:09.745854 11683 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:09.757548  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:09.737321 11306 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:09.782521  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.787596  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:09.789535 11874 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:09.797812  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:09.784129 11815 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.798877 11874 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:09.796742 11821 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:09.802455 11815 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:09.805441 11821 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:09.789915 11804 raft_consensus.cc:2764] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:14:09.815972  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:09.817571  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:09.819945  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 531) took 1087 ms. Trace:
I20260709 13:14:09.820581 11815 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:09.820592  8137 rpcz_store.cc:276] 0709 13:14:08.732979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.848257 (+115278us) service_pool.cc:224] Handling call
0709 13:14:08.848538 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:09.108966 (+260428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:09.819652 (+710686us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54001aac54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:09.819860 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590365952}
I20260709 13:14:09.819896  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:09.823067  8206 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:09.827564 11712 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.830327 11712 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:09.831312  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:09.833910  8206 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:09.834486  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:09.835309 11874 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:09.835644  8175 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:09.836724  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:09.837435 11746 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:09.837755  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:09.838020 11746 raft_consensus.cc:2764] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:14:09.838580 11886 raft_consensus.cc:3055] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:09.838801 11665 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:09.838970 11886 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:09.839234 11665 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.839648 11665 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:09.839877 11886 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:09.844882  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.840325  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:09.848114  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.848287  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:09.849166  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:09.850036  8055 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:09.850190  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:09.822837 11821 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:09.858382  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:09.859256  8061 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:14:09.854429  8056 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:14:09.857631  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:09.860644  8062 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:14:09.858261 11689 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.854084 11746 raft_consensus.cc:2764] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:14:09.861845 11689 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.862277 11746 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:09.862679 11746 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.863025 11746 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:09.864118 11689 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:09.854219 11420 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:09.866039  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:09.869573  8063 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:14:09.845953 11871 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:09.871132  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:09.871873  8055 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:09.858515 11874 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:09.863181 11881 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:14:09.874897 11828 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:09.876605 11874 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.853814 11683 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.887075  8055 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:09.890501 11866 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:09.892969 11874 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:09.895586  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:09.896457  8693 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:09.899067 11864 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:09.899485 11864 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.900069  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:09.901087  8206 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:14:09.924217  8061 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
W20260709 13:14:09.927315  8059 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1461) took 1496 ms (client timeout 1604 ms). Trace:
W20260709 13:14:09.927908  8059 rpcz_store.cc:269] 0709 13:14:08.406918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.411405 (+  4487us) service_pool.cc:224] Handling call
0709 13:14:09.902319 (+1490914us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b5400335c54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:09.902886 (+   567us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1483731200}
W20260709 13:14:09.929155  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 79025.5us
I20260709 13:14:09.929687  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:09.899802 11306 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:09.932387 11871 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:09.932888 11871 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.933249 11871 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:09.933284 11816 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:09.932471 11889 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:14:09.899986 11712 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:14:09.900584 11864 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:09.906546 11866 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:09.951308  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:09.955230  8206 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:09.950977 11866 raft_consensus.cc:697] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:09.958041 11888 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.167s	user 0.002s	sys 0.001s
I20260709 13:14:09.957901 11866 consensus_queue.cc:237] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.952798 11893 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:09.962853 11893 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.964752 11864 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.945124 11712 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:09.970515  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:09.970747 11712 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:09.955883 11821 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:09.965996 11893 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:09.973965  8063 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:09.977722  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:09.992424  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:09.969947 11874 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:09.994890 11874 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:09.998656  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:10.000267  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:10.000952  8059 raft_consensus.cc:2376] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9. Re-sending same reply.
I20260709 13:14:10.005388 11689 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:10.006099 11689 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.010057  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:10.010285 11689 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:10.004698 11871 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.014477  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:10.015244  8204 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:14:10.021618 11665 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.023667 11746 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.024268  8204 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:10.028403 11746 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:10.030898 11712 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:10.032096  8057 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:14:10.032436  8057 pending_rounds.cc:85] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 5
I20260709 13:14:10.032648  8057 pending_rounds.cc:107] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:14:10.032867  8057 raft_consensus.cc:2889] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:10.033205  7956 catalog_manager.cc:5697] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:10.033875  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:10.039558  8723 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:10.041152 11871 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:10.042415  7956 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 3 to 7, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 7 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:10.043438  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:10.047632  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:10.048487  8057 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:10.051226  8063 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:10.052286  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:10.056874 11712 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:10.089003  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:10.109228  7956 catalog_manager.cc:5697] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 4 to 7, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 7 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:10.112043  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:10.112663  8063 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:10.114816  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:10.059782 11709 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:10.045142 11864 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:10.126521  8063 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:10.108685 11712 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:10.114153 11420 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:10.131021  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:10.119418  7956 catalog_manager.cc:5697] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:10.062439 11871 consensus_queue.cc:1059] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:14:10.060024 11874 consensus_queue.cc:1059] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:14:10.125723 11665 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:10.141376  8057 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:10.142673 11871 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:10.134321 11746 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:14:10.144002 11712 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.144704 11746 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:10.145063 11746 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:10.145798 11746 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.147073 11746 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:10.148445  8062 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: [9.7-11.8]
I20260709 13:14:10.167512  8057 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:14:10.169315 11828 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.169793 11828 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.172684  8057 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:14:10.172770 11828 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:10.168421  7956 catalog_manager.cc:5697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 5 to 7, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 7 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:10.176347  7956 catalog_manager.cc:5697] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
W20260709 13:14:10.177704  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.178884 11886 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:10.178427  7956 catalog_manager.cc:5697] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:10.179718  8029 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:10.181969  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:10.180919  7956 catalog_manager.cc:5697] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 7, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 7 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:10.182566  8054 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:10.183872  7956 catalog_manager.cc:5697] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:10.190769  8054 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:10.191617  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:10.192116  8054 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:10.194970  8029 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:10.211854  8029 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:10.217345  8054 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:10.219025  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:10.219776  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:10.220556  8054 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:10.186482 11760 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:10.224460 11760 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.226104 11679 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:10.226516 11679 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:10.226855 11679 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:10.203166 11682 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:10.194860 11818 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.228307 11818 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.229441  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:10.230188 11818 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:10.209717 11828 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:10.231496 11828 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:10.232379 11828 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.259703  7956 catalog_manager.cc:5697] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:10.261157  7956 catalog_manager.cc:5697] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 3 to 8, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 8 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:10.262727  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:10.222380 11815 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:10.265049 11815 raft_consensus.cc:697] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:10.265245 11898 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.266134 11898 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.270900  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2609) took 1616 ms. Trace:
I20260709 13:14:10.271258  8060 rpcz_store.cc:276] 0709 13:14:08.654427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.830826 (+176399us) service_pool.cc:224] Handling call
0709 13:14:09.351888 (+521062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:09.352443 (+   555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:10.270739 (+918296us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b54001e96d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:10.270850 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1144410368}
I20260709 13:14:10.280417 11683 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:10.281566  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:10.283382  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1715) took 1179 ms. Trace:
I20260709 13:14:10.283814  8129 rpcz_store.cc:276] 0709 13:14:09.103983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.104119 (+   136us) service_pool.cc:224] Handling call
0709 13:14:10.283204 (+1179085us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b540008cd54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:10.283327 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1222071040}
W20260709 13:14:10.288810  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.295862  7956 catalog_manager.cc:5697] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 4 to 6, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 6 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:10.314994  7956 catalog_manager.cc:5697] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:10.251466 11760 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:10.340265 11732 consensus_queue.cc:1059] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:10.342068 11731 raft_consensus.cc:3055] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:10.342639 11731 raft_consensus.cc:740] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:10.343895 11731 consensus_queue.cc:260] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.314128 11703 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:14:10.311323 11818 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:10.361528  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:10.381860  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.389382  8054 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:10.266278 11815 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.422780  8133 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:10.401638 11821 raft_consensus.cc:3055] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:10.425024 11821 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:10.317155 11005 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:14:10.402879  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 586) took 1350 ms. Trace:
I20260709 13:14:10.426599  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:10.391404 11818 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.408886  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1718) took 1284 ms. Trace:
I20260709 13:14:10.429873  8726 rpcz_store.cc:276] 0709 13:14:09.052669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.052818 (+   149us) service_pool.cc:224] Handling call
0709 13:14:10.402620 (+1349802us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b540060de54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:10.402791 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642863488}
I20260709 13:14:10.411336 11778 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.412364  8056 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:14:10.370438 11898 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:10.436920  8132 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:10.437261 11818 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:10.415165  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1717) took 1300 ms. Trace:
I20260709 13:14:10.417058 11819 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:10.438154  8132 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:14:10.438469 11819 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.441017 11819 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:10.417748 11776 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:10.442101 11776 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.444103 11736 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:10.444551 11776 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:10.444800  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1295) took 1863 ms. Trace:
W20260709 13:14:10.418185  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 722133us
I20260709 13:14:10.445199  8138 rpcz_store.cc:276] 0709 13:14:08.581279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.657044 (+ 75765us) service_pool.cc:224] Handling call
0709 13:14:08.736699 (+ 79655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:09.785491 (+1048792us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:09.786540 (+  1049us) log.cc:844] Serialized 14 byte log entry
0709 13:14:10.421966 (+635426us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:10.422017 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:10.422062 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:10.444607 (+ 22545us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400174654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:10.444651 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:10.444752 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":552225,"prepare.queue_time_us":900,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":552388,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":67,"spinlock_wait_cycles":2411491584,"thread_start_us":82680,"threads_started":1,"wal-append.queue_time_us":105287}
I20260709 13:14:10.449986 11750 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.450495 11750 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.450769  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:10.451211 11883 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:10.451783 11883 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:10.418649 11777 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:10.419134  8026 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:10.452630 11883 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.454031  8026 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:10.417116  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 793519us
I20260709 13:14:10.456130 11821 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.456692  8026 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:10.457324 11821 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:10.457602 11857 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:10.419878  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:10.458249 11702 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:10.458912  8129 raft_consensus.cc:2410] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:10.458837 11702 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.420032 11752 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.460173  8026 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:10.460099 11752 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.458760 11857 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:14:10.461584 11702 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:10.420137 11770 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.461867 11420 raft_consensus.cc:3055] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:10.462103 11818 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:10.462364 11770 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.462656 11780 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.463306 11780 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.452472 11750 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:10.463779  8026 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:10.464370 11770 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:10.464558 11857 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:10.427994  8133 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:14:10.420611  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:10.428596  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:10.465225 11780 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:10.465399  8136 raft_consensus.cc:2393] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:10.465670  8137 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:10.467634  8026 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:10.469157  8026 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:10.469929 10940 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:10.471170  8212 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:14:10.472699  8204 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:10.473791  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:10.474956  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:10.476526  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:10.419584 11868 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:10.435042  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:10.490572  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:10.437889  8131 rpcz_store.cc:276] 0709 13:14:09.114299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.114431 (+   132us) service_pool.cc:224] Handling call
0709 13:14:09.122057 (+  7626us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:09.122401 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:09.122431 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:09.124327 (+  1896us) log.cc:844] Serialized 26 byte log entry
0709 13:14:09.215510 (+ 91183us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:09.472190 (+256680us) log.cc:844] Serialized 14 byte log entry
0709 13:14:10.414879 (+942689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:10.414948 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:10.414995 (+    47us) raft_consensus.cc:1685] finished
0709 13:14:10.415009 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:10.415115 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":942371,"prepare.queue_time_us":71157,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":26,"raft.queue_time_us":953697,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":170,"thread_start_us":330673,"threads_started":2,"wal-append.queue_time_us":167644}
I20260709 13:14:10.431756  8134 rpcz_store.cc:276] 0709 13:14:09.124799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.124967 (+   168us) service_pool.cc:224] Handling call
0709 13:14:09.125843 (+   876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:09.202635 (+ 76792us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:09.202681 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:09.209383 (+  6702us) log.cc:844] Serialized 26 byte log entry
0709 13:14:09.327555 (+118172us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:09.360529 (+ 32974us) log.cc:844] Serialized 14 byte log entry
0709 13:14:10.408374 (+1047845us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b180045bb60. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ea3c0a 00007f6dc15e3219 00007f6db7f2ba40 00007f6db7f264e4 00007f6db7f26455 00007f6dbb00e060 00007f6dbb020716 00007f6dbb031287 00007f6dbb031208
0709 13:14:10.408528 (+   154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:10.408595 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:10.408635 (+    40us) raft_consensus.cc:1685] finished
0709 13:14:10.408723 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:10.408835 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123865,"prepare.queue_time_us":42239,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":1142100,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":23848,"spinlock_wait_cycles":9768192,"thread_start_us":91576,"threads_started":1,"wal-append.queue_time_us":114996}
I20260709 13:14:10.439031  8132 raft_consensus.cc:606] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:10.492487  8132 raft_consensus.cc:614] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:10.419703 11707 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:10.460408  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:10.462702 11818 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:10.390614 11731 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:10.426116  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:10.478485  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:10.480772  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:10.525043  8206 raft_consensus.cc:2410] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:14:10.434041  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:10.525557  8056 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:10.434662  8135 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:10.399080  7956 catalog_manager.cc:5697] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:10.435097 11778 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:10.493687  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 274742us
I20260709 13:14:10.494771  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:10.498310  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:10.499357  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:10.462284 11420 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:10.519469  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 85297.3us
I20260709 13:14:10.538725  8101 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:10.527235  7956 catalog_manager.cc:5697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:10.538832 11420 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.529072  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:10.540015 11420 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:10.532521  8061 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:10.541944 11714 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:10.541894  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:10.542764  8061 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:10.539978  7956 catalog_manager.cc:5697] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 2 to 7, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 7 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:10.543752  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:10.544301  8136 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:10.544006  7956 catalog_manager.cc:5697] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 5 to 6, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 6 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:10.544770  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:10.545537  8132 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:10.545814  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:10.546110  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:10.546399  8133 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:10.546725  8130 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:10.546792  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:10.547466  8131 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:10.551851  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:10.533538  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:10.552652  8207 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:10.542852 11714 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:10.553185  8130 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:14:10.554497  8104 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:10.555186  8131 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:10.555536  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:10.556164  8132 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:10.557358 11714 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:10.557850 11714 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.558305 11714 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:10.560272  8203 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:14:10.560616  8203 pending_rounds.cc:85] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 6
I20260709 13:14:10.560807  8203 pending_rounds.cc:107] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:14:10.561002  8203 raft_consensus.cc:2889] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:10.561281  8203 pending_rounds.cc:107] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:14:10.561480  8203 raft_consensus.cc:2889] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:10.544313  8061 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:10.564288  8102 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:10.572183  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:10.462103 11752 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:10.488991 11857 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:10.575819  8055 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:10.462121 11702 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.577553  8057 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:10.611654 11702 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.614625  8135 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:10.615412  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:10.615983  8054 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:10.577450  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:10.591310  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:10.617201  8060 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:10.617313  8061 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:10.592911  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:10.520411 11773 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.121s
I20260709 13:14:10.489217 11760 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.618582  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:10.618772  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:10.618712 11760 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.619653  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:10.619913  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:10.620419  8133 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:10.620887 11760 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:10.622687  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:10.623242  8138 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:10.564934 11819 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:14:10.532456 11816 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:10.627722  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:10.520851 11770 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.107s
I20260709 13:14:10.576149 11776 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:14:10.630769 11776 consensus_queue.cc:1243] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:10.631444 11776 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:10.631831 11776 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:10.632817 11776 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.520731 11868 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.529681 11708 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.638744  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:10.640910  8134 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:10.640949  8130 raft_consensus.cc:606] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:10.651672  8130 raft_consensus.cc:614] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:10.641755  8210 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:10.653308  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:10.657305  8138 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:10.600183  8101 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:10.600183  8055 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:14:10.646797  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.523789 11818 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:10.665295  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:10.665906  8207 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:14:10.666483  8210 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:14:10.645449  8133 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:10.683449  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:10.637467 11889 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:14:10.530414 11707 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.518929 11777 consensus_queue.cc:1059] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.162s
I20260709 13:14:10.634308 11776 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:10.702659  8133 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:10.614230 11702 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:10.706301  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:10.637319 11868 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:10.640311 11752 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:10.729199  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45823.4us
I20260709 13:14:10.702018  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:10.730738  8138 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:14:10.660228 11770 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:10.732052 11752 consensus_queue.cc:1059] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:10.601286 11465 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.128s
I20260709 13:14:10.734748  8137 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:14:10.733526 11893 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:10.745335  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:10.680543 11708 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.734248 11881 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:10.717542 11712 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:14:10.728659 11649 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:10.719467 11420 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:14:10.719117 11805 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
W20260709 13:14:10.773509  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 222700us
I20260709 13:14:10.731029 11738 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:14:10.740485 11770 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.802212  8026 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:10.805390  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 241168us
I20260709 13:14:10.781046 11005 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:14:10.848776  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 45678us
I20260709 13:14:10.777899 11649 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.804190 11805 consensus_queue.cc:1243] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:14:10.914889  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.916306 11893 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:10.917232 11682 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:10.918716  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 69011.5us
I20260709 13:14:10.794777 11683 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:14:10.903245 11908 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:14:10.803720 11689 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.938629  8026 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:10.809073 11005 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:10.970764  8026 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:10.971849  8026 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:10.974076 10940 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:10.974619 10940 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.975076 10940 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:10.984086 11707 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:10.958129 11682 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:10.977685 11919 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:10.988081 11919 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:10.988533 11919 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:10.989348 11682 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:10.991170  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:10.987182 11816 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.943543 11903 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:10.995664  8726 raft_consensus.cc:2410] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:14:10.967420  8102 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:10.974709 11005 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:10.980062 11828 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:11.014369 11828 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:11.014895 11828 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:11.014426 11731 raft_consensus.cc:2955] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Committing config change with OpId 10.9: config changed from index 4 to 9, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:11.015726 11649 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:10.975324 11689 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:11.000666  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 462332us
I20260709 13:14:10.954613 11893 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:10.944751 11818 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:11.004433  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 276286us
I20260709 13:14:11.026988  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:11.027724 11874 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:11.028007  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:11.028767 11874 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:14:11.003145  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 322415us
I20260709 13:14:11.024806  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:11.018514  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:11.019749  8210 raft_consensus.cc:2955] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Committing config change with OpId 10.9: config changed from index 4 to 9, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:11.025283  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:10.998416 11707 raft_consensus.cc:3055] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:10.993746 11778 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:10.996408 11903 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.029168 11874 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.029330  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:11.031149  8055 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:11.032806  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:11.033597  8137 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:14:11.022753  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 422625us
I20260709 13:14:11.045768  8207 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:11.046290  8101 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:11.036051 10940 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.041290  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:11.043452  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.048720  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:11.076508 10940 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.054504  8101 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:10.998615 11816 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:11.140830  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.038250 11707 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:11.148373  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 129831us
I20260709 13:14:11.051478 11874 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:11.053962 11778 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.046517 11903 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:11.161509 10940 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:11.037035 11420 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.110s
I20260709 13:14:11.051767 11857 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:11.184751 11857 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:11.184944 11917 log.cc:927] Time spent T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:14:11.185196 11857 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:11.186493 11707 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.187595 11707 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:11.188156 11420 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:11.167657  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:11.190963  8207 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:11.192028  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:11.176026  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:11.167886 11922 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.200040  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:11.146217 11926 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:14:11.200578  8205 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:11.202939  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:11.204357  8211 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:14:11.184229 11916 log.cc:927] Time spent T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 13:14:11.211973  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:11.215497  8211 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:11.218189 10940 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:11.218832  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 189553us
I20260709 13:14:11.218843 10940 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.217927  8210 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:11.220113  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:11.213773  8205 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:11.223201  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:11.210712 11821 consensus_meta.cc:287] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.753s	user 0.007s	sys 0.006s
I20260709 13:14:11.225726  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:11.167943 11778 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.226734 11420 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:11.228569  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:11.229364  8205 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:11.199594 11922 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.207015 11925 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:11.225265 11679 consensus_meta.cc:287] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.996s	user 0.011s	sys 0.008s
I20260709 13:14:11.245998 10940 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.222378 11702 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:11.222285 11770 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:11.235566 11926 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:14:11.262708  8131 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.002s	sys 0.010s
I20260709 13:14:11.245071 11921 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:11.263420  8131 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.003s	sys 0.011s
I20260709 13:14:11.247452 11920 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.265862 10940 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.266319 10940 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.244727 11420 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:11.258291 11821 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:11.273911 11860 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:11.274644 11770 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.274963 10940 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.270705 11928 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:14:11.271023 11911 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:14:11.248052 11679 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:11.293676  8175 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:11.280033 11714 consensus_meta.cc:287] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.721s	user 0.010s	sys 0.000s
I20260709 13:14:11.290165 11770 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.297340 11714 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.297894 11926 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:11.270175 11921 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.300753 11679 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:11.247581 11925 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.272231 11818 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.302254  8058 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:11.336993  8175 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:11.288214 11928 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:11.304387 11776 consensus_meta.cc:287] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.578s	user 0.006s	sys 0.002s
I20260709 13:14:11.342739  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:11.354285 11926 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:11.354444  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:11.355506 11805 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:11.344336 11938 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:11.310775 11922 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.302495 11893 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:11.368091 11893 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:11.348078 11928 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:11.309844 11903 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.370594 11903 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.372624 11925 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.373211 11857 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.375226 11857 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.302574 11420 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.309054 11919 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:11.343433 11944 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:11.354876 11926 raft_consensus.cc:697] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:11.380460 11926 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.292078 11821 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:11.356985 11921 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.356734 11805 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:14:11.403012 11821 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.272877 11920 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.405378 11708 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.417718  8063 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:11.414784  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 360328us
I20260709 13:14:11.424549  8101 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:11.426589  8101 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:11.431032  8205 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:11.415537 11927 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:11.437362  8101 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:11.435506 11921 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:11.433177 11947 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:11.440620 11702 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:11.471055 11702 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.471676 11702 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:11.453297 11898 raft_consensus.cc:3055] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:11.438015 11925 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:14:11.438458 11922 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:14:11.463174 11857 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.468492  8062 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:11.474412  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:11.474736  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:11.491389  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:11.477169  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:11.439589  8101 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:11.480648  8060 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:14:11.482380  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 125522us
I20260709 13:14:11.482313  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:11.494270  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:11.474730  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:11.494925  8205 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:14:11.474426  8029 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:11.491497  8210 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:11.475340  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:11.495553  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:11.501538  8029 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:11.503377  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:11.503991  8210 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:11.480888 11777 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.031s	sys 0.023s
W20260709 13:14:11.510996  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 71508.7us
I20260709 13:14:11.480556 11903 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.434562 11708 consensus_queue.cc:1059] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:14:11.479715 11898 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:11.514125  8210 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:11.516990 10940 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:11.517326  8026 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:11.515547  8029 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:11.512465 11751 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:11.527354  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:11.525130 10940 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:11.468266 11828 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.507200 11933 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:11.510447 11943 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:11.542816 11943 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:11.543012  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:11.543323 11943 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.543762  8132 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:11.544067 11933 raft_consensus.cc:697] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:11.518234 11921 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.544910 11933 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.545111 11921 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:11.520983 11903 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.541635 11898 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.547490 11921 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:11.521718 11934 consensus_queue.cc:1059] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:14:11.548496 11903 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.549024 11903 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:14:11.522046 11005 consensus_meta.cc:287] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.500s	user 0.008s	sys 0.000s
I20260709 13:14:11.549362 11903 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:11.549738 11005 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:11.543710 11943 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:11.551736 11005 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:11.552322  8724 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:11.511642 11944 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:11.553144 11944 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:11.510212 11682 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:11.554879 11920 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.528070 10940 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:11.503451 11857 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.555857 11944 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:11.515987 11777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.031s	sys 0.024s
I20260709 13:14:11.555914 10940 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:11.557333  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:11.557547 11857 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.539178  8029 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:11.557934  8728 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:11.559332 11818 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:11.558286 11750 consensus_queue.cc:1059] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:11.559823 11818 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:11.560653 11828 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.550187 11903 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.562767  8029 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:11.516278 11874 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.562870 11903 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:11.563735 11828 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:11.564436 11828 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.564481  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:11.565084 11874 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:11.565310 11920 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:11.565730 11920 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.564046 10940 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:11.566133 11920 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:11.519742 11679 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:14:11.567737 10940 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:11.509897 11805 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:11.569381 11805 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:11.518296 11947 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:11.570472 11805 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.519452 11707 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:14:11.527930  8203 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:11.573088  8029 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:11.574092 10940 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:11.564977 11828 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:11.576201  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:11.577131  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:11.579384  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:11.579962  8206 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:11.580202  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:11.580900  8205 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:11.583638  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:11.585027  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:11.585501  8061 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:11.586396 10940 raft_consensus.cc:697] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:11.587602  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:11.588317  8056 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:11.590725  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:11.591753  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:11.592593  8130 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:11.584190  8060 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:11.572187 11943 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.597821  8135 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:11.517946 11948 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:11.593850  8175 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:11.527415 11751 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:11.587150 10940 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.598217  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:11.594776  8132 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:14:11.606837  8132 raft_consensus.cc:606] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:11.607306  8132 raft_consensus.cc:614] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:11.580668 11773 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.595202  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:11.588177 11708 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:11.595278 11750 consensus_queue.cc:1243] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:14:11.614570 11913 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.699s	user 0.008s	sys 0.006s
W20260709 13:14:11.622073  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 97325us
I20260709 13:14:11.590292 11420 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:14:11.608819 11751 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:11.607717 11954 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.611364 11874 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:11.632658  8061 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:11.636061  8130 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:11.611882 11919 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:11.646941  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 549) took 1008 ms. Trace:
I20260709 13:14:11.647779  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:11.647657  8133 rpcz_store.cc:276] 0709 13:14:10.638205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:10.723184 (+ 84979us) service_pool.cc:224] Handling call
0709 13:14:10.755509 (+ 32325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:10.755663 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:10.755689 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:10.755725 (+    36us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:10.755751 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:10.755801 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:11.646722 (+890921us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b54003c8cd0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:11.646870 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73853824}
I20260709 13:14:11.624243 11773 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:11.655059 11937 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:11.655578 11937 raft_consensus.cc:697] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:11.656388 11937 consensus_queue.cc:237] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.658802 11778 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.560662 11818 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.650712 11920 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.665690  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:11.668929  8210 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:11.632149 11952 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:11.659791 11943 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:11.703922  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 56200.6us
I20260709 13:14:11.625299 11708 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.646226 11874 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:11.731324  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:14:11.676194 11760 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.004s	sys 0.015s
I20260709 13:14:11.666393 11954 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.735037  8130 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:14:11.839609  8210 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:11.902652  8130 raft_consensus.cc:606] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:11.903206  8130 raft_consensus.cc:614] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:11.839074  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:11.947358  8027 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:11.949910 10940 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:11.952123 10940 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.966019 11683 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:11.966727 11683 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:11.948793 11760 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.193s	user 0.012s	sys 0.027s
I20260709 13:14:11.971472  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:11.723958 11731 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:38710 (request call id 1) took 2867 ms. Trace:
I20260709 13:14:11.822753 11874 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:11.726505 11420 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:11.972316 11919 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:12.005395 11959 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:12.006115 11959 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:12.007054 11954 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:11.705719 11952 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:11.981921 11420 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:11.845760 11828 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:12.044628  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 531) took 1020 ms. Trace:
I20260709 13:14:11.847975 11928 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:12.045182  8063 rpcz_store.cc:276] 0709 13:14:11.023839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:11.024445 (+   606us) service_pool.cc:224] Handling call
0709 13:14:11.024780 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:11.418201 (+393421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:12.044356 (+626155us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b540028a5d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:12.044559 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":903121792}
I20260709 13:14:12.073784  8203 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:12.078496 10940 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:12.069082 11702 consensus_meta.cc:287] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.000s
I20260709 13:14:11.974224 11731 rpcz_store.cc:276] 0709 13:14:08.805235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.805396 (+   161us) service_pool.cc:224] Handling call
0709 13:14:08.902287 (+ 96891us) log.cc:844] Serialized 494 byte log entry
0709 13:14:09.348379 (+446092us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b18004698a0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:14:09.677996 (+329617us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b2c000c0e58. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db990fa11 00007f6db991ea0a 00007f6db991e693 00007f6db991e631 00007f6dbb052b52 00007f6dbb052cf5 00007f6dbaf8f691 00007f6dbaf6feaa 00007f6dbaf76d53 00007f6dbaf76cd9 00007f6dbaf76c69
0709 13:14:10.405913 (+727917us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b1800030c00. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb052d01 00007f6dbaf8f691 00007f6dbaf6feaa 00007f6dbaf76d53 00007f6dbaf76cd9 00007f6dbaf76c69 00007f6dbaf76c31 00007f6dbaf75dfd 000000000038f354 000000000038f199
0709 13:14:11.668498 (+1262585us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12987,"raft.queue_time_us":23879,"raft.run_cpu_time_us":15466,"raft.run_wall_time_us":1635190,"spinlock_wait_cycles":771170304,"thread_start_us":110717,"threads_started":1,"wal-append.queue_time_us":196147,"wal-append.run_cpu_time_us":1969,"wal-append.run_wall_time_us":1036434}
I20260709 13:14:12.113405  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:12.114144  8137 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:12.117576  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:12.118355  8132 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:12.123445  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:12.124415  8133 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:12.125341  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:12.110527 11952 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:12.128052  8136 raft_consensus.cc:3055] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:12.128520  8136 raft_consensus.cc:740] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:12.129451  8136 consensus_queue.cc:260] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:12.149921  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:12.150893  8138 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:12.097177 11702 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:12.157727  8136 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:12.159725  8129 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:14:12.142607 11954 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.166023  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:12.153211 11924 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.037s	sys 0.040s
I20260709 13:14:12.151114 11708 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:12.156210 11780 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.167860  8133 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:12.224182  8136 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:12.224972  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1734) took 1679 ms. Trace:
I20260709 13:14:12.225361  8136 rpcz_store.cc:276] 0709 13:14:10.545233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:10.546431 (+  1198us) service_pool.cc:224] Handling call
0709 13:14:10.546684 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:12.127935 (+1581251us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540028d5a0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf8ddc0 00007f6dbb0181c0 00007f6dbb019231 00007f6dbb015d8f 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c
0709 13:14:12.224677 (+ 96742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:12.224907 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2324863872}
W20260709 13:14:12.174409 11924 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.038s	sys 0.040s
I20260709 13:14:12.239588 10940 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:12.240159 10940 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:12.167205 11954 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:12.234481 11828 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:12.242727 11952 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:12.285765 10940 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:12.226370 11780 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:12.297595 11974 log.cc:927] Time spent T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:14:12.297901 11977 consensus_queue.cc:1059] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:12.302670 11969 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.290831  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:12.292523 11928 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:12.314066 11969 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:12.320164  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:12.322953 11973 raft_consensus.cc:3055] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:12.326177 11770 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:12.342507  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 171557us
I20260709 13:14:12.345677  8208 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:14:12.357010  8137 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:12.333710 11973 raft_consensus.cc:740] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:12.354384 11702 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:12.388947 11751 consensus_meta.cc:287] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.722s	user 0.003s	sys 0.008s
I20260709 13:14:12.416141 11751 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:12.454141 11988 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:12.454517 11984 log.cc:927] Time spent T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:14:12.487103  8063 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:12.488742  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 643) took 1430 ms. Trace:
I20260709 13:14:12.501609  8060 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:12.455192 11683 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:12.456537 11987 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:12.524127 10940 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:12.524612 10940 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:12.458838 11951 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.897s	user 0.002s	sys 0.006s
I20260709 13:14:12.463457 11986 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:12.430146 11973 consensus_queue.cc:260] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:12.425767 11980 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:14:12.485024  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:12.527809  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:12.481549  8061 raft_consensus.cc:2955] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Committing config change with OpId 10.9: config changed from index 4 to 9, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:12.529633  8728 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:12.489137  8724 rpcz_store.cc:276] 0709 13:14:11.037953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:11.038877 (+   924us) service_pool.cc:224] Handling call
0709 13:14:11.039225 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:11.552710 (+513485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:12.467868 (+915158us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b5400422a54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:12.468079 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1179634944}
I20260709 13:14:12.537045  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:12.537779  8056 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:12.540216  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:12.540998  8060 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:14:12.542470  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 121342us
I20260709 13:14:12.544641 11780 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:12.545146 11962 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.979s	user 0.004s	sys 0.005s
I20260709 13:14:12.546196 11752 raft_consensus.cc:3055] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:12.546751 11752 raft_consensus.cc:740] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:12.547541 11752 consensus_queue.cc:260] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:12.548568 11752 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:12.550623  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:12.551570  8063 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:12.527104 11828 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.546269 11683 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:12.554929  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:12.560124  8060 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:12.536559 11986 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:12.563318 11986 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:12.570972 11683 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:14:12.575155  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.575670 11751 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:12.579573 11816 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:12.580214 11816 raft_consensus.cc:3055] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:12.580595 11816 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:12.587965  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:12.588749  8138 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:14:12.589998  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:12.590685  8138 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:12.590977  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:12.591962  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:12.592669  8136 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:12.593595  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:12.594725  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:12.595369  8131 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:12.597617  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:12.533803 11919 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:12.601428  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:12.608906  8060 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:12.590696  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:12.609731  8132 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:12.612180  8138 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:12.613037  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:12.613750  8138 raft_consensus.cc:2368] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10.
I20260709 13:14:12.614398  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:12.614863  8138 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:12.600732  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 65932.3us
I20260709 13:14:12.598265  8134 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:12.607074  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:12.535799 11987 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:12.530753 11988 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.365568  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:13.305058  8061 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.022s	sys 0.009s
I20260709 13:14:13.376264  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:13.376845  8061 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.022s	sys 0.012s
I20260709 13:14:12.970685  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:12.966792  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.628810  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:12.725014  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 181441us
I20260709 13:14:13.378399  8129 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:12.698889  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:12.759191  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 316264us
I20260709 13:14:12.550803 11973 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:12.627938  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:12.880218  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 521) took 2471 ms. Trace:
I20260709 13:14:12.673010  8134 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:13.365406  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:12.956441  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 756) took 1353 ms. Trace:
I20260709 13:14:13.383713  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:13.388510  8063 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:13.388708  8728 rpcz_store.cc:276] 0709 13:14:11.602746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:11.603235 (+   489us) service_pool.cc:224] Handling call
0709 13:14:11.603521 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:12.533497 (+929976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:12.956134 (+422637us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b5400505fd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:12.956367 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2129319168}
I20260709 13:14:12.941143  8138 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
W20260709 13:14:13.374208  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.391531  8138 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:12.974704  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:13.390938  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:13.393484  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 418886us
I20260709 13:14:13.373520 10940 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:13.394014  8136 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:12.786219  8131 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:13.395682  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:13.395880  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:13.396546  8726 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:13.396780  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:13.397364  8131 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
W20260709 13:14:13.375537  8060 consensus_meta.cc:287] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.766s	user 0.010s	sys 0.022s
I20260709 13:14:13.398612  8060 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:13.378067  8204 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:14:12.896395 11960 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 1.317s	user 0.000s	sys 0.013s
W20260709 13:14:13.077333 11956 log.cc:927] Time spent T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 1.514s	user 0.005s	sys 0.002s
I20260709 13:14:13.394322  8058 rpcz_store.cc:276] 0709 13:14:10.408996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:10.410129 (+  1133us) service_pool.cc:224] Handling call
0709 13:14:10.410418 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:11.334426 (+924008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:12.879903 (+1545477us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b540043d854. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:12.880139 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2050678016}
W20260709 13:14:12.963675 11952 consensus_meta.cc:287] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.669s	user 0.008s	sys 0.003s
I20260709 13:14:13.393157  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:13.410957  8138 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:13.411108  8131 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:13.404501  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:13.404071  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:12.617048  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:13.405622 10940 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:12.713183 11955 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 1.151s	user 0.004s	sys 0.024s
W20260709 13:14:12.981148 11953 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 1.439s	user 0.014s	sys 0.014s
I20260709 13:14:12.665866 11986 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:12.897081 11920 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:13.240931 11420 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:12.722983 11977 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:13.139385 11773 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:13.402978 11934 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:13.445626  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:13.610822  8135 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:13.529991  8138 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:13.482771  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:13.533429 10940 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.536188  8060 raft_consensus.cc:2410] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:14:13.536638  8061 raft_consensus.cc:2410] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:14:13.390789 11679 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:12.754221 11874 consensus_meta.cc:287] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.772s	user 0.007s	sys 0.004s
I20260709 13:14:12.600267 11828 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:13.543386  8204 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:12.802139 11689 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:13.535473  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:13.616634  8133 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:13.550822 11954 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:13.552364 11988 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:12.964352 11816 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.555805 11987 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:13.643225  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:13.643833  8205 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:13.558928  8058 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:12.820619 11864 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:13.564461  8129 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
W20260709 13:14:13.566491  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 154029us
I20260709 13:14:13.567454 11927 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:13.567557 11898 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:13.568110  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 155168us
I20260709 13:14:13.684724  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:12.840214 11714 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:13.579880  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:13.531625 11707 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.400179 11922 raft_consensus.cc:2764] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 13:14:12.999001 11465 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
W20260709 13:14:13.588228  8026 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.920s (SENT) [suppressed 16 similar messages]
I20260709 13:14:13.589349  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:13.391083 11857 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:14:13.594700  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 214430us
I20260709 13:14:13.414314  8136 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:13.722585  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1759) took 1111 ms. Trace:
I20260709 13:14:13.723018  8136 rpcz_store.cc:276] 0709 13:14:12.610967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.390610 (+779643us) service_pool.cc:224] Handling call
0709 13:14:13.722516 (+331906us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:13.724498  8136 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:13.489292 11969 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:13.601130  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:13.445996 11952 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:13.603454  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.604364  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:13.728711  8130 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:13.729173  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1764) took 1054 ms. Trace:
I20260709 13:14:13.729472  8130 rpcz_store.cc:276] 0709 13:14:12.674524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.596286 (+921762us) service_pool.cc:224] Handling call
0709 13:14:13.729124 (+132838us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:12.629703 11919 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.615041  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:13.567874 11959 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:13.725947 11920 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.629350  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:13.486881 11736 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.121s
I20260709 13:14:13.642302  8206 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:14:13.642201  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:13.675297 10940 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:13.645663  8176 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:13.743678  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 98084.8us
I20260709 13:14:13.676887  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1748) took 1088 ms. Trace:
I20260709 13:14:13.681134  8205 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:13.747588  8129 rpcz_store.cc:276] 0709 13:14:12.587801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:12.616807 (+ 29006us) service_pool.cc:224] Handling call
0709 13:14:13.646950 (+1030143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:13.747370  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:13.681823  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:13.684077  8133 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:13.748705  8136 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:13.748838  8129 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:13.569682 11986 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.673944 11683 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.689047  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:13.690629  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:13.692723  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:13.751526  8058 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:13.751921  8060 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:13.753057  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:13.753726  8060 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:13.684989  8693 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:13.754819  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 69901.5us
I20260709 13:14:13.756510  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:13.697428  8137 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:13.751539  8063 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:13.697835  8135 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:13.642868 11933 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.642658 11893 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:13.602881 11977 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:13.720835  8062 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:13.725672  8057 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:14:13.735148 11920 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:13.742439  8207 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:13.628193 11874 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:13.748067  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:13.748924  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1760) took 1137 ms. Trace:
I20260709 13:14:13.795984  8133 rpcz_store.cc:276] 0709 13:14:12.611182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.407701 (+796519us) service_pool.cc:224] Handling call
0709 13:14:13.748866 (+341165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:13.797435  8133 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:13.685339  8203 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:14:13.678180  7946 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a478a4313e6149378cae4b6f833e3c2a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:13.641314 11679 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.803447  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:13.803871  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:13.730060 11465 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:13.804561  8722 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:13.770836 11952 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:13.771488  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:13.772244  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:13.772325  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:13.773221  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:13.774063  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:13.737267 11714 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:13.790711  8060 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:13.794301  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:13.794435  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:13.802392  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:13.812187  8726 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
W20260709 13:14:13.812286 12007 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:14:13.813066  8206 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:13.855872  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:13.814163  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:13.782675  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1763) took 1156 ms. Trace:
I20260709 13:14:13.856845  8135 rpcz_store.cc:276] 0709 13:14:12.613433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.391827 (+778394us) service_pool.cc:224] Handling call
0709 13:14:13.770133 (+378306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:13.857861  8135 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: []
W20260709 13:14:13.819406  7946 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 141240us
I20260709 13:14:13.819823  8205 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:13.821879  8059 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:13.835572  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:13.775194 11922 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:13.710681 11927 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.701037 11864 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.794095 11959 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:13.814939  8212 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:13.838385  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:13.813354 12005 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:14:13.841847 10940 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:13.798779 11938 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:13.844120  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:13.749639 11707 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:13.735922 11857 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:13.842710 11689 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:13.686630 11934 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.794368 11933 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:13.813624 11981 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:13.878628  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:13.812773 11979 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:13.912436  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 324215us
I20260709 13:14:13.816295 11952 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:13.819499 11972 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:13.813012 11993 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:13.777489 11683 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:13.885510  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:13.931694  8206 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:13.838888 11967 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:13.794250 11893 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:13.844074 11997 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:13.891121 10940 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.872330  8061 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:13.940241  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:13.941260  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:13.907490  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:13.846086 12010 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:13.916338  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 171729us
I20260709 13:14:13.917603  8057 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ade1b48685164d64a62772d186ac959f"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:47112
I20260709 13:14:13.946475  8057 raft_consensus.cc:606] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:13.946868  8057 raft_consensus.cc:614] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:13.836344 11714 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.955942 11714 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:13.924412 11689 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.959043  8175 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:13.944203 11997 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:13.816870 11465 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.964088 11465 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:13.881078 12002 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:14:13.965515 11805 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:13.966106 11805 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.966638 11805 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:13.969175  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1641) took 2387 ms. Trace:
I20260709 13:14:13.886749 11959 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.923466 11981 raft_consensus.cc:697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:13.924331  8132 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:13.936655  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:13.971619 11981 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.972923  8063 raft_consensus.cc:2393] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:13.942736  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:13.921060 11952 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:13.954152  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:13.969628  8209 rpcz_store.cc:276] 0709 13:14:11.582050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:11.582175 (+   125us) service_pool.cc:224] Handling call
0709 13:14:11.582464 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:11.582787 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:11.582813 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:12.695801 (+1112988us) spinlock_profiling.cc:243] Waited 5.72 ms on lock 0x7b2c00116ca8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db990fa11 00007f6db991f372 00007f6dbb051a01 00007f6dbb01643a 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2
0709 13:14:12.696253 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 13:14:12.879365 (+183112us) spinlock_profiling.cc:243] Waited 4.57 ms on lock 0x7b48002301d8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:14:12.952166 (+ 72801us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:12.952224 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:12.952319 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:13.968906 (+1016587us) raft_consensus.cc:1685] finished
0709 13:14:13.968932 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:13.969080 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":17528,"raft.run_wall_time_us":1292146,"spinlock_wait_cycles":23662080,"thread_start_us":255536,"threads_started":1,"wal-append.queue_time_us":708059}
I20260709 13:14:13.959275 11689 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:13.970582 11959 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:13.911108 11938 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:13.988847  8209 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
W20260709 13:14:13.995082 12004 log.cc:927] Time spent T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
W20260709 13:14:13.998543 12006 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 13:14:13.920686 11979 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:14.004637 12014 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.005354 12014 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.934960 11993 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:14.012516 11805 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.888307 11922 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:14.016355 11986 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:14.017064 11986 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:13.945827 11967 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:13.919965 11972 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:14.021517 11959 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:13.964380 11864 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:13.985255 11465 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.103255 11465 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:13.933595  8203 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:13.932812  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:13.975755 12010 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:14.079499 11707 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:14.087160 12017 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:14:14.123797 11750 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:14.125867  8206 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:14.454695  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:14.344942  8693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.020s	sys 0.027s
W20260709 13:14:14.455885  8693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.020s	sys 0.031s
W20260709 13:14:14.459060  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 645124us
W20260709 13:14:14.359577  8130 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.000s	sys 0.009s
W20260709 13:14:14.460280  8130 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.000s	sys 0.012s
I20260709 13:14:14.456326 11922 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:14.363494  8027 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.002s	sys 0.007s
I20260709 13:14:14.397779  8026 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:14.458427 11689 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:14.369717  8101 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.007s	sys 0.003s
W20260709 13:14:14.471628  8101 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.008s	sys 0.004s
W20260709 13:14:14.366209  8131 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.006s	sys 0.006s
W20260709 13:14:14.369817  8178 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.003s	sys 0.007s
I20260709 13:14:14.471566 11689 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:14.472257  8178 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.004s	sys 0.008s
W20260709 13:14:14.472206  8131 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.007s	sys 0.007s
I20260709 13:14:14.126565 11465 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:14.453379  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 635) took 2108 ms. Trace:
W20260709 13:14:14.310454  8104 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.865s	user 0.026s	sys 0.068s
W20260709 13:14:14.473325  8104 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.028s	user 0.029s	sys 0.080s
I20260709 13:14:14.473742 11689 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:14.475340  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 77697.1us
I20260709 13:14:14.474738 12029 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:14.479982 12029 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:14.473244  8210 rpcz_store.cc:276] 0709 13:14:12.345288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:12.346466 (+  1178us) service_pool.cc:224] Handling call
0709 13:14:12.351661 (+  5195us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:12.423286 (+ 71625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:12.423316 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:12.427477 (+  4161us) log.cc:844] Serialized 26 byte log entry
0709 13:14:12.758598 (+331121us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:12.758647 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:12.758735 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:13.989659 (+1230924us) raft_consensus.cc:1685] finished
0709 13:14:13.989683 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.453315 (+463632us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104361,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":303029,"threads_started":2,"wal-append.queue_time_us":1394525}
I20260709 13:14:14.482183  8026 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:14.483309  8026 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:14.484584 11689 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:14.485960 11689 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:14.486941 11689 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:14.487232 12023 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:14.487389 11689 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:14.486405  8027 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.005s	sys 0.007s
W20260709 13:14:14.501047  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 644235us
W20260709 13:14:14.412506 11954 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.004s	sys 0.010s
I20260709 13:14:14.503786  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:14.487907 11689 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:14.506480 11927 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:14.515121  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1334) took 1118 ms. Trace:
I20260709 13:14:14.515121  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:14.368364 11937 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:14.137603 11973 consensus_meta.cc:287] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.737s	user 0.008s	sys 0.000s
W20260709 13:14:14.513378  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.506861 11959 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:14.515715  8131 rpcz_store.cc:276] 0709 13:14:13.396553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.692195 (+295642us) service_pool.cc:224] Handling call
0709 13:14:13.692517 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:13.692772 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:14.472761 (+779989us) log.cc:844] Serialized 14 byte log entry
0709 13:14:14.514816 (+ 42055us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:14.514866 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:14.514906 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.514969 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.515068 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30726,"prepare.queue_time_us":765299,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":790543,"threads_started":2,"wal-append.queue_time_us":11297}
I20260709 13:14:14.507419 12010 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:14.520408  8026 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:14.469472 11857 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:14.523582  8063 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:14.532902  8208 raft_consensus.cc:2410] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:14:14.508009 11954 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.006s	sys 0.011s
I20260709 13:14:14.532984  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 598) took 1170 ms. Trace:
I20260709 13:14:14.533649  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1753) took 1944 ms. Trace:
I20260709 13:14:14.538995  8724 rpcz_store.cc:276] 0709 13:14:13.362855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.363004 (+   149us) service_pool.cc:224] Handling call
0709 13:14:13.363376 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:13.911896 (+548520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:13.911941 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:13.911995 (+    54us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:13.912038 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.532742 (+620704us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b54004d2e54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:14.532786 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.532930 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1260899072}
I20260709 13:14:14.539122  8134 rpcz_store.cc:276] 0709 13:14:12.589481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.399534 (+810053us) service_pool.cc:224] Handling call
0709 13:14:13.399840 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:13.400250 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:13.400279 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:13.402788 (+  2509us) log.cc:844] Serialized 26 byte log entry
0709 13:14:13.844697 (+441909us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:14.391940 (+547243us) log.cc:844] Serialized 14 byte log entry
0709 13:14:14.453608 (+ 61668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.453682 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:14.533462 (+ 79780us) raft_consensus.cc:1685] finished
0709 13:14:14.533480 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.533598 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":613935,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":23,"raft.queue_time_us":2144,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":50290,"thread_start_us":773162,"threads_started":2,"wal-append.queue_time_us":428702}
I20260709 13:14:14.534026  8026 leader_election.cc:400] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10.
I20260709 13:14:14.536026  8062 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:14:14.548182  8062 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:14.549261  8062 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:14.550264  8062 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:14.551613  8062 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:14.555671  8026 leader_election.cc:403] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:14:14.514673 11465 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:14.517665 12034 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:14:14.558738  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.560036  8063 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:14:14.505288 11864 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:14:14.563093  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:14.495194 11736 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:14:14.604493  8724 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:14.604323  8101 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:14.564642  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.565951  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:14.656571  8063 pending_rounds.cc:85] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 5
I20260709 13:14:14.533008 11997 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:14.657727  8063 pending_rounds.cc:107] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:14:14.558240  8026 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:14.538036 12022 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.658943  8063 raft_consensus.cc:2889] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:14.655974 11750 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:14:14.694278  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2640) took 1102 ms. Trace:
I20260709 13:14:14.694693  8054 rpcz_store.cc:276] 0709 13:14:13.591979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.592092 (+   113us) service_pool.cc:224] Handling call
0709 13:14:13.593207 (+  1115us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:13.593623 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:14:13.692259 (+ 98636us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b48000801d8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:14:13.821372 (+129113us) log.cc:844] Serialized 14 byte log entry
0709 13:14:14.375819 (+554447us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:14.458162 (+ 82343us) log.cc:844] Serialized 26 byte log entry
0709 13:14:14.458328 (+   166us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:14.458841 (+   513us) log.cc:844] Serialized 14 byte log entry
0709 13:14:14.459009 (+   168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.459225 (+   216us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:14.694039 (+234814us) raft_consensus.cc:1685] finished
0709 13:14:14.694062 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.694181 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":361249,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":103614,"raft.run_cpu_time_us":1978,"raft.run_wall_time_us":87372,"spinlock_wait_cycles":54012160,"thread_start_us":709186,"threads_started":2,"wal-append.queue_time_us":692526}
I20260709 13:14:14.551393 12010 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:14.557361 11805 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:14.691056  8724 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:14.560583 11927 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:14.699194 12038 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
W20260709 13:14:14.698778  7945 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a003b4584d3747a381522d67299aaed2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 84d775d1767e4cc4aa7b9c627d8f0fb8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:14.518229 12024 consensus_queue.cc:1059] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.141s
W20260709 13:14:14.700153  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 141985us
I20260709 13:14:14.561088 12041 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.564353 12034 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:14.529796 11937 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:14.560402 11707 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.702240  8176 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:14.655826 12016 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.557554 11893 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:14.563381 11922 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:14.603375 11938 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:14.725013  8026 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:14.775463  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50552.6us
I20260709 13:14:14.776356  8026 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:14.777019  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1768) took 1227 ms. Trace:
I20260709 13:14:14.603767 11465 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:14.787559  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1644) took 1818 ms. Trace:
I20260709 13:14:14.788038  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1646) took 1043 ms. Trace:
I20260709 13:14:14.788727  8209 rpcz_store.cc:276] 0709 13:14:13.744461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.994151 (+249690us) service_pool.cc:224] Handling call
0709 13:14:13.994474 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:14.006754 (+ 12280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:14.006798 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:14.006852 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:14.006904 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.787754 (+780850us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b540023e1d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:14.787815 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.787969 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27712384}
I20260709 13:14:14.790812  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 545) took 1021 ms. Trace:
I20260709 13:14:14.791198  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2773) took 1008 ms. Trace:
I20260709 13:14:14.791287  8058 rpcz_store.cc:276] 0709 13:14:13.769113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.862724 (+ 93611us) service_pool.cc:224] Handling call
0709 13:14:13.900771 (+ 38047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:13.930750 (+ 29979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:13.930782 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:13.930837 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:13.930880 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.790549 (+859669us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b540001bdd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:14.790607 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.790745 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68497792}
I20260709 13:14:14.791893  8205 rpcz_store.cc:276] 0709 13:14:13.786615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.803174 (+ 16559us) service_pool.cc:224] Handling call
0709 13:14:14.790990 (+987816us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b54002c9054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:14.791131 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36188032}
I20260709 13:14:14.794422  8029 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:14.788110  8211 rpcz_store.cc:276] 0709 13:14:12.969284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:12.969897 (+   613us) service_pool.cc:224] Handling call
0709 13:14:12.970241 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:13.820342 (+850101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:13.820369 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:13.820407 (+    38us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:13.820437 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.787250 (+966813us) spinlock_profiling.cc:243] Waited 850 ms on lock 0x7b540023dcd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:14.787314 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.787490 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1954672896}
I20260709 13:14:14.723987 11750 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:14.774576 11805 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:14.817476  8205 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:14.717293 12045 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:14.807518 12025 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:14.716853 12015 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.104s
I20260709 13:14:14.811133 11750 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.775233 11707 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:14.752499 11927 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:14.820786 12024 raft_consensus.cc:3055] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:14.699585 12022 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:14.781610 12016 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:14.962675  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:14.889752  8104 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:14.824715 11805 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:14.867658 11898 consensus_meta.cc:287] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 1.165s	user 0.006s	sys 0.005s
I20260709 13:14:14.891398 11954 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:15.031975  8696 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.001s	sys 0.005s
W20260709 13:14:15.347172  8696 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.003s	sys 0.007s
I20260709 13:14:14.487823 12023 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:15.352753  8212 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:14.799019 11922 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:14.961410 11979 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:14.868435 12025 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:14.839468 11750 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:14.961288  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:15.079514  8693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.004s	sys 0.012s
I20260709 13:14:14.729424 12041 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:15.054662  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:15.367497  8693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.004s	sys 0.012s
I20260709 13:14:15.267495 12024 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:14.829942 12034 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:14.911226 12040 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:14:14.909627 12054 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:14.823274 12010 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:14.840389 12046 raft_consensus.cc:3055] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:14:15.038489  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 434277us
I20260709 13:14:15.415294  8101 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:14.835764 12045 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:14.960268 11993 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:14.910403 12053 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:14.839982 12018 log.cc:927] Time spent T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.360s	user 0.000s	sys 0.002s
I20260709 13:14:14.656440 12028 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10.
I20260709 13:14:15.071798 11988 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.285725  8721 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:14:15.051779 12057 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.451229 12057 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:15.055424 11920 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:15.071802 12052 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:15.355722  8211 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:14:15.367823  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:14.877686 12043 log.cc:927] Time spent T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 13:14:15.367928  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 814973us
I20260709 13:14:15.387399  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:15.356125 11821 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.350004 12015 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
W20260709 13:14:15.393179 12050 log.cc:927] Time spent T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.526s	user 0.001s	sys 0.000s
W20260709 13:14:15.393652 12051 log.cc:927] Time spent T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.440s	user 0.000s	sys 0.001s
I20260709 13:14:15.354923 11954 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:15.580286 12016 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:15.557773 12034 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:15.399580 11937 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:15.401715 11465 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:15.452201 11988 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:15.593642 11954 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:15.422531 12049 log.cc:927] Time spent T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.542s	user 0.001s	sys 0.000s
I20260709 13:14:15.459813 12052 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:15.594129 11988 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:15.596356 12052 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:15.453410 12025 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:15.600927 11927 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:15.601854 11707 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:15.413100 11805 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:15.606773 12057 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:15.611883 11750 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:15.423458 12024 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:15.615000 11738 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:15.414402 12054 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:15.617358 12054 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:15.625932 11689 consensus_meta.cc:287] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 1.104s	user 0.005s	sys 0.003s
I20260709 13:14:15.418911 12046 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:15.661193 12046 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:15.663086 12046 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:15.419045 12062 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:15.426788 12045 raft_consensus.cc:697] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:15.667026 12045 consensus_queue.cc:237] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:15.670661 12062 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:15.671865 12062 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:15.463934 11993 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:15.674969 11993 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:15.677623 11993 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:15.584841 12016 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:15.678411 12016 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:15.678944 12016 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:15.573639 11981 consensus_queue.cc:1059] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:14:15.683223 11981 raft_consensus.cc:3055] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:15.683777 11981 raft_consensus.cc:740] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:15.684882 11981 consensus_queue.cc:260] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:15.574043 11821 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:15.587028 12012 log.cc:927] Time spent T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 1.119s	user 0.003s	sys 0.001s
I20260709 13:14:15.426450 12053 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:15.689010 12053 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:15.587648 12010 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:15.690742 11981 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:15.588791 11952 consensus_meta.cc:287] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 1.605s	user 0.010s	sys 0.002s
I20260709 13:14:15.692401 11952 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:15.694883 11952 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:15.591751 12022 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:15.629302 11750 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:15.358295 12061 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:14:15.406822 11933 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:15.657127 12008 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.088s	sys 0.247s
W20260709 13:14:15.697423 12008 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.088s	sys 0.247s
I20260709 13:14:15.704981 11689 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:15.709363 11689 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:15.781049 12016 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:15.854848 12054 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:15.857965 12054 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:15.030539  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 214977us
I20260709 13:14:15.901700  8026 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:15.906236  8026 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:15.908933  8026 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:14:16.089712  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 181085us
W20260709 13:14:15.909798  8027 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.858s	user 0.056s	sys 0.127s
W20260709 13:14:16.090638  8027 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.054s	user 0.057s	sys 0.127s
I20260709 13:14:16.091931  8026 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:15.031420  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 237109us
I20260709 13:14:16.149678 11465 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.920903  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2641) took 2328 ms. Trace:
I20260709 13:14:16.151135  8029 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:16.150777 11465 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:16.151507  8056 rpcz_store.cc:276] 0709 13:14:13.592307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.592444 (+   137us) service_pool.cc:224] Handling call
0709 13:14:13.599860 (+  7416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:13.602394 (+  2534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:13.602436 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:13.609581 (+  7145us) log.cc:844] Serialized 26 byte log entry
0709 13:14:13.902819 (+293238us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:13.902856 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:13.902926 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:15.920126 (+2017200us) raft_consensus.cc:1685] finished
0709 13:14:15.920203 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:15.920579 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":41045,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":26724,"thread_start_us":215080,"threads_started":1,"wal-append.queue_time_us":693442}
I20260709 13:14:16.153501 11689 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:16.154214  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.154443 11689 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:14:16.154645 11805 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.002s	sys 0.006s
W20260709 13:14:16.155256 11805 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.002s	sys 0.006s
W20260709 13:14:15.940091  8059 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2643) took 2346 ms (client timeout 1666 ms). Trace:
W20260709 13:14:15.964653  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.295010  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1329) took 1901 ms. Trace:
W20260709 13:14:15.972512  8104 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.003s	sys 0.012s
I20260709 13:14:14.777585  8130 rpcz_store.cc:276] 0709 13:14:13.549775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.735903 (+186128us) service_pool.cc:224] Handling call
0709 13:14:13.746451 (+ 10548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:13.746678 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:14.470671 (+723993us) log.cc:844] Serialized 14 byte log entry
0709 13:14:14.776510 (+305839us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:14.776618 (+   108us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:14.776684 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.776770 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:14.776945 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11931,"prepare.queue_time_us":711758,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":92612,"thread_start_us":948781,"threads_started":3}
I20260709 13:14:15.978883  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 563) took 2212 ms. Trace:
I20260709 13:14:15.980027  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 566) took 2211 ms. Trace:
I20260709 13:14:15.998040  8054 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:15.998932  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.000941  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:15.428856  8176 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:16.199831 12028 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.200924 11954 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.200778 12028 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:16.016032  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:16.017977  8203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2774) took 2221 ms (client timeout 1666 ms). Trace:
I20260709 13:14:16.019001  8207 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:16.019322  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2762) took 2390 ms. Trace:
I20260709 13:14:16.019760  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 639) took 2112 ms. Trace:
W20260709 13:14:16.021806  8208 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2776) took 2204 ms (client timeout 2065 ms). Trace:
I20260709 13:14:16.203835 12055 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:16.204353 12055 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:16.204821 12055 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:16.191744  8137 rpcz_store.cc:276] 0709 13:14:13.768968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.856818 (+ 87850us) service_pool.cc:224] Handling call
0709 13:14:13.857080 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:13.972097 (+115017us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:13.972155 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:13.972208 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:13.972251 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.979568 (+2007317us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540008cad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:15.979631 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:15.979860 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264041728}
W20260709 13:14:16.205775  8104 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.004s	sys 0.012s
I20260709 13:14:16.022830  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 644) took 2089 ms. Trace:
W20260709 13:14:16.206391  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 1.31661e+06us
I20260709 13:14:16.191368  8133 rpcz_store.cc:276] 0709 13:14:13.767984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.799183 (+ 31199us) service_pool.cc:224] Handling call
0709 13:14:13.838505 (+ 39322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:13.971037 (+132532us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:14:14.453837 (+482800us) log.cc:844] Serialized 14 byte log entry
0709 13:14:15.356927 (+903090us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b180036acc0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ea3c0a 00007f6dc15e3219 00007f6db7f2ba40 00007f6db7f264e4 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf 00007f6dbb030038
0709 13:14:15.357171 (+   244us) log.cc:844] Serialized 14 byte log entry
0709 13:14:15.357384 (+   213us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:15.357448 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:15.357505 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.978388 (+620883us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b5400062054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:15.978449 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:15.978681 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76587,"prepare.queue_time_us":1309426,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":278844,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":94710,"spinlock_wait_cycles":308806272,"thread_start_us":533569,"threads_started":2,"wal-append.queue_time_us":47391}
I20260709 13:14:16.206498  8210 rpcz_store.cc:276] 0709 13:14:13.932988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.520342 (+587354us) service_pool.cc:224] Handling call
0709 13:14:14.525022 (+  4680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:14.525977 (+   955us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:14.526018 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:14.541533 (+ 15515us) log.cc:844] Serialized 26 byte log entry
0709 13:14:14.818508 (+276975us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:14.818546 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.818605 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:16.022388 (+1203783us) raft_consensus.cc:1685] finished
0709 13:14:16.022428 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.022655 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102898,"raft.queue_time_us":6825,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":18669,"thread_start_us":255232,"threads_started":1,"wal-append.queue_time_us":302490}
I20260709 13:14:16.208472 11979 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:16.024451  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2771) took 2303 ms. Trace:
I20260709 13:14:16.025713  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1652) took 1570 ms. Trace:
I20260709 13:14:16.026697  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 645) took 2088 ms. Trace:
I20260709 13:14:16.038623  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:16.047118  7956 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34120 (request call id 247) took 19291 ms (client timeout 15000 ms). Trace:
W20260709 13:14:16.047284  8696 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.014s	sys 0.012s
I20260709 13:14:15.867236 12016 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:16.155134  8056 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:15.013775  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 544) took 1245 ms. Trace:
I20260709 13:14:16.154081 11465 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:16.158699  8059 rpcz_store.cc:269] 0709 13:14:13.593346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.593528 (+   182us) service_pool.cc:224] Handling call
0709 13:14:15.939520 (+2345992us) spinlock_profiling.cc:243] Waited 91 ms on lock 0x7b54001e96d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:15.939795 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209237376}
I20260709 13:14:16.089965 10940 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:16.203217 11954 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:16.046099  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.155156 11689 raft_consensus.cc:697] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:14:16.212177  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:16.202219  8203 rpcz_store.cc:269] 0709 13:14:13.796228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.798734 (+  2506us) service_pool.cc:224] Handling call
0709 13:14:16.017495 (+2218761us) spinlock_profiling.cc:243] Waited 74 ms on lock 0x7b5400110654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:16.017712 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170190592}
I20260709 13:14:16.209623  8211 rpcz_store.cc:276] 0709 13:14:14.454793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.813791 (+358998us) service_pool.cc:224] Handling call
0709 13:14:14.823858 (+ 10067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:15.460335 (+636477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.025232 (+564897us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b5400241dd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:16.025549 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1222335744}
I20260709 13:14:16.209879  8209 rpcz_store.cc:276] 0709 13:14:13.938256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.814522 (+876266us) service_pool.cc:224] Handling call
0709 13:14:14.814844 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:15.374574 (+559730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:15.374616 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:15.374670 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:15.374713 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.026204 (+651491us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54002413d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:16.026261 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.026513 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1286727424}
I20260709 13:14:16.209300  8212 rpcz_store.cc:276] 0709 13:14:13.721348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.900027 (+178679us) service_pool.cc:224] Handling call
0709 13:14:13.900581 (+   554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:15.353246 (+1452665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.024029 (+670783us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400110b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:16.024296 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3338288384}
I20260709 13:14:16.308768  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:16.211937  8208 rpcz_store.cc:269] 0709 13:14:13.817574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.878396 (+ 60822us) service_pool.cc:224] Handling call
0709 13:14:16.021338 (+2142942us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b54000d5054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:16.021577 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1424783232}
I20260709 13:14:16.202893  8204 rpcz_store.cc:276] 0709 13:14:13.628422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.629184 (+   762us) service_pool.cc:224] Handling call
0709 13:14:13.629501 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:13.629835 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:13.629873 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:13.646202 (+ 16329us) log.cc:844] Serialized 26 byte log entry
0709 13:14:14.138873 (+492671us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:14.386706 (+247833us) log.cc:844] Serialized 14 byte log entry
0709 13:14:14.386997 (+   291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.387097 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:15.586397 (+1199300us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:16.018579 (+432182us) raft_consensus.cc:1685] finished
0709 13:14:16.018772 (+   193us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.019057 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":319515,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":35,"raft.queue_time_us":137210,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":19938,"spinlock_wait_cycles":5773696,"thread_start_us":648606,"threads_started":2,"wal-append.queue_time_us":779730}
I20260709 13:14:16.301467  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:16.202955  8206 rpcz_store.cc:276] 0709 13:14:13.907465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:14.471441 (+563976us) service_pool.cc:224] Handling call
0709 13:14:14.471687 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:14.810857 (+339170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:14.810887 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:14.810926 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:14.810957 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.019274 (+1208317us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b54000b9ad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:16.019349 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.019580 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":779591936}
I20260709 13:14:16.302925 11998 consensus_queue.cc:1059] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:14:16.304322 11874 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:16.298295 11933 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:14:16.297678  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 868730us
W20260709 13:14:16.361354  8207 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55942 (request call id 642) took 2442 ms (client timeout 2030 ms). Trace:
I20260709 13:14:16.361585  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:16.362372  8210 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:16.361928  8207 rpcz_store.cc:269] 0709 13:14:13.919288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.942514 (+ 23226us) service_pool.cc:224] Handling call
0709 13:14:16.361097 (+2418583us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b54002c8b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:16.361277 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4702082816}
W20260709 13:14:16.350580 12019 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 2.279s	user 0.039s	sys 0.070s
I20260709 13:14:16.363415  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:16.364775  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:16.365440  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2642) took 2772 ms. Trace:
I20260709 13:14:16.351840  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:16.365844  8055 rpcz_store.cc:276] 0709 13:14:13.592781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.593848 (+  1067us) service_pool.cc:224] Handling call
0709 13:14:13.594200 (+   352us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:13.602631 (+  8431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:13.602668 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:13.609842 (+  7174us) log.cc:844] Serialized 52 byte log entry
0709 13:14:13.840010 (+230168us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:13.840068 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:13.840145 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:16.349608 (+2509463us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:16.364889 (+ 15281us) raft_consensus.cc:1685] finished
0709 13:14:16.364915 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.365221 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13231,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":25291,"spinlock_wait_cycles":32308992,"thread_start_us":152257,"threads_started":1,"wal-append.queue_time_us":265004}
I20260709 13:14:16.367437  8131 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:16.365453  8207 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:16.369261  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:16.369968  8207 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:16.352957  8138 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: []
W20260709 13:14:16.313815  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 222203us
W20260709 13:14:16.232872  8696 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.015s	sys 0.013s
W20260709 13:14:16.299386  8693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.005s	sys 0.026s
W20260709 13:14:16.296512  8101 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.014s	sys 0.018s
W20260709 13:14:16.360498  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 155111us
I20260709 13:14:16.360883 11998 raft_consensus.cc:3055] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:16.271696 12010 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:16.297029 11988 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:16.379578  8101 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.014s	sys 0.023s
W20260709 13:14:16.235407  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 85269.1us
I20260709 13:14:16.296726 11993 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.383749 11993 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:16.296573 11750 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.295953 11952 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:16.386467 11750 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:16.386929 11952 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:16.389209 11750 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:16.380347  8693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.010s	sys 0.031s
I20260709 13:14:16.296139 12025 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:16.393647  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.394196 12025 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:16.232517  8132 rpcz_store.cc:276] 0709 13:14:13.393254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.401491 (+  8237us) service_pool.cc:224] Handling call
0709 13:14:13.528850 (+127359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:13.971618 (+442768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.294702 (+1323084us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b540015b654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:15.294936 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":902421248}
I20260709 13:14:16.297456 12022 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.399871 12022 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:16.401850  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:16.409825  8130 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:14:16.361586 11933 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:16.410532 11933 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:16.411553  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.412637  8135 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:16.378954 12028 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:16.414206  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.415295  8132 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:16.299714 11987 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.116s
I20260709 13:14:16.418519  8060 rpcz_store.cc:276] 0709 13:14:13.768784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.851063 (+ 82279us) service_pool.cc:224] Handling call
0709 13:14:13.851323 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:14.507920 (+656597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:14.507951 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:14.507990 (+    39us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:14.508019 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:15.013491 (+505472us) spinlock_profiling.cc:243] Waited 656 ms on lock 0x7b5400169254. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:15.013552 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:15.013704 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1509594496}
I20260709 13:14:16.421077  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.376437 11998 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:16.424132  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 867346us
I20260709 13:14:16.424676 11998 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:16.425519  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:16.397456 12025 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:16.378928 11689 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:16.402891  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.431196  8134 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:16.300529 12052 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.402974  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.432217 12052 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:16.432735  8137 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:14:16.404273  8138 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:14:16.311030 12080 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:14:16.381016 11988 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:16.435118  8138 pending_rounds.cc:85] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 3
I20260709 13:14:16.390118  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:16.445956  8138 pending_rounds.cc:107] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:14:16.459458  8138 raft_consensus.cc:2889] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:16.402181 12022 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:16.232358 11979 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:16.432171 11919 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:16.461694  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:16.463059  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 143323us
I20260709 13:14:16.460172 11998 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:16.466575  8131 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:16.469578  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.460316 11993 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:16.474511  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:16.476243 11981 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.476606  8696 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:16.477144 11981 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:16.480639 12070 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:16.481567 12070 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:16.482739 11981 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:16.484120 11988 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:16.485044  8137 raft_consensus.cc:2410] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:16.488157 11979 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:16.479735 11773 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:16.489399 11773 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:16.491544 11773 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:16.493230 11977 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:16.479007 11954 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:16.470736 11952 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:16.518038  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 1.10277e+06us
I20260709 13:14:16.518216 12024 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:16.519842  8101 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:16.520331  8062 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
W20260709 13:14:16.522701  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 165953us
I20260709 13:14:16.525449  8210 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
W20260709 13:14:16.523115  8175 connection.cc:256] RPC call timeout handler was delayed by -1.3880706630000077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
W20260709 13:14:16.528236  8175 connection.cc:256] RPC call timeout handler was delayed by -1.2836706629998389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
W20260709 13:14:16.210201  7956 rpcz_store.cc:269] 0709 13:13:56.743045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.743324 (+   279us) service_pool.cc:224] Handling call
0709 13:13:57.165605 (+422281us) spinlock_profiling.cc:243] Waited 44.8 ms on lock 0x7b5400172d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf80090 00007f6dbb01ef3a 00007f6dc0dd51cc 00007f6dc0ed337e 00007f6dbf3942a4 00007f6dbf394232 00007f6dbf394161 00007f6dbf3940dc 00007f6dbf393392 00007f6db935148c 00007f6db93508c6
0709 13:14:16.034105 (+18868500us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:11.125259 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:11.186649 (+ 61390us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:11.186687 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:14:11.186704 (+    17us) tablet.cc:662] Decoding operations
0709 13:14:11.188117 (+  1413us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:11.188173 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:11.188205 (+    32us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:14:11.189488 (+  1283us) tablet.cc:701] Row locks acquired
0709 13:14:11.189513 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:14:11.189802 (+   289us) write_op.cc:270] Start()
0709 13:14:11.189915 (+   113us) write_op.cc:276] Timestamp: P: 1783602851189759 usec, L: 0
0709 13:14:11.189936 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:14:11.245510 (+ 55574us) log.cc:844] Serialized 6679 byte log entry
0709 13:14:12.042360 (+796850us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6db7f0c0ac 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b
0709 13:14:12.937617 (+895257us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b1800315040. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1
0709 13:14:12.938325 (+   708us) op_driver.cc:464] REPLICATION: finished
0709 13:14:13.555703 (+617378us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b5400172d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01332a 00007f6dbaf98f14 00007f6dbaf98ec3 00007f6dbaf98e38 00007f6dbaf98df3 00007f6dbaf980dc 00007f6dbafc1e16 00007f6dbafa9cbb 00007f6dbaf9439d 00007f6dbaf97d75
0709 13:14:13.735172 (+179469us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b1800315040. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:14:13.735330 (+   158us) write_op.cc:301] APPLY: starting
0709 13:14:13.735410 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:13.742091 (+  6681us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:13.742117 (+    26us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:14.564525 (+822408us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:14.727609 (+163084us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:14:14.779075 (+ 51466us) write_op.cc:312] APPLY: finished
0709 13:14:14.891605 (+112530us) log.cc:844] Serialized 119 byte log entry
0709 13:14:15.697905 (+806300us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:15.698241 (+   336us) write_op.cc:454] Released schema lock
0709 13:14:15.698319 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:15.698724 (+   405us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:15.698751 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":49,"spinlock_wait_cycles":103073664,"child_traces":[["op",{"apply.queue_time_us":561550,"apply.run_cpu_time_us":346914,"apply.run_wall_time_us":1963343,"num_ops":17,"prepare.queue_time_us":462321,"prepare.run_cpu_time_us":89554,"prepare.run_wall_time_us":1894706,"raft.queue_time_us":151007,"raft.run_cpu_time_us":21495,"raft.run_wall_time_us":1114149,"replication_time_us":1748185,"spinlock_wait_cycles":1152187776,"thread_start_us":2597008,"threads_started":5,"wal-append.queue_time_us":1740623,"wal-append.run_cpu_time_us":63263,"wal-append.run_wall_time_us":1291012}]]}
I20260709 13:14:16.521884  7959 catalog_manager.cc:5697] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 5 to 9, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 9 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:16.536053  8130 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:16.537079  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.538262 11465 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:16.538869 11465 raft_consensus.cc:697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:16.538594  7959 catalog_manager.cc:5697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:16.540030 11465 consensus_queue.cc:237] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:16.541127 12014 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:16.540905  7959 catalog_manager.cc:5697] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 8 to 10, now has a pending config: VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65), VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67), VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 10 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:16.542874 11972 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:16.535812  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.527084 11954 raft_consensus.cc:2764] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:14:16.545883  8138 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:16.547132 11679 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:16.547318  7959 catalog_manager.cc:5697] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 7 to 11, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 11 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:16.534188 12052 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:16.556556  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:16.560180  8138 raft_consensus.cc:2368] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9.
I20260709 13:14:16.558590  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:16.554592  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 129353us
I20260709 13:14:16.560822  8137 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:16.565392  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.566311  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.566641  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:16.567302  8135 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:14:16.567607  8134 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:16.569753  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.572711  8131 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:16.579196 11934 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:16.580986  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 43794.9us
I20260709 13:14:16.583905 11683 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:16.589712 11821 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:16.591742  8130 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:16.595023 11828 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:16.586081  7959 catalog_manager.cc:5697] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:16.569073 11933 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:16.603689  8131 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:16.605096  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.605706  8130 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:16.607646  8104 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:16.609361  8104 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:16.610751 11954 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:16.611166 11954 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:16.611588 11954 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:16.618963 12066 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:16.619251 12060 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.619539 12066 raft_consensus.cc:697] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:16.619835 12060 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:16.620795 12066 consensus_queue.cc:237] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:16.622298 12060 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:16.623853  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.624667  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.625404  8726 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:16.625742  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.626579  8721 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:16.626962  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.627564  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.628016  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.626978  8723 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:16.629155 12010 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:16.630311  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:16.631126  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 167931us
I20260709 13:14:16.632046  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:16.634517  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 171067us
I20260709 13:14:16.597194  7959 catalog_manager.cc:5697] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:16.650426  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.637192 11933 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:16.586632 12055 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:16.647532 12014 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:16.652088 11679 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:16.661100  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.662873  8726 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:16.663389 11920 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:16.673126  8723 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
W20260709 13:14:16.676019 12030 log.cc:927] Time spent T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 2.198s	user 0.012s	sys 0.012s
I20260709 13:14:16.681263  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 565) took 2912 ms. Trace:
I20260709 13:14:16.673904 12069 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:16.676131 11679 raft_consensus.cc:697] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:16.692006 11679 consensus_queue.cc:237] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:16.695950 12055 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:16.690016 11920 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:16.698822  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 61423us
I20260709 13:14:16.681597  8129 rpcz_store.cc:276] 0709 13:14:13.768694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.852508 (+ 83814us) service_pool.cc:224] Handling call
0709 13:14:13.889789 (+ 37281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:13.890158 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:13.890194 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:13.908757 (+ 18563us) log.cc:844] Serialized 26 byte log entry
0709 13:14:14.360741 (+451984us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:14.360822 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:14.360958 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:16.573863 (+2212905us) spinlock_profiling.cc:243] Waited 3.14 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:16.680823 (+106960us) raft_consensus.cc:1685] finished
0709 13:14:16.680853 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.681111 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":66398,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":77,"spinlock_wait_cycles":7217408,"thread_start_us":413326,"threads_started":1,"wal-append.queue_time_us":525420}
I20260709 13:14:16.703301  8104 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:16.705760  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:16.706772 12069 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:16.706837  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.707360 12069 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.707777 12069 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:16.710731 12020 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:16.713002  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.713625  8722 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:16.717116  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.718691  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.719588  8062 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:16.720563  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.721866  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.723604  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.728607  8056 raft_consensus.cc:2410] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:14:16.654042 12010 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:16.729661  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.731549  8063 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:14:16.734037  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.735355  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:16.737025 12084 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:14:16.740975  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.741688  8059 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:16.723959  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:16.724790  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:16.749311  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.753757 10940 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.677685  7959 catalog_manager.cc:5697] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:16.759598 12023 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:16.762643  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.763967  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.765692  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.766077  8137 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:16.766604  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.766937  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.767795  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:16.751844 12060 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.768479  8054 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:16.766638  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.768975 10940 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:16.769381  8062 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:16.770246  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:16.770815  8062 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:16.770992 10940 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:16.771694  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.772725  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:16.773411  8723 raft_consensus.cc:2368] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9.
I20260709 13:14:16.773540  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.768476  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.774858  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.775187  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.775866  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.776429  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.776962  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.777746  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.778569  8060 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:16.779122  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:16.782959 11922 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:16.852842  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.784515  8063 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:16.853933  8060 raft_consensus.cc:2410] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:14:16.854183  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.785516 12023 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.855386 12023 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:16.788789  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.789328  8176 leader_election.cc:400] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9.
I20260709 13:14:16.856873  8176 leader_election.cc:403] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 13:14:16.856693  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 380270us
I20260709 13:14:16.792791  8054 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:16.793579  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:16.856736  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.857431  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.795392  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.795799 12031 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.861744 12031 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:16.864001 12031 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:16.864097 12064 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.864688 12064 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:16.866781  8057 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:14:16.867663 12023 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:16.869656  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 80581us
W20260709 13:14:16.870508  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45594.7us
I20260709 13:14:16.872177 11874 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9.
I20260709 13:14:16.872903 12045 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:16.874068  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.803311  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.885156  8725 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:16.808015 11920 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:16.813369  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:16.814023  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 108416us
I20260709 13:14:16.803474  8062 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:16.814751 12062 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:16.887327 12062 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.887696 12062 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:16.816654 11967 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:16.823185 12041 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:16.887708  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:16.895444 11922 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:16.895844 11922 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:16.896303 11922 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:16.898523  8027 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:16.899121  8104 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:16.839463 12060 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:16.841570 11954 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:16.911778 11954 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:16.791935  8101 connection.cc:256] RPC call timeout handler was delayed by -1.55075822300023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
I20260709 13:14:16.859316  7959 catalog_manager.cc:5697] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:16.860962  8059 raft_consensus.cc:2376] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10. Re-sending same reply.
I20260709 13:14:16.914166  7959 catalog_manager.cc:5697] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 5 to 10, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 10 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:16.915760  7959 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 7 to 9, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 9 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:16.917411  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.918530  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.919128  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.919255  8132 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:16.919337  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.874162 12013 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:16.927524  8727 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:14:16.927696 12013 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:16.928093 12013 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:16.929790  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 61062.8us
I20260709 13:14:16.930648  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.931969  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.933243  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:16.882047 12087 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
W20260709 13:14:16.935555  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 51807.1us
W20260709 13:14:16.882526  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 158728us
I20260709 13:14:16.884451 12064 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:16.899375 11920 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:16.938856 11920 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.938807  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:16.901788  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:16.939378  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:16.940094  8135 raft_consensus.cc:2376] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10. Re-sending same reply.
I20260709 13:14:16.902659  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.838533  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:16.938759  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:16.910074 12060 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:16.918612  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.801561 11816 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:16.927302  8134 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:16.939459 11920 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:16.942605  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 43690.1us
I20260709 13:14:16.944403  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:16.948212  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 402868us
W20260709 13:14:16.948680  8175 connection.cc:256] RPC call timeout handler was delayed by -1.240808782999693s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
I20260709 13:14:16.904441 12069 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:16.949714  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:16.949018  8175 connection.cc:256] RPC call timeout handler was delayed by -1.1588369260000491s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1529s for a response
I20260709 13:14:16.952323  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:16.954026  8175 connection.cc:256] RPC call timeout handler was delayed by -1.1480369260000316s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
I20260709 13:14:16.949517  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.959091  8133 raft_consensus.cc:2410] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:16.952137  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.959523  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:16.953569  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 195608us
I20260709 13:14:16.960660  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.961032  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.958684  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:16.963349  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.961570  8723 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:16.947882  8101 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:16.965866  8102 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:16.966696 12016 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:16.967155 12016 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:16.948021 12041 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.962854  7959 catalog_manager.cc:5697] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 7 to 10, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 10 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:16.942966 12062 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:16.969774  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:17.021363  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:16.985729  8175 connection.cc:256] RPC call timeout handler was delayed by -1.047236926000096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1653s for a response
W20260709 13:14:17.126216  8175 connection.cc:256] RPC call timeout handler was delayed by -1.0238369260000582s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
I20260709 13:14:17.126293  8131 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
W20260709 13:14:17.126616  8175 connection.cc:256] RPC call timeout handler was delayed by -1.0238369260000582s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
I20260709 13:14:16.984905  8129 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:14:17.102649  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 136859us
I20260709 13:14:17.127799  8131 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:14:17.126952  8175 connection.cc:256] RPC call timeout handler was delayed by -1.0238369260000582s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
I20260709 13:14:17.131541  8131 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:14:17.133342  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:17.134459  8131 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:17.021076  8135 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:14:16.961953 11954 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:17.135820  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:16.967563 12016 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:17.138625  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:17.140693  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 202007us
I20260709 13:14:17.145706 12062 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:17.146409 12062 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:16.989255 12066 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:17.151222 11922 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:17.151695  8058 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:14:17.152443 11857 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.951910 12045 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.204s
I20260709 13:14:17.152899 11857 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:17.155517 11857 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:17.156868  8136 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:17.157094  8723 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:14:17.158648  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 150812us
I20260709 13:14:17.158615 11998 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:17.160068  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:17.159962 12041 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:17.162503  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:17.164160  8693 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:17.164572 12098 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:14:17.165343 11773 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:17.167668  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 184844us
I20260709 13:14:17.171334  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.171700  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:17.173302  7959 catalog_manager.cc:5697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 8 to 9, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 9 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:17.173761 12013 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:17.071552 11750 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:17.175665  8131 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:17.174893  7959 catalog_manager.cc:5697] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:17.187316 12069 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:17.153261 11922 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:17.162201 12041 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:17.138010 11954 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:17.202981  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:17.203914  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.176609 11773 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:17.191682  7959 catalog_manager.cc:5697] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 1 to 7, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 7 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:17.193674 12062 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:14:17.194440 11954 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:17.221488  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:17.222267  8135 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:17.222008  7959 catalog_manager.cc:5697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:17.210942 11750 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:17.232416  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:17.230211  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.234622  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 553) took 1239 ms. Trace:
I20260709 13:14:17.239060  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.225716 11750 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.236263  8058 rpcz_store.cc:276] 0709 13:14:15.987331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.994268 (+  6937us) service_pool.cc:224] Handling call
0709 13:14:15.994700 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:17.152087 (+1157387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.226972 (+ 74885us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540025f8d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:17.227156 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2659769216}
I20260709 13:14:17.243072  8135 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:14:17.246876  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.246954  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.248018  8209 raft_consensus.cc:2410] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:14:17.233851 11998 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:17.232064 12070 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:14:17.249136 11998 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:17.251443 11998 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:17.253104 11816 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:17.255438 12045 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:17.256639  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.257956  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.259156  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.259819  8208 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:17.260721  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.261509  8211 raft_consensus.cc:2410] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:14:17.262444  8728 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:14:17.248018  8204 raft_consensus.cc:2410] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:14:17.242909 11750 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:17.270438  8210 raft_consensus.cc:1813] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Handling vote request from an unknown peer 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:17.271013  8210 raft_consensus.cc:2410] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:14:17.271351 12013 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:17.272591 11920 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:17.246048  7959 catalog_manager.cc:5697] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 9 to 10, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 10 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:17.293231  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:17.293951  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.297225  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.303346  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:17.293533 12062 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.304919 12016 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:17.305553 12062 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:17.306979  8206 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:17.307961 12062 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:17.309163  8208 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:17.309342  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.308120  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:17.310887  8206 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:17.310990  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:17.313175 11967 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:17.314134  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.314873  8203 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:17.316699  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.313508 11920 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:17.312183 11816 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:14:17.310083  8205 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:17.317412  8212 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:14:17.318918  7959 catalog_manager.cc:5697] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 7 to 8, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 8 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:17.293525 11750 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:17.346318 11967 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:17.347443  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:17.348151  8205 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:17.348809  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.351039  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:17.351745  8211 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:17.352386  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.324075  8209 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:17.323693 12062 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.357527  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:17.348253 11750 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:17.369997  8212 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:17.349025 11967 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:17.368794  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.368631  7959 catalog_manager.cc:5697] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:17.380003  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:17.323365 12023 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:14:17.369343 11816 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.391603 12016 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:17.396265  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.404129  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:17.404863  8728 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:17.404745  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 55153.6us
I20260709 13:14:17.404141  8203 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:17.389338 12031 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:17.365188 11874 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:14:17.406463  8104 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:17.410445  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.411571  8212 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:14:17.402990  7959 catalog_manager.cc:5697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:17.413976  8204 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:17.392980 12062 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:17.425525 12041 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:17.426352 11857 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:17.426913 11857 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:17.426054 12062 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:17.428879 11857 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:17.429533  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.395406 11816 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:17.438885  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:17.439553  8203 raft_consensus.cc:2368] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10.
I20260709 13:14:17.439942 11816 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:17.426527  7959 catalog_manager.cc:5697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:17.440347  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.442742  8205 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:17.445030  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:17.449538  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.409566 12031 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:17.480307 12031 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:17.481366  8203 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:17.461266  7959 catalog_manager.cc:5697] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 2 to 5, term changed from 4 to 9, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68), 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 9 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:17.501859 12101 log.cc:927] Time spent T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:14:17.529286  8211 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:17.539089  8206 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:17.546739  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:17.547005  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 335922us
I20260709 13:14:17.548942  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.549701  8205 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:17.452294 11750 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:17.918526  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.724650  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:17.557936  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 372940us
I20260709 13:14:17.660218  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.960836  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1779) took 1041 ms. Trace:
I20260709 13:14:17.626394  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.809571  8728 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:17.954766  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:17.668269  8101 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.021s	sys 0.029s
W20260709 13:14:17.564189  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 51462.4us
W20260709 13:14:17.964430  8208 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.009s	sys 0.024s
I20260709 13:14:17.695865  7959 catalog_manager.cc:5697] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 5 to 6, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 6 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:14:17.964879  8101 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.875s	user 0.023s	sys 0.034s
I20260709 13:14:17.557672  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:17.964980  8208 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.010s	sys 0.024s
I20260709 13:14:17.971050  8730 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
W20260709 13:14:17.821514 11954 consensus_meta.cc:287] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.584s	user 0.004s	sys 0.004s
I20260709 13:14:17.966480  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:17.821965 12086 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.001s	sys 0.014s
I20260709 13:14:17.973698  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:17.974090  8211 raft_consensus.cc:2410] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:14:17.974352  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 416756us
W20260709 13:14:17.954073 12014 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.015s	sys 0.011s
W20260709 13:14:17.954296 12023 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.004s	sys 0.007s
W20260709 13:14:17.974885  8210 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.008s	sys 0.008s
W20260709 13:14:17.978236  8210 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.009s	sys 0.009s
I20260709 13:14:17.781848 12013 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:17.621351 11773 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:17.980214  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:17.980553  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:17.983287  7959 catalog_manager.cc:5697] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:14:17.985577 12031 consensus_meta.cc:287] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.000s
I20260709 13:14:17.985744 12013 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:17.988512  8102 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:17.961342  8132 rpcz_store.cc:276] 0709 13:14:16.919740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:16.919946 (+   206us) service_pool.cc:224] Handling call
0709 13:14:16.920228 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:16.967992 (+ 47764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:16.968019 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:17.090894 (+122875us) log.cc:844] Serialized 26 byte log entry
0709 13:14:17.443525 (+352631us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:17.443595 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.443686 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:17.786889 (+343203us) raft_consensus.cc:1685] finished
0709 13:14:17.786909 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:17.960779 (+173870us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115892,"raft.queue_time_us":23062,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":10890,"thread_start_us":305747,"threads_started":1,"wal-append.queue_time_us":641214}
I20260709 13:14:17.989279  8203 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:17.992106  8211 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:17.992949 12112 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:17.993359  8211 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:14:17.988571  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 1.04068e+06us
I20260709 13:14:17.995739  8101 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:17.975905 12106 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:14:17.996826 12112 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:17.996984  8101 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:17.999073 11750 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:17.986763 11857 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:18.000156 11857 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:18.000358  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.002277  8060 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:18.000356  8176 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:18.004523  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:17.998219  8101 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:17.979031 12014 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.015s	sys 0.011s
I20260709 13:14:17.986959  8027 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:18.003998  8138 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 11. Current term is 12. Ops: []
W20260709 13:14:17.977314 12086 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.045s	user 0.003s	sys 0.023s
W20260709 13:14:17.979382 12023 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.005s	sys 0.007s
I20260709 13:14:18.004012 11874 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:18.008706 11773 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:18.035671  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:18.035930 11874 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:18.036504 11874 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:18.036810  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:18.037504  8211 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:18.009714  8137 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:18.038468 11816 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:18.010604  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:18.039453  8062 raft_consensus.cc:2368] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9.
I20260709 13:14:18.040373  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.041512  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.011404  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.012146  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:17.998226 12106 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:18.012789  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.012899  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.006826 12031 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:18.014374  8056 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:14:18.015494  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.016644  8054 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:18.017745  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.020833  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:18.042644  8060 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:18.043663  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:17.979605 11954 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:18.046626  8209 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:18.052945 11738 consensus_queue.cc:1059] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:18.053537 11954 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:18.053891 12121 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:18.047291  8205 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:18.054355 12121 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:18.042904  8061 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:18.054908 12062 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:18.055038  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.049291  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.050228 11816 raft_consensus.cc:697] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:18.056540  8212 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:18.043344 12023 consensus_queue.cc:1243] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:18.042318  8062 raft_consensus.cc:2410] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:18.002197 11857 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:18.027134  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:18.028266 12013 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:18.030588  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:18.058808  8206 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:18.032994 11981 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:18.056969 11816 consensus_queue.cc:237] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:18.060335  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:17.990846  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:18.061723  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:18.064358  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 73583.2us
I20260709 13:14:18.066041 12062 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:18.074724 12062 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.075053 12062 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:18.079222  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 81056.7us
I20260709 13:14:18.081995  8104 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:18.083016  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:18.085132  8175 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.000s	sys 0.032s
W20260709 13:14:18.085465  8175 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.000s	sys 0.032s
I20260709 13:14:18.086201  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.088135  8104 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:18.088985 12112 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:18.089474 12112 raft_consensus.cc:697] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:18.085757  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 670861us
I20260709 13:14:18.102042  8104 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:18.155045  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:18.087409 11954 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:18.163024  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:18.164384  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 73571.4us
I20260709 13:14:18.166087  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:18.156364 11750 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:18.178884  8175 leader_election.cc:400] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9.
I20260709 13:14:18.180974  8696 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:18.182713  8696 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:18.161481 11954 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.186671 11954 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:18.186173 12112 consensus_queue.cc:237] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:18.178911 12023 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:18.219542  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.228754  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.221323 12023 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.192822 11750 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:18.194653 11981 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:18.177065 12106 consensus_queue.cc:1059] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:14:18.200238  8730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:50008
I20260709 13:14:18.246867  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.235435 12023 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:18.294162 12127 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
W20260709 13:14:18.349323  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 264326us
W20260709 13:14:18.350433  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 161297us
W20260709 13:14:18.350984  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 249014us
I20260709 13:14:18.374265  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.375382  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:18.237272  8056 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:14:18.376423  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.376233 11750 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:18.376657  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.376963  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.377990  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.200563 12070 consensus_queue.cc:1059] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:14:18.235996 11967 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:18.380264  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:18.380344  8209 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:18.381990  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.476586  8101 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:18.403508 12123 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.169s	user 0.006s	sys 0.000s
I20260709 13:14:18.247377 12124 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:18.296756 12119 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:18.476018 11773 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.489531 11773 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:18.491834  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.492084 11773 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:18.479310 12106 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:18.492976 12106 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:18.483343 12146 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:18.494941 12146 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:18.483201 12086 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:18.495396 12146 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:18.495563 12086 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:18.483423 12124 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.496511 12086 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:18.497219 12124 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:18.497810 12086 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:18.498946  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2801) took 1199 ms. Trace:
W20260709 13:14:18.407742 12118 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.245s	user 0.000s	sys 0.006s
W20260709 13:14:18.408543 12117 log.cc:927] Time spent T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.175s	user 0.002s	sys 0.005s
I20260709 13:14:18.502698  8723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 623) took 1337 ms. Trace:
W20260709 13:14:18.409731 12126 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.234s	user 0.000s	sys 0.007s
I20260709 13:14:18.503249  8721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 676) took 1108 ms. Trace:
I20260709 13:14:18.476100 11967 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:18.411792 12013 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:18.411932 12031 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:18.412418 11981 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:14:18.412636 12122 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.177s	user 0.001s	sys 0.006s
I20260709 13:14:18.415087 11874 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:18.417081  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:18.416815 12121 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:18.427276  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 396980us
I20260709 13:14:18.431634  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:18.515419  8206 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:14:18.246191  8730 raft_consensus.cc:606] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:18.451936  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 216024us
W20260709 13:14:18.454367  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50784.6us
I20260709 13:14:18.464388  8061 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:18.458323  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:18.464252  8058 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:18.476804  8054 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:18.405340 12135 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:18.406972 11954 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:18.524262  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2789) took 1313 ms. Trace:
W20260709 13:14:18.478302  8727 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.009s	sys 0.006s
I20260709 13:14:18.484627  8055 raft_consensus.cc:2410] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:14:18.411010 12062 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:18.505818  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:18.499369  8210 rpcz_store.cc:276] 0709 13:14:17.299339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.304927 (+  5588us) service_pool.cc:224] Handling call
0709 13:14:17.305285 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:17.340719 (+ 35434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.340764 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:17.359588 (+ 18824us) log.cc:844] Serialized 26 byte log entry
0709 13:14:17.978618 (+619030us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:17.978656 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.978721 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.394364 (+415643us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:18.498608 (+104244us) raft_consensus.cc:1685] finished
0709 13:14:18.498637 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.498877 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17936,"raft.queue_time_us":43838,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":9964,"spinlock_wait_cycles":2410752,"thread_start_us":585881,"threads_started":1,"wal-append.queue_time_us":786659}
I20260709 13:14:18.507077  8101 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:18.503638  8721 rpcz_store.cc:276] 0709 13:14:17.394511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.394612 (+   101us) service_pool.cc:224] Handling call
0709 13:14:17.413395 (+ 18783us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:17.446152 (+ 32757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.446199 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:17.545655 (+ 99456us) log.cc:844] Serialized 26 byte log entry
0709 13:14:17.970775 (+425120us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:17.970829 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.970908 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.503013 (+532105us) raft_consensus.cc:1685] finished
0709 13:14:18.503031 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.503186 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6430,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":424621,"threads_started":1,"wal-append.queue_time_us":618357}
I20260709 13:14:18.503094  8723 rpcz_store.cc:276] 0709 13:14:17.165099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.166209 (+  1110us) service_pool.cc:224] Handling call
0709 13:14:17.166546 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:17.190123 (+ 23577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.190164 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:17.221326 (+ 31162us) log.cc:844] Serialized 26 byte log entry
0709 13:14:17.511736 (+290410us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:17.511797 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.511892 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.502411 (+990519us) raft_consensus.cc:1685] finished
0709 13:14:18.502435 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.502609 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9857,"raft.queue_time_us":40598,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":5675,"thread_start_us":290003,"threads_started":1,"wal-append.queue_time_us":841223}
I20260709 13:14:18.517249  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2806) took 1151 ms. Trace:
I20260709 13:14:18.527933  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:18.532562  8208 rpcz_store.cc:276] 0709 13:14:17.365935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.381445 (+ 15510us) service_pool.cc:224] Handling call
0709 13:14:17.403989 (+ 22544us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:17.429516 (+ 25527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.429553 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:17.430039 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 13:14:17.970375 (+540336us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:17.970452 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.970554 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.409059 (+438505us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:18.512771 (+103712us) raft_consensus.cc:1685] finished
0709 13:14:18.512799 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.517169 (+  4370us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1557,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":3458944,"thread_start_us":539858,"threads_started":1,"wal-append.queue_time_us":732575}
I20260709 13:14:18.534492  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:18.535429  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 77205.4us
I20260709 13:14:18.529651  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 672) took 1309 ms. Trace:
I20260709 13:14:18.540194  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:18.540854  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:18.543128  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:18.544034 11954 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:18.522866 12135 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.524747  8207 rpcz_store.cc:276] 0709 13:14:17.210589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.210733 (+   144us) service_pool.cc:224] Handling call
0709 13:14:17.213938 (+  3205us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:17.214299 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.214327 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:17.214764 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 13:14:17.687900 (+473136us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:17.687960 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.688052 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.524014 (+835962us) raft_consensus.cc:1685] finished
0709 13:14:18.524043 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.524198 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":979,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":9532,"thread_start_us":472726,"threads_started":1,"wal-append.queue_time_us":960578}
I20260709 13:14:18.544260  8210 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:18.546005 12121 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:18.530119 12013 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:18.536284  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:18.587191  8129 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:18.587945  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:14:18.588227  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 52053.6us
I20260709 13:14:18.587080  8204 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:18.590152  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:18.590819  8056 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:18.576874  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:18.586601  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:18.592314  8203 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:18.592621  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:18.593235  8137 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:14:18.529013 12064 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:18.594105  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:18.562566  8727 rpcz_store.cc:276] 0709 13:14:17.220154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.220263 (+   109us) service_pool.cc:224] Handling call
0709 13:14:17.229453 (+  9190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:17.734298 (+504845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.734329 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:17.779708 (+ 45379us) log.cc:844] Serialized 26 byte log entry
0709 13:14:18.526389 (+746681us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:18.526443 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:18.526634 (+   191us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.526694 (+    60us) raft_consensus.cc:1685] finished
0709 13:14:18.526712 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.529580 (+  2868us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1455,"raft.queue_time_us":42942,"raft.run_cpu_time_us":68,"raft.run_wall_time_us":69,"thread_start_us":999427,"threads_started":2,"wal-append.queue_time_us":540194}
W20260709 13:14:18.602474  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.605078  8204 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:18.686254  8209 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:14:18.690739  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 98855.5us
I20260709 13:14:18.610692  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:18.586633 12135 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:18.706588  8726 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:18.709311  8206 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:18.605193  8057 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:18.594815 12140 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:14:18.710893  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 203877us
I20260709 13:14:18.719844  8101 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:18.721112  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 59070.8us
W20260709 13:14:18.737113  8026 leader_election.cc:336] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.920s (SENT)
I20260709 13:14:18.754056 12158 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:18.627897 12125 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:18.684346 12014 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.735942  8061 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:18.600800 11967 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:18.756197  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:18.688632 11874 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:18.687937 12062 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:18.741186  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:18.757936  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 162418us
I20260709 13:14:18.709101 12031 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:18.760318 11738 consensus_queue.cc:1059] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:14:18.777056 12130 raft_consensus.cc:3055] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:18.777508 12130 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:18.778630 12130 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:18.780134 12130 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:18.780929 12133 log.cc:927] Time spent T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.355s	user 0.001s	sys 0.000s
I20260709 13:14:18.757117 12125 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.783011 12125 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:18.783473 12131 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.355s	user 0.001s	sys 0.000s
I20260709 13:14:18.788022  8208 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:14:18.788065 12024 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:18.788379  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:18.788640 12024 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.789041 12024 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:18.790550 11967 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:18.792337 12064 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:18.792809 12064 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:18.662977 11954 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:18.754647  8026 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.982s (SENT) [suppressed 36 similar messages]
I20260709 13:14:18.760456  8212 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:14:18.806388  8212 pending_rounds.cc:85] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 3
I20260709 13:14:18.806639  8212 pending_rounds.cc:107] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 8.4
W20260709 13:14:18.806541  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 51923.1us
I20260709 13:14:18.806923  8212 raft_consensus.cc:2889] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:18.661473 12106 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:18.775014 12013 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:18.755190 12140 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:18.756551 11773 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:14:18.830554 12140 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:18.863672  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 278247us
W20260709 13:14:18.863010  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 143272us
I20260709 13:14:18.873024 12023 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:18.771988 12014 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:18.867336 12146 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:18.881726  8205 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:14:18.866061  8102 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:18.828253 11981 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:14:18.898041 12124 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:18.904500 12136 raft_consensus.cc:3055] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:14:18.905164  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 183296us
I20260709 13:14:18.911550 12158 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:18.916126 12158 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:18.916741  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:18.918002 12013 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:18.919273 12137 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:18.919744  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:18.921231  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:18.921671  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 55945.4us
I20260709 13:14:18.925694  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:18.926805  8102 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:18.931568 12160 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:18.920470 12159 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.936048 12023 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:18.938230 12154 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:18.941224 12014 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:18.942030 12158 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:18.938329 12124 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:18.947585  8724 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:18.953119  8101 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:18.954012 12158 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:18.954427 12158 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.920365 12155 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:18.954711 12023 raft_consensus.cc:3055] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:18.955085 12155 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:18.954839 12158 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:18.955603 12155 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:18.955422 12023 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:18.957001 12023 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:18.911731 12136 raft_consensus.cc:740] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:18.933144 12150 log.cc:927] Time spent T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Append to log took a long time: real 0.391s	user 0.001s	sys 0.002s
I20260709 13:14:18.966948 12136 consensus_queue.cc:260] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:18.969525 12140 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:18.971017  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:18.931015 11954 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:18.932489 12159 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:18.974241  8203 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
W20260709 13:14:18.974687  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:18.985572  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 65983.6us
I20260709 13:14:18.962249  8101 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:18.980674  8206 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
W20260709 13:14:18.974041  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.981899 12156 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:18.980108 12164 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:14:18.937285 12013 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:14:18.982496  7946 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b578d88c1c1f4ca58b7291fed4909f09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 965e59f272f143449b18f65db4e61d98 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:18.984514 12031 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:18.985563 11954 raft_consensus.cc:697] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:19.005993 12031 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:19.006927 11954 consensus_queue.cc:237] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:19.002779 12112 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.009s	sys 0.009s
I20260709 13:14:19.012163  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:18.998307  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:14:18.999402  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 85197.8us
I20260709 13:14:19.002852 12156 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:19.003455 12159 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:19.004208 12013 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:19.004930 11773 consensus_queue.cc:1059] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:18.996790  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:19.016369 12013 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:19.015409  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:19.018438 12166 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:19.018826 12166 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.019136 12166 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:19.021392 12164 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:19.022079 12164 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:19.024138 12013 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:19.024746 12164 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:19.026464 12013 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:19.030027 12041 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:18.981557 11874 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:19.034693  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 72509.4us
I20260709 13:14:19.036744  8101 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:19.037590 12159 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:19.039013  8101 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:19.041535  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 55094.6us
W20260709 13:14:19.016130 12112 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.009s	sys 0.009s
I20260709 13:14:19.013527 12154 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:19.026685 12166 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:19.061347  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 46937.7us
I20260709 13:14:19.035040 12106 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:19.064977  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:19.077245  8029 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:19.053921 12013 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:18.997861 12125 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:19.045488 11874 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:19.132882  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:19.133915  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 73216.1us
I20260709 13:14:19.144857  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:19.137001  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:19.065627 12159 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:19.153287  8029 leader_election.cc:400] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10.
W20260709 13:14:19.118690 12131 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.064s	user 0.002s	sys 0.000s
W20260709 13:14:19.163206  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 89303.1us
I20260709 13:14:19.119652 12013 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:19.168484  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:19.169209  8132 raft_consensus.cc:2376] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12. Re-sending same reply.
I20260709 13:14:19.154049 11750 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:19.173697  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:19.132779 12173 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.174872  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:19.175854  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:19.178776  8205 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 12: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:19.176474  8131 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:19.154851 12140 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:19.243103  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:19.253887  8138 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:19.152288 12164 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 13:14:19.173799 12013 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:19.264909  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:19.271479  8205 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
W20260709 13:14:19.272104  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 198525us
I20260709 13:14:19.273387  8137 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:19.275907  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:19.276631  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:19.275936  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:19.277793  8138 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 6.
I20260709 13:14:19.279040  8211 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:19.280166  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:19.281841  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 146341us
I20260709 13:14:19.283454  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:19.285331  8057 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:19.290803 12159 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:19.292232 12159 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:19.270794 11874 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:19.307193  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:19.298908 12024 consensus_meta.cc:287] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.509s	user 0.006s	sys 0.004s
I20260709 13:14:19.322434  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:19.271988 12064 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:19.326109  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 162031us
I20260709 13:14:19.329850  8101 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:19.319231 11750 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:19.333168  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:19.327477 12024 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:19.333760  8133 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:19.334275  8723 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 12: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:19.334950 12180 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:19.335415 12180 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.335721 12180 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:19.339236 11959 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
W20260709 13:14:19.340751 12175 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 13:14:19.342720  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:19.343729  8131 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:19.344527  8133 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
W20260709 13:14:19.351438 12174 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:14:19.376192  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.410429  8729 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
W20260709 13:14:19.412266  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 95048.1us
W20260709 13:14:19.421406  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 240059us
W20260709 13:14:19.421124  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 248337us
I20260709 13:14:19.269451 12173 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:19.325294 12154 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:19.332607 11750 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:19.455451 12162 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:14:19.492971  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 214842us
I20260709 13:14:19.549896  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:19.585826  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:19.545102 11959 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:19.587246  8133 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:19.587496 11959 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:19.589663 11959 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:19.607930  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:19.653777  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:19.655016  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 108267us
I20260709 13:14:19.454823 12115 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.131s
W20260709 13:14:19.671003  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 388317us
W20260709 13:14:19.665993  8104 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.026s	sys 0.087s
I20260709 13:14:19.543880 12162 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:19.674917  8104 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.026s	sys 0.087s
W20260709 13:14:19.678447  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 348647us
W20260709 13:14:19.688308  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 551371us
I20260709 13:14:19.687404  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:19.689066  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.688777  8101 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:19.657372 12140 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:19.716674 12031 consensus_meta.cc:287] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.710s	user 0.008s	sys 0.000s
I20260709 13:14:19.640906 12156 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:19.725355 12164 consensus_queue.cc:1059] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:19.673925 12154 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:19.690901 12162 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:19.745640  8175 connection.cc:256] RPC call timeout handler was delayed by -1.5238583190002828s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2735s for a response
W20260709 13:14:19.746929  8175 connection.cc:256] RPC call timeout handler was delayed by -1.0510724460002621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
I20260709 13:14:19.704129 12186 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:19.759140  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 70467.9us
I20260709 13:14:19.746063 12119 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:19.750109 12125 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:19.789391  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 91136.9us
I20260709 13:14:19.743846 12140 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:19.743528 12031 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:19.761657 12154 raft_consensus.cc:697] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:19.830793 12154 consensus_queue.cc:237] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:19.768150 12186 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:19.838864  8729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 776) took 1309 ms. Trace:
W20260709 13:14:19.826625  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 122788us
I20260709 13:14:19.752082 12188 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:14:19.827005  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 43663.9us
W20260709 13:14:19.828572  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 68408us
I20260709 13:14:19.858204  8729 rpcz_store.cc:276] 0709 13:14:18.529764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.970673 (+440909us) service_pool.cc:224] Handling call
0709 13:14:19.838713 (+868040us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b5400582854. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:19.838812 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":475431680}
I20260709 13:14:19.835970  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2812) took 1373 ms. Trace:
I20260709 13:14:19.851014  8137 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:19.896827 12176 consensus_queue.cc:1059] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:14:19.898303  8210 rpcz_store.cc:276] 0709 13:14:18.462629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.654825 (+192196us) service_pool.cc:224] Handling call
0709 13:14:19.173351 (+518526us) spinlock_profiling.cc:243] Waited 8.63 ms on lock 0x7b5800168398. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:14:19.173516 (+   165us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:19.419842 (+246326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:19.419873 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:19.420325 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 13:14:19.553958 (+133633us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:19.554009 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.554101 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:19.834154 (+280053us) raft_consensus.cc:1685] finished
0709 13:14:19.834172 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:19.835906 (+  1734us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87133,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"spinlock_wait_cycles":19848576,"thread_start_us":306785,"threads_started":2,"wal-append.queue_time_us":361698}
I20260709 13:14:19.853077 12024 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:19.841574 11857 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:14:19.905295 11857 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:19.795902 12119 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:19.834368 12187 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:19.907701 12187 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:19.853458 12158 consensus_meta.cc:287] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.897s	user 0.007s	sys 0.000s
I20260709 13:14:19.910753 12158 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:19.911335 12155 consensus_meta.cc:287] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.955s	user 0.006s	sys 0.005s
I20260709 13:14:19.849992 12181 raft_consensus.cc:993] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
W20260709 13:14:19.835616 12197 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:14:19.912060  8138 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: [9.4-9.4]
W20260709 13:14:19.920068  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:19.849638 12180 consensus_meta.cc:287] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.513s	user 0.006s	sys 0.004s
I20260709 13:14:19.923362  8026 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:19.845229 12135 consensus_meta.cc:287] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 1.130s	user 0.001s	sys 0.008s
I20260709 13:14:19.921306  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:19.945369  8057 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:19.840348 12114 raft_consensus.cc:3055] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:19.845522 12204 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:19.843010 12031 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:19.935535  8132 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.016s	sys 0.033s
W20260709 13:14:19.997680  8132 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.018s	sys 0.035s
W20260709 13:14:19.987033  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.846138 12186 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:19.968973  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:19.968621  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.003922  8056 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:19.916973 12188 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:20.008600 12188 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:19.913245 12158 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:20.015357 12173 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:19.913740 12181 raft_consensus.cc:1081] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
W20260709 13:14:20.033354  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 110088us
W20260709 13:14:20.024326  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 121848us
I20260709 13:14:20.036490  8058 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:14:19.898821 12130 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.039291  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:20.040337  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:20.043493  8056 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:19.956643 12114 raft_consensus.cc:740] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:20.023350 12171 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.905s	user 0.009s	sys 0.000s
I20260709 13:14:19.967630 12204 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:20.046967  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.045047  8057 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:19.967686 12187 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:20.056931  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.911806 12155 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:20.050653  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:20.017431 12140 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:20.058369  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:20.024149 12176 raft_consensus.cc:3055] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:14:20.064260 12013 consensus_meta.cc:287] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.788s	user 0.011s	sys 0.000s
I20260709 13:14:20.078815  8137 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:14:20.072631  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:19.956905 12180 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:19.957460 12135 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:20.082309  8129 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:14:20.081566  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:20.080852  8057 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:20.094938 12204 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.095892  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:20.096925 12204 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:20.020334 12162 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.100467 12162 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:20.100996 12162 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:20.101560 12162 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:20.069322 12182 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:20.086583  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.104128  8063 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:20.088227  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.093029  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.067611 12176 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:20.069548 12206 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:14:20.113384 12176 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:20.114634  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:20.115597  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.072767 12187 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:20.107295  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.108664  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:20.104017  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.055173 12130 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.093612 12013 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.112401 12135 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:20.103677  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.110369  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:20.126691  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.127068  8061 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:20.116691  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.120605  8131 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:20.122027  8054 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 12: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:20.072551 12191 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:20.122027  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 330321us
I20260709 13:14:20.073311 12173 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:20.131699  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.131179  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.132915  8129 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:20.116508 12187 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.131779  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.134138  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.129446  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.135726  8133 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:14:20.135804  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 100742us
I20260709 13:14:20.136451  8055 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:20.147403  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.150821  8137 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:20.159674  8135 raft_consensus.cc:2393] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:20.103426 12140 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:14:20.179821  8134 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:14:20.121326 12031 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:14:20.187413  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.188277  8059 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:20.189939  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.198690  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.182071 12176 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:20.237186  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.210695  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:20.225381  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 78266.8us
W20260709 13:14:20.235131  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 60313.2us
I20260709 13:14:20.239488  8029 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:20.121435 12182 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:20.255088  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 65829.3us
I20260709 13:14:20.169953 12173 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:20.261199  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.117547 12114 consensus_queue.cc:260] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.177259 12124 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:14:20.274403  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.278643  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:20.284629  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45747.8us
I20260709 13:14:20.284881 12229 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:20.285398 12229 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:20.285789 12229 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:20.286396  8026 leader_election.cc:336] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.685s (SENT)
I20260709 13:14:20.276768  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:20.286916  8026 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:20.287773  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 48391.7us
I20260709 13:14:20.288863  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:20.289299  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:20.290179  8059 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:20.290587  8133 raft_consensus.cc:2410] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:14:20.295199  8175 connection.cc:256] RPC call timeout handler was delayed by -1.1887076849998266s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2187s for a response
I20260709 13:14:20.300722  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.175768 12191 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.257980 12169 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.095s
I20260709 13:14:20.310001 12196 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:20.311787  8135 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:20.314846  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.299655 12178 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.195s
I20260709 13:14:20.321669  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:20.326923 12208 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:14:20.328518  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1381) took 1057 ms. Trace:
I20260709 13:14:20.328850  8132 rpcz_store.cc:276] 0709 13:14:19.270944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.271148 (+   204us) service_pool.cc:224] Handling call
0709 13:14:19.284471 (+ 13323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:19.292342 (+  7871us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:19.292600 (+   258us) log.cc:844] Serialized 14 byte log entry
0709 13:14:19.998617 (+706017us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:20.011693 (+ 13076us) log.cc:844] Serialized 26 byte log entry
0709 13:14:20.011837 (+   144us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:20.011879 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.011968 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:20.328321 (+316353us) raft_consensus.cc:1685] finished
0709 13:14:20.328339 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:20.328448 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9048,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":365239,"raft.run_cpu_time_us":576,"raft.run_wall_time_us":113669,"thread_start_us":705603,"threads_started":1,"wal-append.queue_time_us":833038}
I20260709 13:14:20.317627 12013 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:20.313910 12155 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:20.313771 12166 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:20.316955 12215 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:14:20.331764  8061 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:20.332314  8059 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:20.313979 12180 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:20.335176  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.338467 12191 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:20.325030 12130 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:20.341735  8056 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:20.334332 12187 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:20.345358 12212 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
I20260709 13:14:20.346184 12182 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:20.351671  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.343343 12173 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:20.356331  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 74689.9us
I20260709 13:14:20.358668  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.361086  8724 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:20.363152  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:20.365403 12116 raft_consensus.cc:3055] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:20.365739  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:20.365975 12116 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:20.366489 12231 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:20.366981 12231 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:20.367520 12231 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:20.365468 12218 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:20.376435  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.376745  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:20.378153  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.381781  8058 raft_consensus.cc:2393] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:20.385035 12119 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:20.385694 12146 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:20.395637  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:20.401567  8062 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:14:20.403048  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.403863  8060 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:20.411587  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:20.412325  8136 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:14:20.417172  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:20.418866  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:20.419590  8059 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:14:20.420744  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.425226  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:20.431476  8728 raft_consensus.cc:2376] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10. Re-sending same reply.
I20260709 13:14:20.432823  8101 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:20.433821  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:20.433714  8135 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:20.434906  8101 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:20.435954  8101 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:20.439620  8102 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:20.476493  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:20.479188  8061 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:20.416252 12151 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:14:20.481177  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 45294.7us
I20260709 13:14:20.482924  8101 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:20.475468 12062 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:20.485589  8059 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:20.484972  8101 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:20.422219 12146 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:20.436746 12187 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:20.366906 12116 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.487274  8101 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:20.426755 12024 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.486308 12062 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.436995 12191 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:20.484969 11954 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:20.484704 12135 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:20.501694  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 213182us
I20260709 13:14:20.484436 12151 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:20.426993 12086 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:20.511556  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:20.488782 12164 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:20.427408 12173 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 13:14:20.515132  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.484855 12214 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:20.515569  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:20.503136 11750 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:20.507125 12062 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:20.540189  8029 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:20.542176 12228 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:20.517767 12151 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:20.510897 11954 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:20.497567 12146 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:20.584425  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:20.585060  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 81406.3us
I20260709 13:14:20.585297  8056 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:20.508848 12024 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:20.585681  8026 leader_election.cc:336] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.734s (SENT)
I20260709 13:14:20.589110  8026 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:20.584609 12146 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:20.593429 12236 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:14:20.598019 12024 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:20.598703 12024 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:20.599280 12024 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.601827 12024 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:20.543362 12164 raft_consensus.cc:697] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:20.584373 11750 raft_consensus.cc:697] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:20.584909 11954 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:20.573696 12214 raft_consensus.cc:697] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:20.590317 12239 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:20.630152  8211 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:20.632411  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:20.634176  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:20.634904  8208 raft_consensus.cc:2410] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:20.552385 12086 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.611014 12211 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:20.643096  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.644191  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.645138  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:20.630877  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:20.553767 12238 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:14:20.647938  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:20.651052  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:20.651580  8211 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:20.653885 11750 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:20.654616 12086 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:20.655145  8730 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
W20260709 13:14:20.635504 12188 consensus_meta.cc:287] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.626s	user 0.017s	sys 0.000s
I20260709 13:14:20.661499  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:20.662528  8208 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:14:20.662166 12229 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.632325 12246 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.664100  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.622844 12228 consensus_queue.cc:1059] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:14:20.678023  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:20.682492  8210 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:20.637112 12239 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:20.690717 12239 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:20.679711  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:20.703298  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 406937us
I20260709 13:14:20.675818 12188 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.689042 12228 raft_consensus.cc:3055] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:20.732178 12228 raft_consensus.cc:740] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:20.733050 12228 consensus_queue.cc:260] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.733811 12188 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:20.687582 12214 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.694257  8205 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 12: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:20.683594 12086 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:20.723917  8061 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:20.623359 12240 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:14:20.649529 12233 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:14:20.723845  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:20.724630 12241 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:14:20.725476  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:20.726204  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.723652 12023 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:20.727645 12062 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.729624 12229 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:20.637045 12130 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:14:20.726213  8210 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:20.749845  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:20.730242 12194 raft_consensus.cc:3055] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:20.752920  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.801519  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.802243  8137 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:20.751252  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:20.746779  8723 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 777) took 2195 ms (client timeout 2204 ms). Trace:
W20260709 13:14:20.771684  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.749375  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.750433  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:20.795678  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 181904us
I20260709 13:14:20.750433  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.760855  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.741976  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:20.804539  8723 rpcz_store.cc:269] 0709 13:14:18.551219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.788133 (+236914us) service_pool.cc:224] Handling call
0709 13:14:20.746527 (+1958394us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b5400505fd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:20.746688 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1253560704}
I20260709 13:14:20.799234  8054 raft_consensus.cc:2393] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:20.759073  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:20.811365  8135 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:20.739688  8056 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:20.806901  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:20.812125  8208 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:20.811193  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 69301.1us
I20260709 13:14:20.805855  8130 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:20.814762  8211 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:20.816383  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:20.842728  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:20.817124  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.734239 12228 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:20.755016 12136 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:20.830615  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:20.849159  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.850095  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:20.743739 12135 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:20.737298 12166 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.737131 12155 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:20.847848  8129 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 12: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:20.757057 12064 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:20.852022  8137 raft_consensus.cc:2393] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 11: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:20.860491  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:20.816526  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:20.800782 12194 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:20.812840 12169 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:20.816394 12086 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:20.863040  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.864462  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:20.863040  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 48914.8us
I20260709 13:14:20.865017  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:20.863967  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:20.868701  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:20.814145 12238 consensus_queue.cc:1059] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:14:20.871497  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:20.871888  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:20.872560  8204 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:20.873581  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:20.874755  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:20.875344  8137 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:20.875931 11954 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:20.875046 12216 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:14:20.685150 12246 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.880872  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:20.891036  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.891227  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:20.891827  8136 raft_consensus.cc:2410] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:14:20.868083 12155 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.903082  8204 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:20.906404  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 670) took 1764 ms. Trace:
I20260709 13:14:20.903159  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.918171  8203 rpcz_store.cc:276] 0709 13:14:19.142036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.145234 (+  3198us) service_pool.cc:224] Handling call
0709 13:14:19.165571 (+ 20337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:19.421593 (+256022us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:19.421772 (+   179us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:19.711096 (+289324us) log.cc:844] Serialized 26 byte log entry
0709 13:14:20.000168 (+289072us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:20.000209 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.906002 (+905793us) spinlock_profiling.cc:243] Waited 6.3 ms on lock 0x7b54002c8b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:20.906051 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:20.906100 (+    49us) raft_consensus.cc:1685] finished
0709 13:14:20.906116 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:20.906330 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2373,"raft.queue_time_us":94827,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":107,"spinlock_wait_cycles":14490880,"thread_start_us":490518,"threads_started":2,"wal-append.queue_time_us":344001}
I20260709 13:14:20.908352  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:20.924341  8137 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
W20260709 13:14:20.924402  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 257546us
W20260709 13:14:20.927645  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.927500  8208 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:20.954223  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.955166  8057 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:20.744582 12164 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:20.873329 12250 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:20.865559 12233 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:14:20.888665 12124 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:20.958889  8204 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:14:20.901635 12176 consensus_meta.cc:287] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.001s
I20260709 13:14:20.961607  8212 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:20.962188  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.967042 12202 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.902875 12169 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:20.978271 12178 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.978485 12169 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:20.904949 12014 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:14:20.978915 12231 consensus_meta.cc:287] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.002s
I20260709 13:14:20.979352 12231 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.979879  8205 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:20.980530  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:20.981263  8723 raft_consensus.cc:2393] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:20.981297 12231 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:20.862522 12166 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:20.970106 12124 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:20.982353 12124 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:20.982713  8132 raft_consensus.cc:2410] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:14:20.983386 12166 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:20.969956  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:20.984670 12151 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:20.985486 12252 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:14:20.985836 12213 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:20.986250 12213 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:20.895654 12086 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:20.982038 12246 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:20.987182  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:20.978883 12169 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:20.989244  8203 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:20.990188  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:20.990715  8203 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:20.974532  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:20.991449  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:20.991887  8212 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:20.992094  8131 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:20.992767  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:20.978854 12178 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:20.993244  8208 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:14:20.993613  8208 pending_rounds.cc:85] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 5
I20260709 13:14:20.993892  8208 pending_rounds.cc:107] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:14:20.994233  8208 raft_consensus.cc:2889] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:20.905511 11954 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:20.900004 12062 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:20.988193  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:20.993847  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:20.986591 12213 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:20.977936 12155 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:20.859305 12135 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:21.001749  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:20.968446 12219 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.023s	sys 0.032s
I20260709 13:14:21.016707  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:21.016812  8059 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
W20260709 13:14:21.017395  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 212186us
I20260709 13:14:21.018250  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:20.903020 12146 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:21.007833  8212 raft_consensus.cc:2393] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 7: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:21.000702  8203 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:21.029979  8131 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:21.029109  8029 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:20.899085 12239 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:21.041855  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:21.045976  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:21.002897 12023 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:14:20.996306 12014 raft_consensus.cc:697] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:21.043783  8204 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:20.972190 12250 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:14:21.041433 12219 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.023s	sys 0.032s
I20260709 13:14:21.032137  8133 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:14:21.048368  8062 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:21.053284  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:21.054095  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:21.055373  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:21.057711  8104 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:21.044039  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 154191us
I20260709 13:14:20.894783 12194 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.060925  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:21.066900  8204 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:21.062229  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.065090  8104 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:21.065765  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:21.068656  8696 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:21.068780  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:21.072083  8026 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:21.075340  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:21.076488  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:21.078117  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:21.081319  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:21.082826 12255 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:14:21.083600 12254 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.084066 12254 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.086823 12166 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:21.087224 12166 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:21.087571 12166 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:21.088572 12086 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:21.088963 12086 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.089277 12086 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:21.094224  8026 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:21.094559  8204 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:21.002364 12202 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:21.069911 12062 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:21.073004 11954 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:14:21.110397  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 81352us
I20260709 13:14:21.086184 12254 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:21.105834  8204 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:21.093920 12106 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:21.111394  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:21.104730  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:21.119424  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:21.120153  8026 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:21.120857  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:21.125183  8203 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:21.097767  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:21.128561  8204 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:14:21.130040  8204 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:21.131255  8204 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:21.132483  8204 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:21.133687  8204 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:21.098796 12240 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:21.104743 12023 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:14:21.180493  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 82798.1us
W20260709 13:14:21.185351  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:21.217830  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 152864us
I20260709 13:14:21.225514  8204 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:55942
W20260709 13:14:21.226169  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 106071us
I20260709 13:14:21.227151  8026 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:21.113323 12135 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:21.277460  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 158092us
I20260709 13:14:21.276604  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:21.226233  8204 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:21.309302  8204 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:21.223978  8693 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:21.319679  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 95755.2us
I20260709 13:14:21.104755 12188 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.047s
W20260709 13:14:21.349642  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 281063us
I20260709 13:14:21.125748 12106 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.320937  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:21.327625 12257 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:14:21.328066  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.328421 12263 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.232s
W20260709 13:14:21.330024 12269 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
W20260709 13:14:21.331060 12260 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.232s	user 0.000s	sys 0.000s
I20260709 13:14:21.110934 12136 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.119s
I20260709 13:14:21.338529 12238 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:21.127224 12219 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:21.124205 12173 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.215s
I20260709 13:14:21.340091 12146 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:21.341610 12155 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:21.341478 12216 consensus_queue.cc:1059] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:14:21.343526  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:21.345044 12259 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.127s	user 0.000s	sys 0.002s
I20260709 13:14:21.344684 12265 consensus_queue.cc:1059] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:14:21.350544  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.350461  8696 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:21.351625  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:21.362874  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:21.374148  8204 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.230675 12262 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:14:21.385563  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 161226us
I20260709 13:14:21.349311 11954 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:21.362305 12106 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:21.361330 12254 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:21.381512 12257 raft_consensus.cc:697] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:21.393141  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:21.396201  8057 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:21.397891  8204 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:21.398609  8203 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 14. Ops: []
I20260709 13:14:21.399195  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:14:21.408766  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58397.9us
I20260709 13:14:21.408847  8723 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:21.409904  8723 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:21.411172  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:21.408766  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 87935.9us
I20260709 13:14:21.361002 12240 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:14:21.388567 12151 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:21.390100 12256 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:21.431811 12239 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:21.399442 12254 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:21.432555 12254 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:21.391992 12155 raft_consensus.cc:697] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:21.436676 12155 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:21.400169 12257 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.421764 12262 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:21.219056 12250 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.424671  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:21.422390  8723 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.447933  8207 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:21.448334  8723 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:14:21.147370 12014 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.445039 12267 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.421017 12246 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:21.438846 12121 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:21.456167  8204 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:21.399076 11954 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.422703 12238 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:21.461611  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:21.461853  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:21.471905  8207 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:21.413558 12166 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:21.455709 12262 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:21.492648  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.434123 12213 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.454267 12202 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:21.509619  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.530296 11954 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:21.483032 12264 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:21.530407 12219 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:21.511897 12277 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.512302 12238 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:21.531925  8027 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:21.460739 12267 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:21.561154  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 147847us
I20260709 13:14:21.561828  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:21.563189  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:21.556900 12219 raft_consensus.cc:3055] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:21.567399  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:14:21.568732  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 341655us
I20260709 13:14:21.567600 12219 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:21.572150 12219 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.573345 12219 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:21.575012 12254 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.575762 12231 raft_consensus.cc:3055] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:21.576124 12231 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:21.577028  8723 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:21.576951 12231 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:21.570894  8026 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:21.566821 11981 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:21.577035 12254 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:21.582154  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:21.582891  8726 raft_consensus.cc:2368] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14.
I20260709 13:14:21.583127 12213 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:21.583405 12106 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:21.585361 12106 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:21.586302 12166 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:21.589918  8207 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:21.579378  8176 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:21.592406  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.565501 12119 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:21.592839  8029 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:21.555383 12277 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:21.683763  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 90996.5us
I20260709 13:14:21.698220  8209 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:21.598042  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:21.598374  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:21.597909  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:21.692281  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 587) took 1205 ms. Trace:
I20260709 13:14:21.728483  8723 raft_consensus.cc:2376] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12. Re-sending same reply.
I20260709 13:14:21.729588  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:21.734753  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:21.729904  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 159087us
I20260709 13:14:21.735657  8207 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:21.577922 12231 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:21.597711  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:21.736456  8026 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:21.728888  8061 rpcz_store.cc:276] 0709 13:14:20.487407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.487610 (+   203us) service_pool.cc:224] Handling call
0709 13:14:20.487962 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:20.793326 (+305364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.692054 (+898728us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b540009c254. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:21.692225 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":520115072}
I20260709 13:14:21.737349  8027 leader_election.cc:400] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14.
I20260709 13:14:21.740779  8026 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:21.591611 12124 consensus_meta.cc:287] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.609s	user 0.010s	sys 0.000s
I20260709 13:14:21.567354 12280 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:21.748216  8721 raft_consensus.cc:2368] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11.
I20260709 13:14:21.749172 12240 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:21.749476 12263 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:14:21.749989 12216 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:21.750609  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:21.704134 12213 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:21.823168 12213 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.823648 12176 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:21.824007 12176 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:21.824798 12176 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.825261 12213 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:21.752530  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:21.825968 12176 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:21.826375  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:21.826642  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:21.752648  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:14:21.599653 12270 log.cc:927] Time spent T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:14:21.600231 12275 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 13:14:21.761859  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.587867 12135 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:21.764649  8026 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:21.849066  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:21.594544 12106 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:21.581807 11981 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:21.751096 12070 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:21.688541 12121 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:21.593844 12211 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:21.669296 12166 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:21.770301  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:21.683977 11857 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:14:21.695199 12169 consensus_meta.cc:287] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.706s	user 0.009s	sys 0.000s
I20260709 13:14:21.599829 12238 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.773734 12178 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:21.777025  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:21.780923  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:21.783862  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.784138  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:21.785946  8730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 782) took 1360 ms. Trace:
I20260709 13:14:21.691295 12262 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:21.787755  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:21.789300  8137 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:21.854027 12013 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:21.683463 12285 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:21.803758  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:21.884938  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 81258.3us
I20260709 13:14:21.875396  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:21.888772  8054 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:21.804507 12023 raft_consensus.cc:3055] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:21.806128 12229 raft_consensus.cc:3055] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:21.732651 12267 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:21.741901 12239 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.817526  8203 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:14:21.821661  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1826) took 1457 ms. Trace:
I20260709 13:14:21.894248  8132 rpcz_store.cc:276] 0709 13:14:20.364498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.364666 (+   168us) service_pool.cc:224] Handling call
0709 13:14:21.821463 (+1456797us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b5400042754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:21.821587 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1416833280}
I20260709 13:14:21.851879 11954 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:14:21.822327  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 581) took 2547 ms. Trace:
I20260709 13:14:21.738214 12214 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.835167  8696 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:21.848585  8102 leader_election.cc:400] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11.
I20260709 13:14:21.904084  8102 leader_election.cc:403] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:14:21.904233  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69158.3us
W20260709 13:14:21.851496  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 86954.4us
I20260709 13:14:21.905804  8026 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:21.857507  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 454509us
I20260709 13:14:21.858485  8725 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:21.861537  8721 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:21.855856  8104 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:21.907343  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:21.908455  8054 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
W20260709 13:14:21.926882  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.859233 12119 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:21.762969 12124 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:21.798678  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:21.871310  8131 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:21.949134  8206 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:21.858752 11981 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:21.881795  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:21.956884  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.876559  8729 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:21.894313 12023 raft_consensus.cc:740] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:21.858629 12211 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:21.821856 12277 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:21.899786  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:21.900790  8055 rpcz_store.cc:276] 0709 13:14:19.275269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.275438 (+   169us) service_pool.cc:224] Handling call
0709 13:14:19.275768 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:20.144488 (+868720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.822136 (+1677648us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b540028aad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:21.822277 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1978910976}
I20260709 13:14:21.854974 12160 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:14:21.859236 12136 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:14:21.972643  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1384) took 2688 ms. Trace:
I20260709 13:14:21.849021  8722 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:21.856992 12106 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:21.944137  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 114094us
I20260709 13:14:21.972901  8133 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:21.854909 12135 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:21.946398  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:21.977829  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:21.876833 12166 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:21.876787 11857 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:21.955718  8060 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:21.874523  8730 rpcz_store.cc:276] 0709 13:14:20.425186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.425332 (+   146us) service_pool.cc:224] Handling call
0709 13:14:21.785775 (+1360443us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540041e6d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:21.785890 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407219072}
I20260709 13:14:21.893517 12229 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:14:21.962885  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 107112us
W20260709 13:14:21.983227  8059 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1557) took 1334 ms (client timeout 1773 ms). Trace:
I20260709 13:14:21.964511  8696 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:21.966492  8026 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:21.983690  8059 rpcz_store.cc:269] 0709 13:14:20.644031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.801234 (+157203us) service_pool.cc:224] Handling call
0709 13:14:21.977491 (+1176257us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54003ca854. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:21.977595 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493276032}
I20260709 13:14:21.984836  8026 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:21.967100  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:21.861115 12070 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.944545 12121 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:21.948129 12183 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:21.870052 12169 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:21.948455 12281 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:21.973031  8138 rpcz_store.cc:276] 0709 13:14:19.284449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.284574 (+   125us) service_pool.cc:224] Handling call
0709 13:14:19.307857 (+ 23283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:19.918478 (+610621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.972407 (+2053929us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b54001a9d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:21.972592 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1388985728}
I20260709 13:14:21.954752 12013 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:21.980661  8131 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:14:21.908526 12213 consensus_queue.cc:1059] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:14:21.903402 12239 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:21.947386 12243 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
W20260709 13:14:21.992643  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.947395 12214 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:21.991086 12250 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:21.999150  8026 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:21.984917 12086 consensus_meta.cc:287] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.895s	user 0.012s	sys 0.000s
I20260709 13:14:22.020658  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:21.979130 12290 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:22.003362 12265 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:22.008352  8130 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:21.970649 12182 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:14:21.983348 12254 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:22.007575 12289 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:22.009979 12231 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:22.009322 12151 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11.
I20260709 13:14:22.028894  8693 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:22.023622 12285 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:22.053438  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 60729us
I20260709 13:14:22.003201 12136 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 13:14:22.054598  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 55772.4us
I20260709 13:14:22.083014  8026 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:22.077991 12159 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:22.081409 12224 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:22.009810 12267 consensus_queue.cc:1059] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:14:22.030473 12290 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:22.073917  8102 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:22.089005  8026 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:22.045372 12254 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:22.044101 12289 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:22.094292  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 65469.6us
I20260709 13:14:22.003052 12023 consensus_queue.cc:260] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:22.011441 12119 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:22.107826  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 140795us
I20260709 13:14:22.092871 12014 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:22.089897 12064 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:22.088389 12160 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:14:22.109766 12160 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.110321 12160 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:22.112656 12160 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:22.113221 12160 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:22.113617 12160 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:22.114037 12160 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:22.046129 12229 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:22.102599 12254 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:22.116685  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.096740 12231 raft_consensus.cc:2764] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:14:22.030049 12086 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:22.105861  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.092519 12182 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:22.104861  8026 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:22.090132 11954 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:22.091205 12277 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:22.129954  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:22.094975 12290 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:22.108062 12289 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:22.140249  8724 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:22.145315  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:22.122890 12064 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:22.031831 12211 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:22.154491  8129 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:22.131072 12231 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:22.156059  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:22.165905  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:22.054891 12166 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:22.159893  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 86077.2us
I20260709 13:14:22.160598  8101 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:22.123257 12240 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
W20260709 13:14:22.183992  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7503678670000227s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
I20260709 13:14:22.120802 12262 consensus_queue.cc:1059] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:14:22.191282  8101 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:22.200127  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7395678670000052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1529s for a response
W20260709 13:14:22.200608  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7287678669999877s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
I20260709 13:14:22.201591  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:22.170619  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.192782  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.193696  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:22.202948  8061 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:14:22.195827  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:22.203254  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.202827  8056 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:22.203897  8054 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:22.204039  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:22.204250  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.204685  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.139245 12182 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:22.205488  8054 raft_consensus.cc:2410] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:14:22.090138 11857 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:22.142216 12023 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:22.139953 12124 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:22.219944  8062 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:14:22.107081 12224 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:22.228521  8137 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:22.229642  8137 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:14:22.223387  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.146502 12214 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:22.136302 12113 consensus_queue.cc:1059] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:14:22.231731 12214 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:22.161125 12281 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:22.232220 12214 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:22.232733 12214 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:22.232817  8054 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:22.159303 12014 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:22.233999  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:22.234575  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:22.235715  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6459678669998539s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
W20260709 13:14:22.240491  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6279678670000521s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1653s for a response
W20260709 13:14:22.256286  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6045678670000143s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16790000000000002s for a response
W20260709 13:14:22.256753  8175 connection.cc:256] RPC call timeout handler was delayed by -1.5649678669999503s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1723s for a response
W20260709 13:14:22.257371  8175 connection.cc:256] RPC call timeout handler was delayed by -1.5199678669998775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1773s for a response
I20260709 13:14:22.219681 12211 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:22.261559  8056 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:22.239655  8130 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:22.138733 11954 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:22.138605 12277 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:22.303057  8175 connection.cc:256] RPC call timeout handler was delayed by -1.4893678670000554s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
I20260709 13:14:22.303860  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:22.256851 12286 log.cc:927] Time spent T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
W20260709 13:14:22.306053  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 157519us
I20260709 13:14:22.324477  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:22.199329 12276 consensus_queue.cc:1059] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.092s
I20260709 13:14:22.227000 12106 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:22.235719 12136 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:22.258409 12298 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:22.303450  8175 connection.cc:256] RPC call timeout handler was delayed by -1.4740678670000307s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1824s for a response
I20260709 13:14:22.234671 12135 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:22.345872  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.346534  8054 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
W20260709 13:14:22.340411  8175 connection.cc:256] RPC call timeout handler was delayed by -1.4434678669999812s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
W20260709 13:14:22.347815  8175 connection.cc:256] RPC call timeout handler was delayed by -1.2265678669998579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
I20260709 13:14:22.258450 11750 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:22.349632  8175 connection.cc:256] RPC call timeout handler was delayed by -1.1536678669999674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21800000000000003s for a response
W20260709 13:14:22.351543  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 150021us
I20260709 13:14:22.259536 12281 raft_consensus.cc:697] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:22.296921 12169 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:22.365360  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.401252  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.423040  8061 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:22.425956  8061 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:22.413614  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:22.394974  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:22.418875  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:22.437922  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.440864  8130 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:22.355216 12070 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:22.429941 12288 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.384s	user 0.000s	sys 0.001s
I20260709 13:14:22.419937 12086 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:22.483903  8212 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:22.484441  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:22.485769 12295 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.327s	user 0.001s	sys 0.000s
I20260709 13:14:22.346149 12064 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:22.488453  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.423707 12281 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:22.491995 12312 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:14:22.492136 11981 consensus_meta.cc:287] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.530s	user 0.004s	sys 0.004s
W20260709 13:14:22.511632  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 116696us
W20260709 13:14:22.492319 12176 consensus_meta.cc:287] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.664s	user 0.006s	sys 0.002s
I20260709 13:14:22.412719 12182 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:22.416788 12239 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:22.467144  8061 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:22.477118 12169 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:22.523850  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.532895  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:22.531975  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 68639.1us
I20260709 13:14:22.533604  8209 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:22.393392 11750 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:22.527686  8175 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:22.529750  8061 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:22.498236 12254 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:22.505182 12156 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:14:22.541986  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:22.543744  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:22.551739  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:22.555372  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:22.525116 11981 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:22.558868 11750 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:22.543668 12156 consensus_queue.cc:1243] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:14:22.494201 12224 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:22.569033  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:22.556931 12169 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:22.549662 12176 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:22.568542 12322 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:22.577225  8728 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:22.569339  8061 raft_consensus.cc:2410] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:14:22.578018  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:22.584034  8210 raft_consensus.cc:3055] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:22.584280  8130 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:22.584489  8210 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:22.595835  8130 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:14:22.610796  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:22.611526  8135 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:22.600395  8210 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:22.614225  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 70649.4us
W20260709 13:14:22.635354 12313 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:14:22.636319 12297 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:14:22.610826 12322 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:22.637750 12289 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:22.638963 12314 log.cc:927] Time spent T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:14:22.639880 12289 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:22.640573  8723 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 13:14:22.625870 12320 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:14:22.633719  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:22.647666  8210 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:22.667745 12322 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:22.668959  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:22.670399 12325 log.cc:927] Time spent T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:14:22.671134  8175 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:22.671739 12301 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:22.678862  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2835) took 1789 ms. Trace:
I20260709 13:14:22.679224  8208 rpcz_store.cc:276] 0709 13:14:20.889789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.930144 (+ 40355us) service_pool.cc:224] Handling call
0709 13:14:20.930433 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:20.994743 (+ 64310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.678639 (+1683896us) spinlock_profiling.cc:243] Waited 62.5 ms on lock 0x7b540019ebd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:22.678803 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143759616}
I20260709 13:14:22.646813 12254 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:22.684330  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:22.694860 12321 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:14:22.696434  8135 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
W20260709 13:14:22.697371 12326 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:14:22.700979  8130 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
W20260709 13:14:22.698968  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.679925 12183 consensus_meta.cc:287] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.677s	user 0.009s	sys 0.001s
I20260709 13:14:22.723166 12183 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:22.724159  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1829) took 2221 ms. Trace:
I20260709 13:14:22.724519  8134 rpcz_store.cc:276] 0709 13:14:20.503238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.503346 (+   108us) service_pool.cc:224] Handling call
0709 13:14:20.512227 (+  8881us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:20.538654 (+ 26427us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:20.538974 (+   320us) log.cc:844] Serialized 14 byte log entry
0709 13:14:20.624197 (+ 85223us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:20.624244 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:20.624288 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.973040 (+1348752us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b1800286240. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ef72a6 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419
0709 13:14:22.723965 (+750925us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b54001aa754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:22.724010 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:22.724110 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":891,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":97,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":7265,"spinlock_wait_cycles":63946368,"thread_start_us":84599,"threads_started":1,"wal-append.queue_time_us":113586,"wal-append.run_cpu_time_us":976,"wal-append.run_wall_time_us":1015842}
W20260709 13:14:22.726569 12290 consensus_meta.cc:287] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.572s	user 0.009s	sys 0.000s
I20260709 13:14:22.726953 12290 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:22.728309 12226 consensus_queue.cc:1059] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:22.710161 12115 raft_consensus.cc:3055] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:22.729000 12115 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:22.729384 12226 raft_consensus.cc:3055] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:22.729751 12226 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:22.729959 12115 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:22.715086 12116 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:22.675045 12169 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:14:22.717437  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.735860  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.679522 12301 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:22.721256  8132 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:22.728572  8129 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:22.737974  8212 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:22.738762  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2850) took 1007 ms. Trace:
I20260709 13:14:22.739228  8212 rpcz_store.cc:276] 0709 13:14:21.731190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:22.471378 (+740188us) service_pool.cc:224] Handling call
0709 13:14:22.483786 (+ 12408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:22.738456 (+254670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.738697 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:22.705008  8026 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:22.741225  8026 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:22.730976 12115 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:22.742197  8210 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 4. (term mismatch)
I20260709 13:14:22.745007  8210 pending_rounds.cc:85] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 3
I20260709 13:14:22.745393  8210 pending_rounds.cc:107] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 9.4
I20260709 13:14:22.746479  8210 raft_consensus.cc:2889] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:22.770130  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.771091 12013 consensus_queue.cc:1059] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:14:22.743543 12214 consensus_meta.cc:287] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.000s
I20260709 13:14:22.774977 12214 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:22.777621 12214 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:22.779098 12160 consensus_meta.cc:287] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.655s	user 0.009s	sys 0.001s
I20260709 13:14:22.779656 12160 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:22.781060  8137 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:22.784382  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2852) took 1032 ms. Trace:
I20260709 13:14:22.784693  8210 rpcz_store.cc:276] 0709 13:14:21.752196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:22.582382 (+830186us) service_pool.cc:224] Handling call
0709 13:14:22.582724 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:22.784199 (+201475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.784343 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:22.786928  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 102667us
I20260709 13:14:22.734890 12166 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:22.760752 12216 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:22.748517 11857 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:22.794123  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:22.796226 12301 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:22.756407 12290 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:22.799292  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:22.801633 11920 raft_consensus.cc:3055] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:22.802016 11920 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:22.802452 12160 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:22.802759 11920 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:22.803917 11920 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:22.804286  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.804873  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.806479 12239 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.806948 12239 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:22.807938  8722 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:22.808458 12243 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.808959 12239 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:22.808892 12243 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:22.810604  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:22.812719  8058 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:22.813797  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:22.815259  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:22.815948  8054 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 12: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:22.817301 12243 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:22.817454  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.819177  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:22.819986  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:22.821218  8062 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 4. (index mismatch)
I20260709 13:14:22.823273  8175 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:22.824334  8054 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:14:22.827323  8133 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: []
W20260709 13:14:22.819159 12300 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:14:22.730998 12226 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:22.831151  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.822665 12166 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:22.834266  8058 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:22.834872 12300 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:22.834224  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:22.835246  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:22.835888  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.830654 12231 consensus_queue.cc:1059] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:14:22.841710 12231 raft_consensus.cc:3055] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:22.842069 12231 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:22.840342  8054 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:22.842648 12228 raft_consensus.cc:3055] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:22.845544 12116 consensus_queue.cc:1059] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:22.846776  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:22.847527  8211 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:22.847743  8056 raft_consensus.cc:2410] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:22.848218  8210 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:22.848296  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:22.849087  8212 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:14:22.849000  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:22.839649 12226 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:22.849720  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:22.863965  8131 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:22.849861  8204 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:22.850406  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:22.838207  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.854440  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.854285  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.855852  8132 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:22.828044 12013 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:14:22.849565  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.874795  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:22.875556  8208 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:22.904021  8210 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:14:22.901870  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.902845  8693 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:22.910320  8212 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:22.913399  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.856217 12228 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:22.914817 12228 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:22.915928 12228 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:22.868299 12166 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:22.968536  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:22.934367  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:22.950368 12225 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 2.709s	user 0.006s	sys 0.016s
I20260709 13:14:22.920140 12239 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:14:22.951303  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 116129us
W20260709 13:14:22.954708 12336 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:14:22.955685 11773 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:22.955758 12114 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:22.974545  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:22.970880 12211 consensus_meta.cc:287] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.665s	user 0.009s	sys 0.000s
I20260709 13:14:22.979981  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:22.976120  8211 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:22.979080  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:22.984625  8175 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:22.973140 12239 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:22.987717  8211 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:22.985576 12300 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:22.976970 11773 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.991165  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:22.996677  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:22.993829 12169 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:23.000981  8212 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:14:23.002620 12257 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:22.985507 12301 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:22.996536  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:22.998464  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:23.005208  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:23.004079 12176 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:23.006708 12298 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:23.007287 12298 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.008108 11920 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:23.008515 11920 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.008829 11920 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:23.009619 12285 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:23.010473 12191 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.010953 12233 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:23.011081 12159 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:23.011732 12159 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.012171 12303 raft_consensus.cc:3055] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:23.012557 12303 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:23.013429 12303 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.013899 12159 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:23.014536  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.010959 12191 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.015363 12298 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:23.016830 12191 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:23.017442 12300 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:20.846370  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:23.018229 12303 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:23.015388  8102 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:22.996061 11773 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:23.028542  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:23.029582  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:23.027635 12228 raft_consensus.cc:3055] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:22.971570 12231 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.032675 12301 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:23.034019  7950 maintenance_manager.cc:419] P e3b771f7515a4bb7beb66bc429c57201: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033354
I20260709 13:14:23.034938  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:22.996236 12211 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:14:23.036211  8729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 690) took 1513 ms. Trace:
I20260709 13:14:23.039860  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:23.040843  8729 rpcz_store.cc:276] 0709 13:14:21.522791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.522909 (+   118us) service_pool.cc:224] Handling call
0709 13:14:21.523199 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:21.957730 (+434531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:23.036006 (+1078276us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b54004d21d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:23.036156 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":812053888}
I20260709 13:14:23.042649 12013 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:23.032615 12228 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:23.042837 12176 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:23.044768 12228 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.045446 12176 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:23.045912 12176 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:23.031051 12191 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:23.049562 12191 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.049961 12191 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:23.047708  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:23.048740 12300 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:23.052098  8057 pending_rounds.cc:85] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 5
I20260709 13:14:23.053053  8057 pending_rounds.cc:107] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:14:23.053454  8057 raft_consensus.cc:2889] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:23.052835 12300 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:23.057766 12300 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:23.058305 12341 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.002s
I20260709 13:14:23.058877  8696 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:23.038766 12209 consensus_queue.cc:1059] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:14:23.060989 12273 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.061488 12273 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.062367 11954 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:23.063802  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 691) took 1514 ms. Trace:
I20260709 13:14:23.064191  8724 rpcz_store.cc:276] 0709 13:14:21.549397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.550517 (+  1120us) service_pool.cc:224] Handling call
0709 13:14:21.559540 (+  9023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:22.143179 (+583639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:23.063584 (+920405us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b540060f754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:23.063750 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1335104896}
I20260709 13:14:23.042946 12226 consensus_queue.cc:1059] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:14:23.067028 12233 consensus_queue.cc:1059] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:14:23.069998  7959 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37774 (request call id 140) took 12656 ms (client timeout 15000 ms). Trace:
I20260709 13:14:23.037644 12257 consensus_queue.cc:1059] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 13:14:23.072171  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.072816 11954 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:23.073293 12298 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:14:23.073484 12159 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:23.069768 12301 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:23.078258  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:23.112227 12176 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.113029 12166 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:14:23.070443  7959 rpcz_store.cc:269] 0709 13:14:10.400194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:10.400350 (+   156us) service_pool.cc:224] Handling call
0709 13:14:23.056309 (+12655959us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:18.467759 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:18.468456 (+   697us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:18.468510 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:14:18.468529 (+    19us) tablet.cc:662] Decoding operations
0709 13:14:18.470291 (+  1762us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:18.472280 (+  1989us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:18.472309 (+    29us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:14:18.474305 (+  1996us) tablet.cc:701] Row locks acquired
0709 13:14:18.474378 (+    73us) write_op.cc:260] PREPARE: finished
0709 13:14:18.474527 (+   149us) write_op.cc:270] Start()
0709 13:14:18.474699 (+   172us) write_op.cc:276] Timestamp: P: 1783602858474463 usec, L: 0
0709 13:14:18.474718 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:14:18.475477 (+   759us) log.cc:844] Serialized 9323 byte log entry
0709 13:14:19.355205 (+879728us) op_driver.cc:464] REPLICATION: finished
0709 13:14:19.917255 (+562050us) write_op.cc:301] APPLY: starting
0709 13:14:19.917416 (+   161us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:20.055762 (+138346us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:20.095594 (+ 39832us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:20.098146 (+  2552us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:20.125703 (+ 27557us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=23
0709 13:14:20.125769 (+    66us) write_op.cc:312] APPLY: finished
0709 13:14:20.298509 (+172740us) log.cc:844] Serialized 158 byte log entry
0709 13:14:20.374798 (+ 76289us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:20.386706 (+ 11908us) write_op.cc:454] Released schema lock
0709 13:14:20.386798 (+    92us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:20.387181 (+   383us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:20.387211 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5429,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":543176,"apply.run_cpu_time_us":10151,"apply.run_wall_time_us":469942,"num_ops":23,"prepare.queue_time_us":169967,"prepare.run_cpu_time_us":25250,"prepare.run_wall_time_us":121321,"raft.queue_time_us":269686,"raft.run_cpu_time_us":28491,"raft.run_wall_time_us":399718,"replication_time_us":880286,"thread_start_us":681253,"threads_started":5,"wal-append.queue_time_us":136661,"wal-append.run_cpu_time_us":8126,"wal-append.run_wall_time_us":2738612}]]}
I20260709 13:14:23.112768 11773 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.114197 12169 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.072434 12226 raft_consensus.cc:3055] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:14:23.116747 12305 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.535s	user 0.002s	sys 0.000s
I20260709 13:14:23.143262 12191 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.117408  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:23.147167  8060 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:23.120764  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:23.148037  8210 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
W20260709 13:14:23.128810  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 50665.4us
I20260709 13:14:23.129441 12287 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:23.114979 12300 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.144805  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:23.146250 11954 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.148331  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:23.128495 12298 consensus_queue.cc:1243] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:23.149092  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:23.157999  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.138660 12226 raft_consensus.cc:740] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:23.159173  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:23.159857 12226 consensus_queue.cc:260] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.161159 12226 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:23.161222 12124 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:23.161584  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 784) took 1581 ms. Trace:
I20260709 13:14:23.162233 12124 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:23.162752 12124 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.163187 12124 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:23.163684 12135 raft_consensus.cc:3055] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:23.164112 12135 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:23.164896 12135 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.165814 12135 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:23.168988  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:23.169114  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:23.169941 12277 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:23.170632 12301 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:23.171002 11773 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:23.173815  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:23.174882  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:23.175446  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 695) took 1321 ms. Trace:
I20260709 13:14:23.175840  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:23.175801  8726 rpcz_store.cc:276] 0709 13:14:21.854561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.855129 (+   568us) service_pool.cc:224] Handling call
0709 13:14:21.864214 (+  9085us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:22.009456 (+145242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:22.009483 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:22.009525 (+    42us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:22.009556 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:23.175237 (+1165681us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400198f54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:23.175280 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:23.175396 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333535488}
I20260709 13:14:23.177155  8060 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:23.178573 12176 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:23.178874  8058 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:23.179852  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:23.179858  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.181142  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:23.139058 12272 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:23.170063  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:23.182770  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:23.178531  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:23.183885  8056 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:23.180445  8054 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:23.162034  8727 rpcz_store.cc:276] 0709 13:14:21.581106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.581222 (+   116us) service_pool.cc:224] Handling call
0709 13:14:23.161417 (+1580195us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400653954. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:23.161531 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2316951296}
I20260709 13:14:23.178015  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:23.179615 12300 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:23.186678  8206 raft_consensus.cc:2393] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:23.187011  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:23.187448  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:23.187925  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:23.188500  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:23.189092  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:23.189145 12277 consensus_queue.cc:1059] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:23.189393  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:23.189181  8206 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 12: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:23.191907  8102 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:23.194918  8056 raft_consensus.cc:2410] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:23.155467 12287 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.195932  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:23.197115  8723 raft_consensus.cc:1813] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Handling vote request from an unknown peer 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:23.197731  8723 raft_consensus.cc:2410] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:14:23.198737 12287 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:23.190071 12277 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:23.140452 12169 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.200896  8063 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:23.205941  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:23.181254 11954 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.209936 12124 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.211381 11954 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:23.212240  8130 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:14:23.212525 12124 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:23.214416 11920 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.215279  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2846) took 1485 ms. Trace:
I20260709 13:14:23.215626  8203 rpcz_store.cc:276] 0709 13:14:21.730108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.959880 (+229772us) service_pool.cc:224] Handling call
0709 13:14:21.960284 (+   404us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:21.971933 (+ 11649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:21.971967 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:21.991079 (+ 19112us) log.cc:844] Serialized 52 byte log entry
0709 13:14:22.132591 (+141512us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:22.132989 (+   398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.133055 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:23.116089 (+983034us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:23.215026 (+ 98937us) raft_consensus.cc:1685] finished
0709 13:14:23.215046 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:23.215228 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3747,"raft.queue_time_us":9300,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":1795,"spinlock_wait_cycles":5701504,"thread_start_us":110832,"threads_started":1,"wal-append.queue_time_us":540957}
I20260709 13:14:23.217284  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:23.217757  8203 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:23.216426 11920 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:23.196466 12298 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:23.223484  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:23.227407 12338 log.cc:927] Time spent T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.275s	user 0.002s	sys 0.000s
I20260709 13:14:23.231103 12169 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:23.201596 12277 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:23.243630  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.253525  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.246140 12228 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:23.261891  8203 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
W20260709 13:14:23.267277  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.267184  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:23.268612  8696 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:23.267707  8203 raft_consensus.cc:2376] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10. Re-sending same reply.
I20260709 13:14:23.272544  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:23.273510  8203 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:23.273577  8204 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:23.274873  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:23.275744  8203 raft_consensus.cc:2393] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:23.276628  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:23.277432  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:23.278743  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:23.279737  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:23.281018  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:23.284505  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:23.285244  8203 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:23.286087  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:23.286152  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:23.289498  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:23.272322 11773 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:14:23.290475  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.307015  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:23.309113  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:23.310756  8696 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:23.315995  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:23.319846  8063 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:14:23.313542  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:23.319869  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:23.321331  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:23.287372 12281 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.287204 12300 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:23.322247  8058 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:23.288508 12124 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:23.291455 12298 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:23.311838 12169 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:23.310662 12166 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:23.341717  8058 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:23.344043  8101 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:23.322196 12243 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:23.359994  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:23.360999  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.322690 11773 raft_consensus.cc:697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:23.270146 12277 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.362090  8058 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:14:23.331459 12298 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.333081 12064 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:23.366791  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 50902.8us
I20260709 13:14:23.357368 11920 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.357540 12154 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:23.363359 12191 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:23.383618  8101 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:23.366989 12298 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:23.364384 12124 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:23.330746 12281 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.366891 12023 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:14:23.405072  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.402743  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:23.392508  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.402848  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2853) took 1580 ms. Trace:
I20260709 13:14:23.403061  8208 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:23.363721 12349 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:23.399395  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:23.471676  8204 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:23.402406 12273 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:23.410951 12344 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:14:23.382151 12166 raft_consensus.cc:697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:23.448859  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:23.382091 12136 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:14:23.394817 12299 log.cc:927] Time spent T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 1.296s	user 0.014s	sys 0.000s
I20260709 13:14:23.403548  8104 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:23.455802  8209 rpcz_store.cc:276] 0709 13:14:21.822854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.826399 (+  3545us) service_pool.cc:224] Handling call
0709 13:14:23.402679 (+1576280us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b540025cbd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:23.402770 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1624381824}
I20260709 13:14:23.447456 12287 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.387923 12277 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:23.420825 12191 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:23.393395 12115 raft_consensus.cc:3055] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:23.459378  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:23.464298  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:23.465111  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:23.492412  8058 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:23.442464  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:23.443269  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:23.493388  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 50237.5us
W20260709 13:14:23.493554  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 72655.4us
I20260709 13:14:23.493945 12273 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:23.494249  8101 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:23.495536 12159 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:23.496122 12159 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:23.496229 12160 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:23.496768 12159 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:23.490769 12328 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:23.498579 12328 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:23.499356 12328 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.491724 12115 raft_consensus.cc:740] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:23.501467  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2847) took 1771 ms. Trace:
I20260709 13:14:23.501884  8207 rpcz_store.cc:276] 0709 13:14:21.730440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.961911 (+231471us) service_pool.cc:224] Handling call
0709 13:14:21.978104 (+ 16193us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:21.993531 (+ 15427us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:14:21.993559 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:21.994006 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 13:14:22.087653 (+ 93647us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:22.088000 (+   347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.088082 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:23.394217 (+1306135us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:23.501193 (+106976us) raft_consensus.cc:1685] finished
0709 13:14:23.501213 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:23.501411 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9706,"raft.queue_time_us":12054,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":13295,"spinlock_wait_cycles":48980096,"thread_start_us":93195,"threads_started":1,"wal-append.queue_time_us":104050}
I20260709 13:14:23.482447 12349 raft_consensus.cc:697] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:23.457494  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:23.507154  8211 raft_consensus.cc:1813] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Handling vote request from an unknown peer 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:23.507613  8211 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 11: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:23.472926  8060 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 7.
I20260709 13:14:23.474310  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:23.411217 12229 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:23.481597 12258 raft_consensus.cc:3055] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:23.389196 12064 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.484476  8210 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
W20260709 13:14:23.484584  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 103209us
I20260709 13:14:23.386816 12319 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:14:23.392789 12301 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:23.486510  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 83064.2us
I20260709 13:14:23.390558 12154 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.487566 12194 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:23.491938  8138 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:23.492575  8693 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:23.504307  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:23.505716  8129 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:14:23.507578 12135 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.506287  8133 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:14:23.529634  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:23.529868  8206 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:23.506567  8132 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:23.509956  8204 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:23.510182  8211 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:23.532239  8209 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:14:23.513622  8135 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:23.513451 12349 consensus_queue.cc:237] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.459169 11773 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.519368  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:23.525267  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:23.388999 11920 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.509248 12162 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:23.547688  8056 raft_consensus.cc:2410] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:14:23.489773 12124 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.549839  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1394) took 2797 ms. Trace:
W20260709 13:14:23.552199  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 56581.9us
I20260709 13:14:23.528512  8210 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:23.538199  8104 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:23.557868  8133 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:14:23.542028 12154 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:23.534065 12246 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:23.512346 12258 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:23.549382  8211 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:23.561038  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:23.546514  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.561816  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.543993 12064 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:23.554741  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:23.563319  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:23.551847  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:23.562289  8129 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:23.564628  8211 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:14:23.548625 12160 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:23.547333 12162 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.517897  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:23.535264 12135 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.552997  8136 rpcz_store.cc:276] 0709 13:14:20.751919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.895645 (+143726us) service_pool.cc:224] Handling call
0709 13:14:20.968888 (+ 73243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:20.969153 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:20.992525 (+ 23372us) log.cc:844] Serialized 14 byte log entry
0709 13:14:21.120849 (+128324us) log.cc:844] Serialized 14 byte log entry
0709 13:14:21.347417 (+226568us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b1800285f40. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ef72a6 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f29522 00007f6db7f2b7c1
0709 13:14:21.348098 (+   681us) spinlock_profiling.cc:243] Waited 87.4 ms on lock 0x7b1800285f40. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f29522 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:14:21.551340 (+203242us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b1800285f40. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:14:21.750727 (+199387us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b1800285f40. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbb00e060 00007f6dbb020716 00007f6dbb031287 00007f6dbb031208
0709 13:14:21.849032 (+ 98305us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:21.849092 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:21.849137 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.849197 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:23.549687 (+1700490us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b54004092d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:23.549789 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10617,"prepare.queue_time_us":23821,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":73,"raft.queue_time_us":20349,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":57390,"spinlock_wait_cycles":443297152,"thread_start_us":225363,"threads_started":2,"wal-append.queue_time_us":516421,"wal-append.run_cpu_time_us":11354,"wal-append.run_wall_time_us":1318632}
I20260709 13:14:23.575305  8210 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:23.574183  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 57533.8us
I20260709 13:14:23.577308  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:23.560563 12136 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:23.585109  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:23.586683  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:23.592619 12281 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:23.574869 12162 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:23.597049  8129 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:23.598800  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:23.578186 12261 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:23.600059 12160 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:23.606065  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:23.606722  8058 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:23.607883 12283 raft_consensus.cc:3055] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:23.608294 12283 raft_consensus.cc:740] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:23.609122 12283 consensus_queue.cc:260] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.612458 12261 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.617592 11920 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:23.587076 12136 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.575148 12113 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:14:23.586288 12300 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:23.602698 12258 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.591473 12348 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:14:23.601840 12086 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:23.552659 12166 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.659657  8130 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:14:23.659721  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:23.685145  8722 consensus_queue.cc:237] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:23.661041  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 60054.9us
I20260709 13:14:23.661825  8058 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:23.611501 12308 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:23.668125  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:23.689975  8693 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:23.668191 12136 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:23.662154 12355 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:14:23.692597  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.693897  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.664582 12283 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:23.694433  8102 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:23.665467 12261 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:23.698715 12086 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:23.692575  8129 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:23.695122  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:23.701903 12086 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.703482 12086 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:23.668201 12253 consensus_queue.cc:1059] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:14:23.668663 12301 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:14:23.692380 12176 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:14:23.709862  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 105023us
I20260709 13:14:23.690442 12300 raft_consensus.cc:697] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:23.714511  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:23.696069 12287 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:23.694164 11920 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:23.718050 11920 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.705708 12319 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:23.702579 12258 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:23.690727 12243 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:14:23.724030 12135 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:23.724404 11920 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:23.725452 12300 consensus_queue.cc:237] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.726819 12113 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:23.740264 12357 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:14:23.727303 12231 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:23.728749  8101 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:23.721239 12240 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:23.732398 12345 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.413s	user 0.001s	sys 0.000s
I20260709 13:14:23.734206  8062 raft_consensus.cc:2410] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:14:23.739892 12113 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.728094 12328 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:23.730593 12160 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:23.744601  8101 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:23.758572  8101 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:23.752617  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:23.755609  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.755362  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:23.756326  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:23.763546  8101 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:23.764653  8101 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:23.747910 12135 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:23.768925  8727 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:14:23.767452  8101 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:23.766644 11954 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:23.771389  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:23.772329  8101 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:23.773209  8212 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:23.778406 12364 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:14:23.780156  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.783031 12364 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:23.765271 12355 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:14:23.784757 12356 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:14:23.766637 12319 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:23.766782 12154 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:23.770627 12135 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.791265 12307 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.795537 12272 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:23.797072  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:23.870507  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.868790 12136 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.870543 12298 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.771116 12224 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.789310 11954 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:23.750066 12240 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.773492 12328 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:23.770285 12228 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:23.893116 12135 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:23.880510 12319 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.873124  8211 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 13:14:23.902700  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:23.885392  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 86080.7us
I20260709 13:14:23.895913 12272 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:23.908874  8727 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:23.909512 12319 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.910691  8727 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:14:23.893599 12162 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:23.954473  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 46732.9us
W20260709 13:14:23.960024  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:23.964386  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 178713us
I20260709 13:14:23.965746  8101 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:23.897384 12086 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.895769 12307 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:23.969159  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:23.970768  8058 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:14:23.902891 12224 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.952956 12233 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:23.905858 12228 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:23.979565 12324 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 1.423s	user 0.000s	sys 0.012s
I20260709 13:14:23.988606  8723 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:23.990320  8212 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:23.992993 12162 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:23.995400  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:23.952283 12365 consensus_queue.cc:1059] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:14:23.994961 12369 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:24.004217 12239 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:24.010542 12277 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:23.980813 12233 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:23.977871  8693 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:24.046088  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 68321.1us
I20260709 13:14:24.030332  8211 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 7. Current term is 8. Ops: []
W20260709 13:14:24.048237  8061 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1589) took 1807 ms (client timeout 1629 ms). Trace:
I20260709 13:14:23.969655 12166 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:24.048857  8061 rpcz_store.cc:269] 0709 13:14:22.241128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:22.259085 (+ 17957us) service_pool.cc:224] Handling call
0709 13:14:24.047959 (+1788874us) spinlock_profiling.cc:243] Waited 97.9 ms on lock 0x7b5400219b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:24.048147 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225140480}
I20260709 13:14:23.968632 12243 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:14:24.053490  8055 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:14:24.058758  8055 pending_rounds.cc:85] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 11
I20260709 13:14:24.059032  8055 pending_rounds.cc:107] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:14:24.059275  8055 raft_consensus.cc:2889] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:24.061344 12136 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:24.071209 12240 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:24.006196 12239 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:24.078826  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:24.080188  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 114508us
I20260709 13:14:24.054248  8061 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:23.980044 12301 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:24.081489  8101 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:24.086825  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:24.088006  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.057181 12364 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:14:24.081622 12370 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:14:24.080869 12226 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.100018  8101 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:24.105803  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:24.113358  8063 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:24.114605  8211 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:24.076665 12166 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:24.131682  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:24.178354  8211 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
W20260709 13:14:24.128176  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.095664 12364 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.188174 12166 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:24.123117 12226 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:24.112843 12182 consensus_queue.cc:1059] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 13:14:24.186388 12364 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:24.181515 12211 raft_consensus.cc:3055] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:24.202687 12211 raft_consensus.cc:740] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:24.209250  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:24.210062  8212 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:24.220611 12373 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:24.221143 12373 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:24.221556 12373 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:24.224624 12086 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:24.226058 12224 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:24.226267  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:24.227247  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:24.227910  8211 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:24.228777 12182 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.229373 12182 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:24.227237  8063 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 11: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:24.233095  8212 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:24.239814  8211 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:14:24.048254 12369 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:24.090411 12301 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:24.137382 12371 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:14:24.265589  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 156708us
I20260709 13:14:24.182875 12298 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:24.245613 12283 consensus_meta.cc:287] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.543s	user 0.011s	sys 0.000s
I20260709 13:14:24.231261 12319 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:24.267742 12183 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.311s	user 0.010s	sys 0.030s
I20260709 13:14:24.304502  8061 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: [8.5-10.6]
W20260709 13:14:24.266466 12346 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:14:24.311420 12154 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:24.312305 12374 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.312809 12374 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:24.313510 12369 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:24.314889 12374 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:24.315241  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:24.315879  8727 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:24.319011  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:24.319608  8727 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:24.320398 12375 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:24.322623  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:24.323246  8210 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:24.286839 12231 consensus_meta.cc:287] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.000s
I20260709 13:14:24.281556 12371 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:14:24.305786 12183 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.350s	user 0.011s	sys 0.030s
I20260709 13:14:24.335012 12364 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:24.364017 12226 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:24.365950  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:24.334299 12224 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:24.370330  8210 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:24.336254 12365 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:24.353562 12258 consensus_meta.cc:287] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.617s	user 0.011s	sys 0.000s
I20260709 13:14:24.337845 12371 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:24.314211 12211 consensus_queue.cc:260] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:24.384790  8727 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:14:24.392817  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:24.401196  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:24.403117  8210 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:24.374987 12233 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:24.414788  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:24.420768  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:24.422243  8061 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:24.426736  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:24.432257  8212 raft_consensus.cc:2410] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:14:24.433149  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:24.388156 12364 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:24.373893 12224 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:24.375720 12365 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:24.413782 12211 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:24.462710  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2740) took 1289 ms. Trace:
I20260709 13:14:24.464758  8054 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:14:24.465189  8054 pending_rounds.cc:85] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 5
I20260709 13:14:24.465442  8054 pending_rounds.cc:107] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:14:24.465713  8054 raft_consensus.cc:2889] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:24.463244  8062 rpcz_store.cc:276] 0709 13:14:23.172907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.188875 (+ 15968us) service_pool.cc:224] Handling call
0709 13:14:24.462449 (+1273574us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54001e91d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:24.462631 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1251698048}
I20260709 13:14:24.476598  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:24.477540  8062 raft_consensus.cc:2410] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:24.398048 12258 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:24.504762  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.515650  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:24.516435  8212 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:24.424612  8210 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:24.517418  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:24.432917 12182 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:24.527410  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 635) took 3152 ms. Trace:
I20260709 13:14:24.530401  8027 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:24.534039  8728 rpcz_store.cc:276] 0709 13:14:21.368168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.369597 (+  1429us) service_pool.cc:224] Handling call
0709 13:14:21.798191 (+428594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:22.583792 (+785601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:23.523467 (+939675us) spinlock_profiling.cc:243] Waited 779 ms on lock 0x7b540041e6d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:24.520432 (+996965us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b540041e6d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:24.520557 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2776123520}
I20260709 13:14:24.565618  8061 raft_consensus.cc:2410] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 13:14:24.569607 12328 consensus_meta.cc:287] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.667s	user 0.007s	sys 0.001s
I20260709 13:14:24.571635 12224 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:24.572278 12272 consensus_meta.cc:287] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.617s	user 0.002s	sys 0.008s
I20260709 13:14:24.572721 12272 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:24.522109 12258 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:24.575775 12272 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:24.576993 12258 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:24.566284 12300 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:14:24.578838  8060 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:24.580981  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2862) took 1456 ms. Trace:
W20260709 13:14:24.582088  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.581395  8203 rpcz_store.cc:276] 0709 13:14:23.124513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.308890 (+184377us) service_pool.cc:224] Handling call
0709 13:14:24.580766 (+1271876us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b5400240ed4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:24.580921 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329470336}
I20260709 13:14:24.584787  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:24.587354  8027 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:24.589396  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:24.551512 12233 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:24.593358 12226 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:14:24.595450  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:24.582127 12379 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:14:24.570173 12328 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:14:24.608372 12391 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:14:24.606122 12224 consensus_queue.cc:1059] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:14:24.606833  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:24.605681 12231 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:24.609607 12231 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.610112 12231 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:24.612283 12389 log.cc:927] Time spent T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:14:24.615358  8203 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:14:24.592217 12323 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 2.030s	user 0.006s	sys 0.007s
I20260709 13:14:24.619338 12300 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:24.619918 12300 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:24.622057 12300 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:24.593022 12258 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:24.646651  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.603865 12239 consensus_queue.cc:1059] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:14:24.656853 12023 raft_consensus.cc:3055] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:24.657262 12023 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:24.658905 12388 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:14:24.635214 12233 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.605942 12374 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:14:24.666229  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2848) took 2935 ms. Trace:
I20260709 13:14:24.649369 12301 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:24.677479 12231 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:24.666594  8205 rpcz_store.cc:276] 0709 13:14:21.730718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.979687 (+248969us) service_pool.cc:224] Handling call
0709 13:14:21.980310 (+   623us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:21.993467 (+ 13157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:21.993516 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:21.995074 (+  1558us) log.cc:844] Serialized 26 byte log entry
0709 13:14:22.351105 (+356031us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:22.351141 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.351205 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:24.386967 (+2035762us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:24.663483 (+276516us) raft_consensus.cc:1685] finished
0709 13:14:24.663503 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:24.666153 (+  2650us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3788,"raft.queue_time_us":15174,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":2344,"spinlock_wait_cycles":8052608,"thread_start_us":355183,"threads_started":1,"wal-append.queue_time_us":475945}
W20260709 13:14:24.679068  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 90295.4us
I20260709 13:14:24.635283 12258 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:24.694519  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.679924 12226 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:24.658056 12023 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:24.665781 12233 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:24.715144 12261 consensus_meta.cc:287] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 1.006s	user 0.011s	sys 0.001s
I20260709 13:14:24.732103  8696 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:24.735540 12386 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:14:24.737071  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:24.719208 12301 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.740309  8696 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:24.747248  8696 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:24.749634 12194 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:24.734664 12272 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:14:24.750593 12194 raft_consensus.cc:697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:24.733953 12365 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:24.718801 12226 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:24.746790 12393 log.cc:927] Time spent T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:14:24.746497 12243 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:24.741894 12182 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:24.751775 12194 consensus_queue.cc:237] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:24.763480 12233 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:24.764050 12258 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:24.748584 12301 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:24.772502  8208 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:14:24.773408 12301 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:24.775460 12113 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:24.777923 12365 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:24.778471 12365 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:24.780174  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:24.780979 12365 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:24.781558 12300 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:24.781771 12169 consensus_queue.cc:1059] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:24.782232 12300 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:24.782650 12169 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:24.783236  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:24.783460 12169 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:24.784380 12169 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:24.785184  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:24.785358 12166 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:14:24.783062  8206 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2859) took 1659 ms (client timeout 2050 ms). Trace:
W20260709 13:14:24.785841  8206 rpcz_store.cc:269] 0709 13:14:23.123667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.191347 (+ 67680us) service_pool.cc:224] Handling call
0709 13:14:24.782913 (+1591566us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54002be154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:24.783008 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":740119424}
I20260709 13:14:24.786423  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:24.787011  8208 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:24.785431 12169 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:24.787425  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1576) took 2625 ms. Trace:
I20260709 13:14:24.785192  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:24.747344 12401 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:24.787714 12135 consensus_meta.cc:287] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.877s	user 0.009s	sys 0.001s
I20260709 13:14:24.788311 12401 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:24.788707 12135 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:24.787955  8059 rpcz_store.cc:276] 0709 13:14:22.162477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:22.162651 (+   174us) service_pool.cc:224] Handling call
0709 13:14:22.163027 (+   376us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:22.223981 (+ 60954us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:14:22.238588 (+ 14607us) log.cc:844] Serialized 14 byte log entry
0709 13:14:22.503748 (+265160us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:22.511484 (+  7736us) log.cc:844] Serialized 52 byte log entry
0709 13:14:22.511629 (+   145us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:22.526153 (+ 14524us) log.cc:844] Serialized 14 byte log entry
0709 13:14:22.528915 (+  2762us) log.cc:844] Serialized 14 byte log entry
0709 13:14:22.529158 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.988477 (+459319us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b1800488e60. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:14:23.000385 (+ 11908us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b1800488e60. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f29522 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:14:24.048807 (+1048422us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b1800488e60. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ef72a6 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419
0709 13:14:24.787152 (+738345us) spinlock_profiling.cc:243] Waited 60.6 ms on lock 0x7b5400051754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:24.787197 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:24.787236 (+    39us) raft_consensus.cc:1685] finished
0709 13:14:24.787247 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:24.787375 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35070,"prepare.queue_time_us":975111,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":16328,"raft.queue_time_us":7103,"raft.run_cpu_time_us":400,"raft.run_wall_time_us":15633,"spinlock_wait_cycles":227823744,"thread_start_us":179431,"threads_started":1,"wal-append.queue_time_us":261947,"wal-append.run_cpu_time_us":3093,"wal-append.run_wall_time_us":1119721}
I20260709 13:14:24.785804 12307 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:24.794943  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:24.821004  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:24.821619  8204 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:24.733772 12261 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:24.797711 12154 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.797653 12319 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.795903 12135 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:24.758211 12023 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:24.796129 12298 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.903815  8129 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:24.904139  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.922570  8130 raft_consensus.cc:2393] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:24.905192  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.906697  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.923353  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.924679  8136 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:24.925688  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:24.926415  8727 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:14:24.783231 12300 raft_consensus.cc:3055] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:24.921880  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.927627  8129 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:24.911054  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:24.901389  8696 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:24.919684  8211 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:24.928090  8063 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:24.919620  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 124765us
I20260709 13:14:24.908718  8059 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:14:24.928364  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.929944  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:24.930171  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:24.896677 12229 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:24.898243 12273 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:24.942255  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 90187.6us
I20260709 13:14:24.907204 12378 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.944747  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:24.945401  8175 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.629s (SENT) [suppressed 35 similar messages]
I20260709 13:14:24.908811 12365 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.957731  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.946621  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.909118 12277 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:24.949591  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.959676  8135 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:24.950030  8724 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:14:24.908581 12384 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:24.916710 12258 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.961274  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.961890  8132 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:24.952472  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.921273 12401 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:24.960716  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.932173 12300 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:24.908352 12154 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:24.975567  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.950709 12253 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:24.915130 12319 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:24.955014 12383 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:24.978534  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.978891  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.982754  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:24.995716  8129 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:24.960141 12411 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.935189 12298 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:24.993788  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:24.971184 12371 consensus_meta.cc:287] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.000s
W20260709 13:14:24.997056  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 48835us
I20260709 13:14:24.944546 12159 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:14:25.000598  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.004840  8060 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:24.951524 12273 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:24.944500 12229 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:24.967911 12162 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:24.982992 12253 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.968111 12365 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.016624  8133 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:25.012564  8132 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:24.974723 12258 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:25.021855  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 76531.4us
I20260709 13:14:24.974922 12384 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.022863  8204 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:14:24.977174 12277 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:25.024729  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 94848.9us
I20260709 13:14:25.031790  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 636) took 1070 ms. Trace:
I20260709 13:14:25.020120 12253 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:25.010892 12371 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.032155  8138 rpcz_store.cc:276] 0709 13:14:23.961117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.961293 (+   176us) service_pool.cc:224] Handling call
0709 13:14:23.971475 (+ 10182us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:23.971929 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:24.091404 (+119475us) log.cc:844] Serialized 14 byte log entry
0709 13:14:24.799807 (+708403us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:24.800521 (+   714us) log.cc:844] Serialized 52 byte log entry
0709 13:14:24.800696 (+   175us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:24.800744 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.800831 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:25.031590 (+230759us) raft_consensus.cc:1685] finished
0709 13:14:25.031607 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.031738 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":425386,"prepare.queue_time_us":9973,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":466028,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":192,"thread_start_us":304094,"threads_started":2,"wal-append.queue_time_us":470373}
I20260709 13:14:24.976173 12378 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.049970  8136 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:25.043979  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:25.055671  8137 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:25.054534 12412 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.019129 12328 consensus_queue.cc:1059] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:14:25.032967 12169 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.019927 12162 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.044930 12229 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.053381 12166 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:25.049113 12183 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:25.049355 12365 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.075125 12319 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.058061 12412 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.079237 12378 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:25.006881 12411 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:25.081691 12399 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:14:25.082584 12413 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:14:25.083230 12373 consensus_meta.cc:287] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.861s	user 0.009s	sys 0.003s
I20260709 13:14:25.035756 12307 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:14:25.055231 12415 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:14:25.108970 11981 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.113714 12231 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:25.116087 12412 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:25.084789 12162 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:25.085790  8063 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:25.045645 12371 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.062402 12261 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:25.019272 12300 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.038702 12301 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:25.099865  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 62609us
I20260709 13:14:25.101716  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1600) took 1786 ms. Trace:
I20260709 13:14:25.105044  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1604) took 1788 ms. Trace:
I20260709 13:14:25.169087  8055 rpcz_store.cc:276] 0709 13:14:23.316793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.326633 (+  9840us) service_pool.cc:224] Handling call
0709 13:14:24.049999 (+723366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.059751 (+  9752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.102692 (+1042941us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b5400219b50. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:25.104989 (+  2297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1652055424}
I20260709 13:14:25.170452  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.171756  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.173265  8055 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:25.168186  8054 rpcz_store.cc:276] 0709 13:14:23.315583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.318043 (+  2460us) service_pool.cc:224] Handling call
0709 13:14:24.464316 (+1146273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.466422 (+  2106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.101516 (+635094us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54001e91d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:25.101660 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2635372416}
I20260709 13:14:25.118573  8062 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:14:25.129333 12115 consensus_queue.cc:260] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.127403 12413 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:25.136071  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:25.136755 11816 raft_consensus.cc:3055] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:25.118261 12383 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:25.143838 12384 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:25.145308 12258 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.148375 12273 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:25.149788 12277 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.149910  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:25.151002 12298 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.151715 12154 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.095096 12226 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.095600 12287 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:25.153141  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:25.178623 12425 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:25.067364 12169 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:25.164661  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 120766us
I20260709 13:14:25.200904  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:25.096755 12135 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:14:25.152055 12307 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:25.167695  8101 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:25.151044 12414 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:25.099514 12183 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.106477 12159 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
I20260709 13:14:25.193653  8063 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.119146 12373 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.185535 11816 raft_consensus.cc:740] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:25.186589 12383 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:25.206128  8138 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:14:25.208051  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:25.179455 12301 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.217941  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1413) took 1713 ms. Trace:
I20260709 13:14:25.213985  8063 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:25.226317  8134 rpcz_store.cc:276] 0709 13:14:23.504500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.504605 (+   105us) service_pool.cc:224] Handling call
0709 13:14:23.506536 (+  1931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:23.544811 (+ 38275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:23.544853 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:23.544907 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:23.544952 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.217678 (+1672726us) spinlock_profiling.cc:243] Waited 38.1 ms on lock 0x7b54001ab154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:25.217735 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.217875 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87640832}
I20260709 13:14:25.229367  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.196542 12384 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:14:25.210722 12371 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:25.223958 12159 consensus_queue.cc:1243] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:25.205334 12425 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.236531 12413 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.241807 12384 raft_consensus.cc:3055] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:25.265662 12384 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:25.266683 12166 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:25.266472 12384 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.267135 12166 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:25.267674 12384 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:25.218956 12307 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.271945 12169 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:25.210444 12414 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.280555  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:25.281489  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:25.220921  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 151661us
I20260709 13:14:25.203677 12287 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.227085 12277 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:25.283268  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.262183  8723 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:14:25.286077  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:25.294211  8724 raft_consensus.cc:2393] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:25.290901  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:25.262506  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:25.295393  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:25.296052  8129 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:14:25.296826  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:25.297413  8129 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:25.298518  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:25.302601  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:25.303345  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:25.294355  8055 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:25.307171 12415 consensus_queue.cc:1059] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:25.308279 12415 raft_consensus.cc:3055] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:25.308619 12415 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:25.309424 12415 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.311868 12287 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:25.256830  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.283442  8104 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:25.317720  8138 raft_consensus.cc:2393] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:25.317911  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:25.318888  8104 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:25.243975 12373 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:25.254743 12385 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.704s	user 0.000s	sys 0.003s
I20260709 13:14:25.252307 12428 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:25.282591 12371 raft_consensus.cc:697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:25.249640 12425 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:25.327771  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:25.250908 12301 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:25.333550  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.336910  8055 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:25.247897 12411 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:25.340149  8205 raft_consensus.cc:2393] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 12: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:25.323836 12422 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:25.352959  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.247310 12253 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.326280 12023 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:25.356448  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2856) took 2235 ms. Trace:
I20260709 13:14:25.364663  8209 rpcz_store.cc:276] 0709 13:14:23.121524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.517734 (+396210us) service_pool.cc:224] Handling call
0709 13:14:23.532036 (+ 14302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:23.535067 (+  3031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.356262 (+1821195us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b540019e1d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:25.356391 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29708672}
I20260709 13:14:25.340940 12415 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:25.356583  8131 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:25.355525  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:25.362670  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 207767us
I20260709 13:14:25.367242  8055 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:25.338567 12169 raft_consensus.cc:476] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Not starting pre-election -- already a leader
I20260709 13:14:25.323959 12354 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:14:25.316787 12277 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.339898  8061 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:25.246541 11816 consensus_queue.cc:260] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:25.405234  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 86421.5us
I20260709 13:14:25.354837 12422 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:25.414897  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 214101us
I20260709 13:14:25.416658  8727 raft_consensus.cc:2393] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:25.417058  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:25.417483  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 617) took 1451 ms. Trace:
I20260709 13:14:25.443732  8056 rpcz_store.cc:276] 0709 13:14:23.966120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.966257 (+   137us) service_pool.cc:224] Handling call
0709 13:14:23.966964 (+   707us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:24.098580 (+131616us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:24.098611 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:24.119312 (+ 20701us) log.cc:844] Serialized 26 byte log entry
0709 13:14:24.388666 (+269354us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:24.388709 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:24.388781 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:25.254217 (+865436us) spinlock_profiling.cc:243] Waited 6.01 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:25.417187 (+162970us) raft_consensus.cc:1685] finished
0709 13:14:25.417210 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.417424 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16225,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":7779,"spinlock_wait_cycles":13822848,"thread_start_us":400052,"threads_started":2,"wal-append.queue_time_us":347377}
I20260709 13:14:25.301451 12383 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.448565  8104 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.412632 12191 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.416996 12135 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:25.466490  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:25.465850  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:25.465503  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:25.470640  8209 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:25.416438 12301 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:25.442071 12422 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:25.471184  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:25.466537  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:25.482156  8207 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:25.482493  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:25.483189  8205 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:25.483448  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:25.483865  8104 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:25.484594  8207 raft_consensus.cc:2393] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:25.485658  8208 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:25.458611 11816 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:25.505985  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2884) took 1154 ms. Trace:
I20260709 13:14:25.486354 12301 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.506444  8211 rpcz_store.cc:276] 0709 13:14:24.351147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.351308 (+   161us) service_pool.cc:224] Handling call
0709 13:14:24.351709 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.928446 (+576737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.505770 (+577324us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b54002bf554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:25.505929 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1305714944}
I20260709 13:14:25.508672 12301 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:25.510174 12253 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.455942 12283 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.488864 12328 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:25.491032 12434 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:25.486055 12135 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:25.459139 12136 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:25.499230 12375 raft_consensus.cc:3055] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:25.460556 12423 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:25.526841  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:25.536587 12399 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:14:25.530855  8209 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:25.573599  8101 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.369277 12023 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.573927  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:25.543349  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:25.574766  8209 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:25.574923  8211 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:25.575913  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:25.577286  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:25.548959  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:25.486428  8104 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:25.552835  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:25.579558  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1740) took 1061 ms. Trace:
I20260709 13:14:25.580019  8212 rpcz_store.cc:276] 0709 13:14:24.518478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.519196 (+   718us) service_pool.cc:224] Handling call
0709 13:14:25.552679 (+1033483us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b5800168398. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:14:25.579490 (+ 26811us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25745280}
I20260709 13:14:25.554845 12277 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:25.446249 12371 consensus_queue.cc:237] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.541117 12383 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:25.541416 12283 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.563275 12209 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:14:25.564335  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 266754us
I20260709 13:14:25.567770  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:25.568526  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:25.592630  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:25.591503  8026 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:25.569715  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:25.569938  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.594480  8055 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:25.594617  8054 raft_consensus.cc:2368] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:14:25.595433  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.595829  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.572576  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.581929  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.571425  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.598057  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.598596  8211 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:25.599886  8696 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:25.600090  8063 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:14:25.600726 12086 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:25.540823 12328 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.601313 12086 raft_consensus.cc:697] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:25.601356 12328 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:25.602491 12086 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.470130 12191 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.527765 12166 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.589775  8057 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:25.596042  8055 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:25.551023 12375 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:25.596974  8056 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:25.582683  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:25.588214 12226 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:25.604593 12229 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.596700  8054 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
W20260709 13:14:25.611151  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 124825us
I20260709 13:14:25.588392 12124 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:25.663843  8204 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:25.665100 12154 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:25.621863 12231 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:14:25.658242  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:25.668099  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 49278.3us
I20260709 13:14:25.592316 12365 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.674615  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:25.704918  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 131423us
W20260709 13:14:25.658983 12391 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:14:25.706238  8101 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.618220 12159 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:14:25.707352  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 134868us
I20260709 13:14:25.606048 12412 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:25.616325 12353 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:25.615487 12258 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.710527  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:25.700582 12301 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:25.668453 12406 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:25.662154 12261 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:14:25.707984 12319 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.621345 12438 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:25.720266  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.617342 12423 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:25.711926  8101 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.729951  8029 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.616845 11816 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.110s
I20260709 13:14:25.656096 12229 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:25.727336 12444 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:14:25.704311 12365 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.655966 12226 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:25.743957  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 152547us
I20260709 13:14:25.745376  8101 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.722200 12426 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:25.703361 12373 consensus_queue.cc:1059] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:14:25.662534 12428 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:14:25.709729 12159 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:25.670907 12135 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.704133 12413 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:14:25.761140  8026 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:25.690091 12124 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.768162  8101 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:25.767648 12442 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 13:14:25.732048 12258 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.751664 12136 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:25.767747 12426 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.729523 12412 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:25.768510  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 401111us
W20260709 13:14:25.787472  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 57604.3us
W20260709 13:14:25.789513  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 199231us
I20260709 13:14:25.799430  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.771435 12159 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:25.788914  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:25.774472 12154 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:25.747601 12319 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.804035  8693 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.807246  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.787874 12373 raft_consensus.cc:3055] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:25.834563  8208 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:25.837934  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.838660  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.839887  8063 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:14:25.837972  8055 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:14:25.843048 12229 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.842957  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:25.843894 12229 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:25.844506 12229 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.825137 12451 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:25.845376 12451 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.794273 12228 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:25.804669 12427 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:25.844928 12229 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:25.794795 12426 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:25.843927 12240 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:25.799513 12136 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:25.849634  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:25.874035  8722 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:25.831385 12364 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:25.815892 12154 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.883306 12154 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:25.884953  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 47114.3us
I20260709 13:14:25.876586  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:25.886096  8029 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.886543  8727 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:25.880249 12124 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:25.880789  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:25.872555 12261 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:25.893235  8722 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:25.896514  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:25.901650  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:25.849525 12451 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:25.804271 12450 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
W20260709 13:14:25.852198 12441 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:14:25.902155  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:25.848378 12373 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:25.905251  8730 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:25.907407  8693 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.892109 12136 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:25.909912  8721 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:25.878252 12427 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:25.884989 12283 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.851068 12253 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:14:25.919579  8029 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.899076 12406 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:25.921458  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.921895 12459 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.921422 12457 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:25.923333  8727 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:25.925040  8029 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:25.925432  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 136577us
I20260709 13:14:25.926090  8026 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.927845  8693 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:25.920796  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 116113us
W20260709 13:14:25.917330 12404 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.905s	user 0.005s	sys 0.000s
I20260709 13:14:25.910416 12448 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:25.933216  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:25.934427  8210 raft_consensus.cc:2410] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:14:25.904666 12446 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:25.918407 12272 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:25.917402 12454 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:25.937894 12458 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:25.899770 12261 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:25.945115  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.923889 12023 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:25.962123 12449 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:14:25.962885 12384 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:25.963436 12384 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.964318 12307 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.965548 12384 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:25.954527 12448 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:25.966424  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1609) took 1036 ms. Trace:
I20260709 13:14:25.939812 12406 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:25.966847  8058 rpcz_store.cc:276] 0709 13:14:24.929969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:24.930103 (+   134us) service_pool.cc:224] Handling call
0709 13:14:24.930329 (+   226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.027414 (+ 97085us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:25.027458 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:25.027510 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:25.027553 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.966179 (+938626us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b540027a954. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:25.966233 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:25.966368 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222756224}
W20260709 13:14:25.970189  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 44175.8us
I20260709 13:14:25.923009 12246 consensus_queue.cc:1059] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 13:14:25.972532  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 47572.5us
I20260709 13:14:25.973836  8026 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:25.973667  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:25.941399 12446 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:25.958899 12365 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:25.958772 12298 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:25.989305  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 59231us
I20260709 13:14:25.974392 12384 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.990835 12384 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:25.991320  8026 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:25.993628  8026 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:25.994714  8026 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:25.980574 12226 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:25.978184 12446 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:25.995767 12445 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 13:14:26.032418  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 74886.8us
I20260709 13:14:26.033237  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:26.035641  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:26.037070  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:26.037642  8137 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:26.037757  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:26.038673  8026 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:25.995144 12298 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:25.977113 12373 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.047868 12432 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:26.002939 12365 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:26.031205 12457 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:26.062541  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:26.065196  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:26.065768  8132 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:14:25.746793 12438 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:26.080761  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.612771 12209 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.085407 12023 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.085945 12023 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.088127 12023 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:26.090399 12443 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.143s
I20260709 13:14:25.721984 12375 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:26.039829 12448 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:26.067278 12384 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:26.070127 12422 consensus_meta.cc:287] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.535s	user 0.006s	sys 0.004s
I20260709 13:14:26.065392 12226 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:26.064911 12457 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.062463 12365 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:25.978282 12427 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.109231  8057 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
W20260709 13:14:26.111481  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 182390us
I20260709 13:14:26.102770 12448 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.844712 12423 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.109707 12226 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:26.120687  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 143996us
I20260709 13:14:26.110526 12375 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:26.109862 12300 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:26.111943 12457 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:26.112682 12373 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:26.109036 12298 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:26.083806 12465 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:14:26.111518 12365 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:26.132424  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:26.134464  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 71095.5us
W20260709 13:14:26.135603  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.118455 12448 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:26.136740  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 484967us
I20260709 13:14:26.128101 12226 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:26.106647 12458 consensus_queue.cc:1175] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8a968cada1e4446ebb2a5e167176e649" responder_term: 8 status { last_received { term: 8 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 8 index: 9 } }
I20260709 13:14:26.109263 12422 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.137039 12261 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:26.146454 12471 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:26.170267 12471 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:26.173354  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:26.189800  8058 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:26.191133  8101 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:26.179870  8026 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:26.202415  8133 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:26.170732 12471 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:26.156635 12462 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:14:26.207041 12475 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:26.136615 12159 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.153796 12459 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:26.208140 12023 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.156237 12438 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:26.195320 12373 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.167707 12319 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:26.241864 12422 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:26.248373 12154 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.145555 12470 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.202204 12298 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:26.154244 12229 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.154892 12446 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:26.212527 12475 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:26.260488 12328 consensus_meta.cc:287] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.003s
I20260709 13:14:26.259198 12373 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:26.296728  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:26.297477  8136 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 12: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:26.297823 12373 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:26.301276 12240 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:26.302894 12457 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.258893 12298 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.304647 12298 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:26.310726 12209 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:26.311666  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2765) took 1035 ms. Trace:
I20260709 13:14:26.276613 12475 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:26.312017  8060 rpcz_store.cc:276] 0709 13:14:25.276063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.408863 (+132800us) service_pool.cc:224] Handling call
0709 13:14:25.443781 (+ 34918us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:25.565768 (+121987us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:25.565804 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:25.566252 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 13:14:25.971358 (+405106us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:25.971392 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.971453 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:26.311469 (+340016us) raft_consensus.cc:1685] finished
0709 13:14:26.311487 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:26.311609 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121535,"raft.queue_time_us":142387,"thread_start_us":404738,"threads_started":1,"wal-append.queue_time_us":423421}
I20260709 13:14:26.268078 12470 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.260866 12459 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:26.314539 12459 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:26.314831 12383 consensus_meta.cc:287] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.725s	user 0.005s	sys 0.005s
I20260709 13:14:26.314688  8061 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:26.259588 12023 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:26.318527  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 187492us
I20260709 13:14:26.319496  8029 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:26.320593  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:26.321321 12209 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:26.321717  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:26.321981 12209 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:26.322464 12375 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:26.314961 12459 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:26.324038  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:26.326375  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:26.326661  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:26.327260  8211 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:26.327260  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:26.329245  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:26.329842  8210 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:26.329900  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:26.330540  8060 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:26.332271  8211 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:14:26.333338  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:26.333581  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:26.334050  8056 raft_consensus.cc:2393] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:26.334105  8062 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:26.334877  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:26.335328  8029 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:26.335642  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:26.332266  8058 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:14:26.337545 12169 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:26.337845  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:26.338495  8054 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:26.338173 12169 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:26.326385  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:26.320812 12136 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.358587 12136 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:26.361716 12471 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:26.356827 12406 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:26.362794  8136 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:26.355198  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:26.326946 12023 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:26.368747  8029 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:26.369194  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:26.369805  8056 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:26.367687  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:26.346155 12300 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:26.345391 12124 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.367686  8132 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:26.371336  8062 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:26.366148 12425 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.382557 12446 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:26.351178 12353 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.374576 12224 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:26.376257 12474 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:26.383001 12298 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.383172 12353 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:26.383559 12474 raft_consensus.cc:3055] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:14:26.383939 12426 consensus_meta.cc:287] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.000s
I20260709 13:14:26.384006 12474 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:26.351256 12383 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:26.384430 12426 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.383179 12224 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:26.385129 12353 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:26.384925 12383 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.385461 12224 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:26.385767 12383 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:26.324749 12328 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:26.386731 12353 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:26.387228 12353 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.387923 12328 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:26.381804  8132 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:26.382323 12425 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:26.405929  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:26.407908  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:26.408427  8136 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:26.408612  8137 raft_consensus.cc:2393] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:26.409288  8130 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:14:26.409401  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:26.391763 12353 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:26.389515  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:26.392233 12300 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.392690 12475 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.392998 12124 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.416347 12475 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:26.417419 12124 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:26.418681 12414 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:26.420986 12300 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:26.421586 12261 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:26.421998 12261 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.422382 12261 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:26.422927  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1417) took 2913 ms. Trace:
I20260709 13:14:26.423411  8131 rpcz_store.cc:276] 0709 13:14:23.509430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.526060 (+ 16630us) service_pool.cc:224] Handling call
0709 13:14:23.526331 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.367048 (+1840717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:26.422630 (+1055582us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54001315d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:26.422858 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4208377216}
I20260709 13:14:26.397631 12365 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:26.364837 12226 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.384927 12474 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:26.427423 12474 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:26.400979 12457 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:26.400828 12451 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.430586  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:26.403311  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:26.411453  8175 leader_election.cc:400] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:14:26.412143  8136 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 12. Ops: []
I20260709 13:14:26.412839  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:26.405620 12423 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.415138  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:26.415614 12191 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:26.410562 12482 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:14:26.431691  8134 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:26.442991  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:26.431696  8135 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:26.446857  8130 raft_consensus.cc:2393] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:26.440498  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:26.448304  8131 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:26.429716 12226 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:26.437412 12471 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:26.433967 12375 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:26.436115 12451 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:26.449455  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:26.433032 12169 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:26.458719 12411 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:14:26.442185 12423 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:26.441001  8129 raft_consensus.cc:2410] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:14:26.444144  8136 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:26.444433  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:26.919747  8063 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:26.941737  8135 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:26.604194 12448 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.970135 12023 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.036466 12240 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:26.900844  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.037218 12023 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.037550 12240 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.040203 12375 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:27.040900 12191 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:27.040946 12457 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:26.943557  8134 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:14:26.918764  8060 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.004s	sys 0.003s
I20260709 13:14:26.994867 12136 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.053928 12457 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:27.055588  8060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.018s	sys 0.019s
I20260709 13:14:27.058779  8061 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:14:26.445235 12365 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:27.056756 12191 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.964071  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:27.063476 12209 consensus_meta.cc:287] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.741s	user 0.017s	sys 0.019s
I20260709 13:14:26.963358  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:26.940477  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:26.944332  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.063674 12425 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:27.066043 12209 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:26.956516  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:26.939715  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:27.069818  8055 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:27.073366  8206 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:27.074301 12425 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:27.074890 12425 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:27.075452 12425 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:27.075160 12375 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.079381  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:27.080418 12261 consensus_meta.cc:287] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.644s	user 0.011s	sys 0.008s
I20260709 13:14:27.083997  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 721) took 1756 ms. Trace:
I20260709 13:14:27.084594  8204 rpcz_store.cc:276] 0709 13:14:25.327728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.328321 (+   593us) service_pool.cc:224] Handling call
0709 13:14:25.328590 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.667628 (+339038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:27.082643 (+1415015us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54002be654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:27.083711 (+  1068us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":770435968}
I20260709 13:14:27.085680 12423 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:27.085242 12261 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.086493 12183 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.087985 12261 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:26.636926 12383 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:26.441951 12459 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.091106 12383 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.092689 12459 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.094753  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2892) took 1626 ms. Trace:
I20260709 13:14:27.095506 12240 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.097220  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:27.090123  8054 consensus_meta.cc:287] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.751s	user 0.008s	sys 0.012s
I20260709 13:14:27.082827  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.094952 12191 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.078253 12375 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:27.094945 12457 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.104988  8211 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:27.109030 12191 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.115680 12414 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.114866 12191 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.116209 12023 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:27.116343 12414 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.117743 12191 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.119802 12383 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.120091 12422 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.120519 12383 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.120828 12422 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.122676 12373 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.123539  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.102159 12423 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.126705  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.126984  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.127892  8054 raft_consensus.cc:2410] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:27.126247  8130 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:27.136015 12373 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.137372  8135 raft_consensus.cc:2410] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:27.135416  8206 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:26.450870 12224 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.144568 12464 raft_consensus.cc:2764] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:14:27.160890 12448 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.163180  8204 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:27.095606  8207 rpcz_store.cc:276] 0709 13:14:25.468661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.486235 (+ 17574us) service_pool.cc:224] Handling call
0709 13:14:25.486733 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.605031 (+118298us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:14:25.655396 (+ 50365us) log.cc:844] Serialized 14 byte log entry
0709 13:14:25.706640 (+ 51244us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:25.706701 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:25.706742 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:27.092197 (+1385455us) spinlock_profiling.cc:243] Waited 6.57 ms on lock 0x7b18003f5500. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ef72a6 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419
0709 13:14:27.094332 (+  2135us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b540019e6d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:27.094392 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:27.094586 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":56721,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":105,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"spinlock_wait_cycles":286652416,"thread_start_us":49580,"threads_started":1,"wal-append.queue_time_us":76385,"wal-append.run_cpu_time_us":2528,"wal-append.run_wall_time_us":1311302}
W20260709 13:14:27.168409  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 665106us
I20260709 13:14:27.170616 12023 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:27.170572 12423 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.171393 12412 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:27.173945  8138 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:27.167043 12448 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:27.175386 12448 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:27.177213 12481 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.822s	user 0.000s	sys 0.038s
I20260709 13:14:27.167208 12414 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.170166  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:27.184530 12412 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:27.184818 12373 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.185027 12412 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:27.185490 12412 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:27.186002 12383 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.182850 12136 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:27.198434 12489 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.093s	user 0.000s	sys 0.004s
I20260709 13:14:27.200811 12136 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:27.202872 12378 raft_consensus.cc:3055] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:27.203464 12378 raft_consensus.cc:740] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:27.204807 12378 consensus_queue.cc:260] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.207367 12448 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.209580  8104 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:27.211853  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2774) took 1179 ms. Trace:
I20260709 13:14:27.212544 12383 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:27.213017 12383 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:27.213491 12383 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:27.214963  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2771) took 1253 ms. Trace:
I20260709 13:14:27.215366  8059 rpcz_store.cc:276] 0709 13:14:25.961092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.961264 (+   172us) service_pool.cc:224] Handling call
0709 13:14:25.961581 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:25.968843 (+  7262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:25.968873 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:25.968910 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:25.968940 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:27.214442 (+1245502us) spinlock_profiling.cc:243] Waited 7.06 ms on lock 0x7b540033efd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:27.214515 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:27.214784 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16235392}
I20260709 13:14:27.219852 12412 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.221906 12412 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:27.224375 12407 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:27.224906 12425 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.225152 12407 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.227555 12425 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:27.231634 12354 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.212263  8057 rpcz_store.cc:276] 0709 13:14:26.032806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.032958 (+   152us) service_pool.cc:224] Handling call
0709 13:14:27.211485 (+1178527us) spinlock_profiling.cc:243] Waited 75.3 ms on lock 0x7b5400335754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:27.211653 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173239552}
I20260709 13:14:27.229975 12154 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.232297 12354 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.234143  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.206115 12378 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:27.202958 12224 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.235085  8207 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:27.237497  8204 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:27.238605  8206 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:27.242746 12354 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:27.243304 12407 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.244027 12413 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:27.244994 12229 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.247658 12422 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.249316 12428 consensus_queue.cc:1059] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:27.244990 12413 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:27.250546 12428 raft_consensus.cc:3055] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:27.250967 12413 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:27.251111 12428 raft_consensus.cc:740] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:27.251848 12166 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:27.255079 12428 consensus_queue.cc:260] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.256984 12428 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:27.257803  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 66869.2us
I20260709 13:14:27.259933  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:27.260596 12023 consensus_queue.cc:1059] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:27.261482 12023 raft_consensus.cc:3055] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:27.261531 12231 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:27.262735 12231 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:27.264289 12231 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.266165  8063 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:14:27.269094  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:27.269650  8063 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:27.271345  8029 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:27.271899 12426 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.272384 12229 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:27.272876 12229 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:27.273304 12229 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:27.273525  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:27.274508  8057 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:27.262045 12023 raft_consensus.cc:740] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:27.279402 12115 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:27.252506 12413 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.283888 12413 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:27.279407 12023 consensus_queue.cc:260] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:27.285553  8133 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1876) took 1248 ms (client timeout 1535 ms). Trace:
I20260709 13:14:27.285960 12023 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:27.286007  8133 rpcz_store.cc:269] 0709 13:14:26.036989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.037474 (+   485us) service_pool.cc:224] Handling call
0709 13:14:27.285102 (+1247628us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b5400408dd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:27.285347 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373700352}
I20260709 13:14:27.297134  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:27.299147  8211 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:27.299511  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:27.299885 12354 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:27.306842  8130 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:27.312251  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:27.290138 12258 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.313490  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.314599  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:27.315474  8062 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:27.317184  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:27.318122  8058 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:27.318637  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:27.319031  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:27.321460  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 1.3309e+06us
I20260709 13:14:27.321838 12354 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.322422 12470 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.329254 12273 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.329892 12273 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.328330  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:27.327088  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:27.333775  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:27.335206  8059 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:27.338578  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.339056  8058 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:27.339576  8063 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:27.341780  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:27.342403  8211 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:27.342833  8059 raft_consensus.cc:2410] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:27.341616 12086 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:27.345953 12283 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.340478  8210 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:27.346084  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:27.346590 12283 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.347569  8054 raft_consensus.cc:2410] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:27.347757 12307 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.348366 12470 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:27.348539 12307 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.349015 12470 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.350572  8055 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:14:27.351681 12423 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:27.351807 12413 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.353171  8057 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:27.347800 12273 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.349570 12283 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.363516  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:27.366976  8026 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:27.353194 12298 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.367568  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.372031  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.372295 12423 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:27.372841  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.374212 12468 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:27.374810 12354 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:27.375902  8062 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:27.375200 12307 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:27.404198  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:27.382293  8726 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:27.404423  8728 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:27.410876  8724 raft_consensus.cc:2393] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:27.410769  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.360545 12426 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:27.419610  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:27.420567  8062 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:14:27.422199  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.422960  8176 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:27.423523  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:27.424096  8059 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:27.427209  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.425325  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:27.430227  8060 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:27.367008 12450 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:14:27.442795 12426 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:27.443933  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.443773  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.444214 12432 raft_consensus.cc:697] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:27.444918  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.415012 12468 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.421913 12423 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.447232  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.422623 12258 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:27.412599 12422 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.448199  8138 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:14:27.422046 12354 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.450205 12437 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:27.450373  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.442984  8054 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:27.451046  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:27.451108  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.451674  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.451855  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:27.453284  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:27.411981 12413 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.455180  8130 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:27.459522  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.463873  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:27.466532  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:27.468827  8723 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:27.458231  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.506213  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.506822  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.507352  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.507973  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:27.508416  8133 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 12: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:27.469839 12423 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:27.509116  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:27.509317  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.462895 12446 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.510313  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.510563 12446 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.511617  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:27.512576 12446 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.512745  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:27.462795  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 84030us
I20260709 13:14:27.514222  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:27.462751  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:27.514865  8063 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:27.516320  8726 raft_consensus.cc:2393] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:27.516490  8063 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:14:27.512627  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:27.429199 12470 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:27.517113  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:27.518060  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.456571  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:27.519399  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:27.519979 12355 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:27.520404 12355 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.520634  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.519300 12430 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:27.521224  8136 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:27.508500  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.520756 12355 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:27.462955  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:27.521381 12430 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.459559  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.879637  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.882767  8136 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:27.477756  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.487526  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:27.488189  8059 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:27.485965  8727 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:27.874428  8101 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:27.488302  8134 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:27.518155  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.466758  8132 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:27.488497  8211 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
W20260709 13:14:27.490957  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 68191us
I20260709 13:14:27.718035  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:27.486583  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 75225us
I20260709 13:14:27.868698  8060 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:27.499423  8054 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:27.482126  8138 pending_rounds.cc:85] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 5
I20260709 13:14:27.471546  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:27.675628  8058 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
W20260709 13:14:27.486527  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 61160.3us
I20260709 13:14:27.456539  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.900127  8135 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:27.476454  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:27.476074  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:27.627604  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 260863us
I20260709 13:14:27.523458  8062 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:27.738111  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1626) took 1409 ms. Trace:
I20260709 13:14:27.889210  8129 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:27.476868  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:27.476006  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.903090  8204 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:14:27.903616  8206 raft_consensus.cc:2410] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 13:14:27.810570  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.458055  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:27.905141  8131 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:27.914583  8207 raft_consensus.cc:2368] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12.
I20260709 13:14:27.478729  8212 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:14:27.487892  8728 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
W20260709 13:14:27.859515 12494 log.cc:927] Time spent T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.396s	user 0.004s	sys 0.000s
I20260709 13:14:27.480620  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.478022  8205 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:14:27.919736  8729 raft_consensus.cc:2393] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:27.780719  8027 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:27.918565  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:27.908752  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 382107us
I20260709 13:14:27.902091  8061 rpcz_store.cc:276] 0709 13:14:26.328674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.328798 (+   124us) service_pool.cc:224] Handling call
0709 13:14:26.329104 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:27.060926 (+731822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:27.705180 (+644254us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b54001e8cd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:27.738019 (+ 32839us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1623225088}
I20260709 13:14:27.921859  8724 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:27.767225 12446 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.458973  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:27.917047  8208 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:27.666217 12450 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.754007 12228 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.499876 12413 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:27.924500  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.656679 12191 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.670467 12258 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:27.917454  8138 pending_rounds.cc:107] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:14:27.943921  8138 raft_consensus.cc:2889] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:27.929297  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:27.473119 12426 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:27.874398 12115 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:27.931116  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:27.930224  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.932081  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:27.468237 12468 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:27.934768  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:27.777305 12159 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:27.937999  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:27.666325 12470 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:27.486783 12434 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:27.938628  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:27.930737 12423 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:14:27.939814 12369 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.940223 12464 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:27.940733  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:27.957099  8054 raft_consensus.cc:2410] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:14:27.940917  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.942466 12226 raft_consensus.cc:3055] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:27.937849  8133 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:27.919203 12154 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:27.946197  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.946318  8176 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:27.473294 12378 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:27.966006  8727 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:14:27.961122  8056 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:27.948226  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:27.966856  8203 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:27.948794  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.954250  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.474642 12422 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.482402 12354 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.500769 12383 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.473886 12229 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.949800  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:27.964293  8207 raft_consensus.cc:2410] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:27.956875  8137 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:27.962080  8696 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:27.974144  8205 raft_consensus.cc:1813] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Handling vote request from an unknown peer 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:27.972407  8209 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:27.482775 12437 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:27.972569 12231 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.027s	sys 0.044s
I20260709 13:14:27.979924  8205 raft_consensus.cc:2410] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:14:27.979568  8130 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:27.965550 12154 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:27.974862  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 47254.6us
I20260709 13:14:27.943482 12228 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.991866 12154 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:27.965693 12226 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:27.967501 12298 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.466240 12432 consensus_queue.cc:237] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.986187  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:27.995680  8210 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:27.991781  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:27.965749 12115 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.954391 12446 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:27.996748  8176 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:28.003929  8057 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:28.004304  8136 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:28.005283 12464 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:27.987720 12413 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:14:28.010476  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 289282us
I20260709 13:14:28.011093 12495 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:28.012185 12495 consensus_queue.cc:1059] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:28.010546 12413 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:28.013993  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:28.014446 12413 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:28.014883  8131 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:28.016863  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:28.016814 12428 consensus_queue.cc:1059] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:28.002264  8132 raft_consensus.cc:2393] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:14:27.991355 12231 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.027s	sys 0.044s
I20260709 13:14:28.019716  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:28.020358  8206 raft_consensus.cc:2410] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:28.023415  8137 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:27.960911 12450 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:28.024770 12438 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:28.024595 12166 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:14:28.019376 12369 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:28.025291  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 52302.8us
I20260709 13:14:28.025416 12159 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:28.040707 12301 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:28.041052  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:28.041671  8206 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:27.979270 12191 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.041097 12159 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:28.044171  8104 leader_election.cc:400] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12.
I20260709 13:14:28.044512  8104 leader_election.cc:403] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:14:28.044750 12159 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.045565  8104 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:28.046523 12423 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:28.032197  8101 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:28.048218  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 119749us
I20260709 13:14:27.995680 12378 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:28.052167  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:27.995390 12470 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:28.054131  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 73998.6us
I20260709 13:14:28.054872  8176 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:28.058315 12301 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:28.065172 12301 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:28.065593 12301 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:28.066668 12182 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:28.070077  8101 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:28.071969  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:28.072191 12191 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:28.073060  8210 raft_consensus.cc:2410] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:14:28.073975  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:28.072885  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:28.074364  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:28.051229 12231 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:28.074676  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:28.077909  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:28.051342 12407 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:28.027957 12298 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.079802  8135 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:28.081375  8205 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:28.089278  8061 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:28.100459  8207 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:28.103133  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:28.129658  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:28.130385  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:28.131189  8205 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:28.079607 12228 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:28.139911  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:28.139760  8058 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:28.098695 12426 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:28.094110 12182 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:28.142604  8207 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:28.093067 12136 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12.
I20260709 13:14:28.059762 12226 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.102217 12354 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:28.150204  8054 raft_consensus.cc:2393] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 12: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:14:28.153446  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 92795.1us
I20260709 13:14:28.091279 12423 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:14:28.153326 12407 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.133482 12430 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:28.169749  8130 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:28.162818  8056 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:28.171456  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:28.069392 12258 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.107072 12231 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.202497 12422 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:28.208935 12422 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:28.202139 12430 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:28.209973 12430 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:28.210379 12430 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:28.211634 12231 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:28.215058 12498 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.150s	user 0.003s	sys 0.000s
W20260709 13:14:28.217072 12502 log.cc:927] Time spent T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.153s	user 0.000s	sys 0.002s
I20260709 13:14:28.218780 12430 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:28.221094  8210 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:14:28.221892 12430 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:28.223984 12301 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:28.226073 12301 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:28.173195 12407 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.227710 12430 consensus_queue.cc:1059] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:28.228653  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:28.229429  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:28.230070  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:28.232589 12355 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.200423 12229 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:28.236181  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.237449  8211 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:28.204378 12328 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:28.245379  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:28.222077  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:28.284157  8723 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:28.253187  8204 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:28.237682 12154 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.242206 12373 raft_consensus.cc:3055] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:28.249151  8208 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
W20260709 13:14:28.322305  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 305802us
I20260709 13:14:28.325053  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:28.327387  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 407177us
I20260709 13:14:28.335340  8129 raft_consensus.cc:2368] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13.
I20260709 13:14:28.336390  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:28.336582  8057 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:28.389978  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:28.250483 12191 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:14:28.289085 12468 raft_consensus.cc:2764] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:14:28.289415 12328 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.317631 12450 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:28.321882 12373 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:28.335845 12428 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:28.386919  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:28.390822  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 336014us
I20260709 13:14:28.325876 12226 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:28.402546  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:28.360872 12470 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:28.385692  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:28.407284  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 313479us
I20260709 13:14:28.379735 12378 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:28.409361  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:28.413161  8135 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:28.413394  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:28.413527  8057 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
W20260709 13:14:28.416042  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 668024us
I20260709 13:14:28.414443  8058 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:28.419103  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:28.421790  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:28.424456  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.402995 12383 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:28.404487 12328 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:28.425122  8059 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
I20260709 13:14:28.432096  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:28.434105  8208 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:14:28.435397  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:28.436388  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:28.443867  8057 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:28.443711  8135 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:28.449697 12495 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.450304 12495 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:28.469558  8137 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 8.
W20260709 13:14:28.470463  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 74756.8us
I20260709 13:14:28.472637  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:28.484007  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 81580us
I20260709 13:14:28.482666  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:28.488241  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 72675.5us
I20260709 13:14:28.488860  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:28.489526  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:28.490095  8135 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:28.432026 12115 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:28.489702  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:28.494239  8133 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:28.495116  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:28.432623 12426 raft_consensus.cc:697] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:28.417781 12428 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:28.501927  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:28.444451 12503 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:28.512068  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 653) took 1026 ms. Trace:
I20260709 13:14:28.469861 12504 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.509358  8130 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:28.513847  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1454) took 1030 ms. Trace:
I20260709 13:14:28.489392  8101 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:28.514556  8059 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:28.613862 12407 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:28.614850  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50669.3us
W20260709 13:14:28.615173  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 49963.5us
I20260709 13:14:28.610498 12426 consensus_queue.cc:237] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:28.565402  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 69162.5us
I20260709 13:14:28.564504  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:28.564805  8133 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:14:28.561836  8058 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
W20260709 13:14:28.713582  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 224255us
I20260709 13:14:28.520326  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:28.715267  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 303694us
I20260709 13:14:28.619167  8058 pending_rounds.cc:85] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 9
I20260709 13:14:28.717504  8058 pending_rounds.cc:107] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:14:28.717922  8058 raft_consensus.cc:2889] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:28.612885 12413 consensus_meta.cc:287] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.598s	user 0.003s	sys 0.005s
I20260709 13:14:28.499173 12136 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:28.722554  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:28.716442  8136 rpcz_store.cc:276] 0709 13:14:27.483895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.904573 (+420678us) service_pool.cc:224] Handling call
0709 13:14:28.513657 (+609084us) spinlock_profiling.cc:243] Waited 78.7 ms on lock 0x7b5400042ed4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:28.513785 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181078912}
I20260709 13:14:28.617707  8026 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:28.511988 12503 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.733966  8027 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:28.612965 12182 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:14:28.523941  7949 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.447s	user 0.016s	sys 0.013s
I20260709 13:14:28.613195 12154 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:28.494354 12509 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.746210  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:28.739554  8696 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:28.737438 12413 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:28.762786 12229 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:28.767277  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:28.750085  8062 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:28.805513  8057 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:28.723017 12383 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:28.751183 12513 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:14:28.751730 12515 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.570101 12166 consensus_queue.cc:1059] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.205s
I20260709 13:14:28.514847 12258 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:28.738763 12503 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:28.837023  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 423674us
I20260709 13:14:28.837769  8104 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:28.803925  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:28.770576  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 153057us
I20260709 13:14:28.717090 12373 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.736795 12355 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:28.841866  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 118337us
W20260709 13:14:28.816547  8205 consensus_meta.cc:287] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.681s	user 0.008s	sys 0.000s
I20260709 13:14:28.744562  8176 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:28.512429  8134 rpcz_store.cc:276] 0709 13:14:27.485177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.976284 (+491107us) service_pool.cc:224] Handling call
0709 13:14:28.004384 (+ 28100us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.058548 (+ 54164us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:14:28.402372 (+343824us) log.cc:844] Serialized 14 byte log entry
0709 13:14:28.466703 (+ 64331us) log.cc:844] Serialized 14 byte log entry
0709 13:14:28.511720 (+ 45017us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:28.511769 (+    49us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:28.511837 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.511908 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.512015 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":475049,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":56,"raft.queue_time_us":294367,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":154,"thread_start_us":336520,"threads_started":3}
W20260709 13:14:28.824975  8203 consensus_meta.cc:287] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.839s	user 0.006s	sys 0.006s
W20260709 13:14:28.826165  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 86684.8us
I20260709 13:14:28.829979  8130 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
W20260709 13:14:28.830910  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 215219us
I20260709 13:14:28.833030  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:28.844703  8205 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:28.844727  8203 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:28.836614  8207 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:28.839673  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:28.845450  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2930) took 1097 ms. Trace:
I20260709 13:14:28.845475  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2916) took 1365 ms. Trace:
I20260709 13:14:28.845475  8726 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:28.802203 12407 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:28.825838 12383 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.867790 12383 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:28.868430 12369 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:28.869329  8057 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:28.846159  8203 rpcz_store.cc:276] 0709 13:14:27.479793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.936503 (+456710us) service_pool.cc:224] Handling call
0709 13:14:28.845421 (+908918us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:28.871553  8027 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:28.873116  8725 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:28.873440  8027 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:28.875065  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:28.875746  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 636) took 1211 ms. Trace:
I20260709 13:14:28.875821  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1895) took 1399 ms. Trace:
I20260709 13:14:28.876741  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:28.876163  8063 rpcz_store.cc:276] 0709 13:14:27.664897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.716828 (+ 51931us) service_pool.cc:224] Handling call
0709 13:14:27.751881 (+ 35053us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:27.776634 (+ 24753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:27.776767 (+   133us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:27.949586 (+172819us) log.cc:844] Serialized 494 byte log entry
0709 13:14:28.412758 (+463172us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:28.412792 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.412848 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:28.875561 (+462713us) raft_consensus.cc:1685] finished
0709 13:14:28.875580 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.875693 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":415796,"threads_started":1,"wal-append.queue_time_us":753219}
I20260709 13:14:28.877622  8726 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:28.878815  8730 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:28.879335 12307 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:28.879748 12513 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.880167 12513 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.881311  8730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 657) took 1424 ms. Trace:
I20260709 13:14:28.881755 12513 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:28.882210 12513 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:28.879787 12307 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:28.882599 12513 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:28.883394  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:28.883575 12182 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:28.883637 12513 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.885015 12513 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:28.885324 12182 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:28.885412 12438 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:28.885797 12182 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.885888 12438 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:28.887270  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 49580.2us
I20260709 13:14:28.856920  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:28.847424 12378 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:14:28.858922 12413 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:28.859463  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:28.936882  8724 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:28.833251 12115 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:28.851806 12258 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.863435  8696 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:28.865829  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 656) took 1379 ms. Trace:
I20260709 13:14:28.865939 12446 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:28.835458 12515 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:28.866601  8026 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:28.867112 12407 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:28.854506  8059 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:28.881675  8730 rpcz_store.cc:276] 0709 13:14:27.456912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.458516 (+  1604us) service_pool.cc:224] Handling call
0709 13:14:27.458996 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.879248 (+1420252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.881087 (+  1839us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54004d21d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:28.881234 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3264637696}
W20260709 13:14:28.945250  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 78722.9us
I20260709 13:14:28.945943  8026 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:28.854633 12495 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:28.888696  8104 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:28.876294  8129 rpcz_store.cc:276] 0709 13:14:27.476772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.917933 (+441161us) service_pool.cc:224] Handling call
0709 13:14:28.875635 (+957702us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b540015b654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:28.875770 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854203648}
I20260709 13:14:28.851079  8205 rpcz_store.cc:276] 0709 13:14:27.747638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.100134 (+352496us) service_pool.cc:224] Handling call
0709 13:14:28.845382 (+745248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:28.919404  8203 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:28.926141 12298 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:28.886264 12182 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:28.919951  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:28.926605 12253 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:14:28.927399 12470 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:28.869268 12354 raft_consensus.cc:3055] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:28.882612  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:28.971650  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 89100.9us
W20260709 13:14:28.935752  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 191282us
I20260709 13:14:28.942840  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:28.742410 12504 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.888227 12136 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:28.945619  8721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 725) took 1044 ms. Trace:
I20260709 13:14:28.944936 12407 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.950843 12378 consensus_queue.cc:1243] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:14:28.951557  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 62929.3us
I20260709 13:14:28.952782 12446 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:28.938077  8131 rpcz_store.cc:276] 0709 13:14:27.485934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.015729 (+529795us) service_pool.cc:224] Handling call
0709 13:14:28.016140 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:28.030839 (+ 14699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:28.030875 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:28.053501 (+ 22626us) log.cc:844] Serialized 26 byte log entry
0709 13:14:28.405140 (+351639us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:28.405177 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.405239 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:28.865616 (+460377us) raft_consensus.cc:1685] finished
0709 13:14:28.865635 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.865756 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7796,"raft.queue_time_us":11504,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":8928,"thread_start_us":300274,"threads_started":1,"wal-append.queue_time_us":793797}
I20260709 13:14:28.831604 12509 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.940140 12115 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.948391  8026 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:28.869285 12328 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.918627 12369 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:28.990489 12328 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:28.977771  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:28.991694  8104 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:28.978413  8101 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:28.970322 12422 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:28.982228 12506 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:28.975142 12298 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:28.977577 12407 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:28.983119 12503 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:28.999145  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:28.883430 12307 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:28.992990 12115 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:28.996331 12136 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:29.002599 12307 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:29.003085 12136 raft_consensus.cc:697] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:29.004031 12136 consensus_queue.cc:237] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:28.996301 12533 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:28.966228 12495 consensus_queue.cc:1059] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:14:28.979595 12446 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:28.987195 12515 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:28.988801 12258 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:28.995071  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:29.012015  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.977797 12430 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:28.970598 12354 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:28.994750 12298 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:28.999282  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.973187  8721 rpcz_store.cc:276] 0709 13:14:27.901055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.940034 (+ 38979us) service_pool.cc:224] Handling call
0709 13:14:27.940464 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.005529 (+ 65065us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:28.343057 (+337528us) log.cc:844] Serialized 14 byte log entry
0709 13:14:28.469738 (+126681us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:28.469787 (+    49us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:28.469850 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.927301 (+457451us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b1800373540. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ef72a6 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419
0709 13:14:28.945411 (+ 18110us) spinlock_profiling.cc:243] Waited 63.7 ms on lock 0x7b54007611d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:28.945449 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.945556 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":406707,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":400,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":15471,"spinlock_wait_cycles":200046592,"thread_start_us":385534,"threads_started":2,"wal-append.queue_time_us":144933}
I20260709 13:14:28.998260  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:28.978866 12438 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:29.001550  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 115054us
I20260709 13:14:29.001951 12531 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:28.980986 12191 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
W20260709 13:14:29.008653  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 60309.2us
I20260709 13:14:28.997510  8027 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:28.977664 12428 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:29.011876  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:28.994814  8061 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:29.010203  8176 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:28.996407 12328 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:14:28.998754  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 135405us
I20260709 13:14:29.008569 12369 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:29.037500  8104 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:28.979525 12413 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:29.011114 12446 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:29.025887  8063 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:29.028273  8026 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:29.044482  8026 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:29.032306  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:29.034142 12354 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.034919 12432 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:29.023972 12298 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:29.040834  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:29.032785 12328 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.051712 12328 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:29.051702 12533 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:29.053133 12533 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.053496 12533 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:29.048120 12426 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:29.058039 12438 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:29.060875 12428 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:29.061297 12428 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.061638 12428 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:29.062525 12422 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:29.063627  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:29.042692  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:29.032853 12506 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:29.066767 12506 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.067121 12506 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:29.067553 12166 consensus_queue.cc:1059] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:29.068357 12166 raft_consensus.cc:3055] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:29.068670 12166 raft_consensus.cc:740] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:29.069464 12166 consensus_queue.cc:260] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.070832  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:29.071504  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:29.071866 12503 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:29.072454 12503 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:29.072780  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:29.073278  8026 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:29.073848 12503 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:29.073937  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:29.070261  8059 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:29.075002  8101 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:29.078797  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 192699us
I20260709 13:14:29.074123 12470 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:29.085688  8203 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:29.041477 12413 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:29.090765  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.092805  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:29.106952  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.108546  8205 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 13: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:29.078368 12229 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:29.105978 12413 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:29.117857 12229 raft_consensus.cc:697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:29.118889 12229 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.119918  8104 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:29.120901 12136 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:29.121387  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:29.121472  8104 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:29.074692 12464 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:29.082574 12470 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:29.122581 12136 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:29.122969 12470 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:29.124941 12413 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:29.112110  8026 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:29.091171 12536 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:29.082175 12426 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:29.112797  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.126672 12426 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.127039 12426 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:29.078971 12513 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:29.134595 12513 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:29.135499 12464 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:29.135960 12464 raft_consensus.cc:697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:29.136828 12464 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:29.113633  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.138724  8210 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
W20260709 13:14:29.139050 12521 log.cc:927] Time spent T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.408s	user 0.004s	sys 0.000s
I20260709 13:14:29.139721  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:29.140331  8210 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:29.114766  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.142928  8204 raft_consensus.cc:2410] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 13 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:14:29.143745  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.144249  8204 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:29.145015  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.145999  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.148118  7960 catalog_manager.cc:5697] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:29.115414  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.151875  8211 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:29.152005 12506 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:29.152691  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.153659  8211 raft_consensus.cc:2410] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:14:29.153446 12182 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:29.154052 12506 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:29.082062 12226 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:29.127012  8205 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:29.155593 12437 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:29.155668  8206 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:29.156492 12383 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.159993 12373 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:29.160156 12166 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:29.194862  8212 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:29.195436  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:29.162223 12538 raft_consensus.cc:3055] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:29.163064 12534 raft_consensus.cc:3055] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:29.163184 12509 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:29.163686  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:29.165483  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44081.9us
I20260709 13:14:29.206030  8104 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:29.166281  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:29.206144  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:29.206943  8725 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
W20260709 13:14:29.114427  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 80320.6us
I20260709 13:14:29.197306  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:29.200222  8204 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:29.113723 12541 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:29.154842  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.150014  7960 catalog_manager.cc:5697] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 7 to 10, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 10 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:29.181927  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 69892.8us
I20260709 13:14:29.181447  8203 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:29.185817  8176 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:29.184368  8129 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:29.189198  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:29.190474  8138 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:29.198128  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:29.209297  8209 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:29.198989  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:29.210865  8722 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:29.197882  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:14:29.168435  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 94592.9us
I20260709 13:14:29.209573  8026 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:29.199004  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:29.211848  8723 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:29.208051  8696 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:29.150007 12115 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:29.200011  8724 raft_consensus.cc:2368] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14.
I20260709 13:14:29.212673  8026 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:29.213069 12542 log.cc:927] Time spent T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:14:29.219194  8026 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:29.223305  8203 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
W20260709 13:14:29.224408  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 144980us
I20260709 13:14:29.214464 12544 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:29.225847 12413 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:29.225926 12544 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:29.193729 12226 raft_consensus.cc:697] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:29.226461 12544 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:29.226866 12544 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:29.227239 12226 consensus_queue.cc:237] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.171412 12533 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:29.231925 12533 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:29.232913 12328 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:29.234530 12544 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.225656  7960 catalog_manager.cc:5697] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 2 to 8, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 8 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:29.230180 12383 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:29.237360 12470 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.244836  8104 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:29.224718 12437 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:29.249603  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.253636  8210 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:29.198607 12538 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:29.209661 12534 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:29.255553  8721 raft_consensus.cc:2368] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10.
W20260709 13:14:29.247869  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 190265us
I20260709 13:14:29.241791 12258 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.224234 12513 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:14:29.266031  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 43960.1us
I20260709 13:14:29.266250  8102 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:29.223996 12506 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:29.267552  8029 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:29.268612  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50569us
I20260709 13:14:29.271739  7960 catalog_manager.cc:5697] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 7 to 8, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 8 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:29.269347  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:29.273155  8061 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:29.277855  8104 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:29.279721  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:29.254783 12522 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:29.283669  8029 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:29.256348 12438 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:14:29.264158 12437 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:14:29.286546  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 67404us
I20260709 13:14:29.295883  8026 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:29.297238  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:29.294831 12226 raft_consensus.cc:2764] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:14:29.299008 12226 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:29.298687 12534 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.299410 12226 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:29.299912 12226 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:29.300064 12534 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:29.302413 12509 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:29.302970 12509 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:29.302690 12538 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:29.303876 12538 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:29.303804 12509 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:29.232929 12407 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.307103 12423 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:29.256449 12523 consensus_queue.cc:1059] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:14:29.307646 12423 raft_consensus.cc:697] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:29.308084 12407 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:29.308833 12407 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:29.308492 12423 consensus_queue.cc:237] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.309435 12407 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:29.259953  7949 maintenance_manager.cc:643] P e3b771f7515a4bb7beb66bc429c57201: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 6.211s	user 0.188s	sys 0.070s Metrics: {"bytes_written":29514,"cfile_init":1,"dirs.queue_time_us":138374,"dirs.run_cpu_time_us":368,"dirs.run_wall_time_us":918,"drs_written":1,"lbm_read_time_us":440,"lbm_reads_lt_1ms":4,"lbm_write_time_us":50719,"lbm_writes_1-10_ms":2,"lbm_writes_10-100_ms":2,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":63,"thread_start_us":501455,"threads_started":2,"wal-append.queue_time_us":445436}
I20260709 13:14:29.298358 12422 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:14:29.289974  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 104221us
I20260709 13:14:29.318138  8725 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:29.285763 12378 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:29.308048 12523 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:29.333380  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:29.334793  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 80247.7us
I20260709 13:14:29.334225  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.336254  8210 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:29.328495  8063 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:29.329465 12546 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:29.286440 12522 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.330121  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:29.331102  8209 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:29.332310  8060 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:29.339851  8730 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:29.320947  8061 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:29.311041 12437 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:29.340813  8722 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:29.275336 12258 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.350149  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.348619  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:29.350785  8203 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:29.348619  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.295154 12506 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.350435  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:29.361656  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 106818us
I20260709 13:14:29.368300  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:29.370734  8204 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:29.369174  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:29.371363  8210 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:29.373515 12115 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:29.342099 12522 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:29.341908 12428 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:29.380910 12428 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:29.347653 12422 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:29.382215 12422 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:29.347620 12182 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:29.346694 12546 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.384534 12546 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:29.385783  8204 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:29.388562  8204 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:14:29.389672  8204 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:29.390564 12159 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.371843 12298 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:29.391077 12159 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.391778 12258 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:29.368952  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:29.394136  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:29.386721  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:29.387054  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:29.395221  8205 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:29.387050  8210 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:29.396435  8723 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:29.400843  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:29.402083  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 54083.7us
I20260709 13:14:29.403213  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:29.333739 12523 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:29.392835 12354 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:29.407079 12470 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:29.407581 12544 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:29.408138  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:29.408486  8210 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:29.408708  8724 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:29.408957  7960 catalog_manager.cc:5697] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 5 to 7, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 7 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
W20260709 13:14:29.410604  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 68354.7us
I20260709 13:14:29.410810  7960 catalog_manager.cc:5697] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 6 to 8, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 8 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:29.412431  7960 catalog_manager.cc:5697] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:29.413762  7960 catalog_manager.cc:5697] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 8 to 10, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 10 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:14:29.467204  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 55254us
I20260709 13:14:29.441579  8730 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
W20260709 13:14:29.482123  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:29.447633  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 43601.1us
I20260709 13:14:29.428948  8696 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:29.493566  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 64697.1us
I20260709 13:14:29.493772  8205 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:29.493366  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:29.500535  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:29.500758  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:29.501096  8210 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:29.502873  8209 pending_rounds.cc:85] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 7
I20260709 13:14:29.503223  8209 pending_rounds.cc:107] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:14:29.503528  8209 raft_consensus.cc:2889] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:29.503903  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:29.430132  7960 catalog_manager.cc:5697] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:29.507215  7960 catalog_manager.cc:5697] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 6 to 8, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 8 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:29.509105  7960 catalog_manager.cc:5697] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 6 to 8, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 8 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:29.455592  8203 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:29.427155  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:29.466463  7950 maintenance_manager.cc:419] P e3b771f7515a4bb7beb66bc429c57201: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9652 bytes on disk
I20260709 13:14:29.546810  8211 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:29.541745  8102 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:29.547250  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 263679us
I20260709 13:14:29.548476  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:14:29.538602  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 111515us
I20260709 13:14:29.538498  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:29.465513 12515 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:29.464792 12301 raft_consensus.cc:3055] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:29.559279  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:29.557127  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:29.559752  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:29.560635  8129 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:29.556221  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:29.561762  8204 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 13: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:29.527146  8210 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:29.447700 12159 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:29.494855 12539 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:29.554163  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:29.554584  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:29.567425  8203 raft_consensus.cc:2410] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:14:29.422667 12550 raft_consensus.cc:2764] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:14:29.554797  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:29.572531  8138 raft_consensus.cc:2410] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:14:29.418308 12426 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:29.557614  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.510538 12428 raft_consensus.cc:2764] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:14:29.467427 12191 consensus_queue.cc:1059] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 13:14:29.494752 12506 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:29.518044 12533 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:29.563930  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:29.533844 12182 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:29.565876  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:29.566375  8136 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:29.801365 12407 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:14:29.772388 12544 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:29.465673 12166 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:14:29.423432 12226 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:29.584129  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 286977us
W20260709 13:14:29.584600  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.557094  7960 catalog_manager.cc:5697] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 7 to 9, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 9 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
W20260709 13:14:29.607093  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.544026  8104 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:29.797349  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:29.797658  8727 raft_consensus.cc:1813] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Handling vote request from an unknown peer 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:29.771389  8133 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:14:29.876169  8727 raft_consensus.cc:2410] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 13:14:29.771441  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 187351us
W20260709 13:14:29.770815  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 216708us
W20260709 13:14:29.786988  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 253854us
W20260709 13:14:29.771641  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 197793us
I20260709 13:14:29.877525  8696 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:29.877544  8102 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:29.560082 12515 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.866182  7949 maintenance_manager.cc:643] P e3b771f7515a4bb7beb66bc429c57201: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.297s	user 0.002s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":376,"lbm_reads_lt_1ms":4}
I20260709 13:14:29.569590  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:29.558864 12328 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:29.800704  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:29.881536  8137 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:14:29.882493  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:29.873092  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.885038  8134 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 11. Ops: []
W20260709 13:14:29.885172  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 341221us
I20260709 13:14:29.873883  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:29.876426  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 310609us
I20260709 13:14:29.878566  8029 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:29.567415 12301 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:29.879546  8026 leader_election.cc:400] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13.
I20260709 13:14:29.570660 12258 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:29.564985 12115 consensus_queue.cc:1059] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.315s
I20260709 13:14:29.891338 12548 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.466719 12307 raft_consensus.cc:2955] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Committing config change with OpId 9.8: config changed from index 5 to 8, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:29.898492 12545 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:29.939859  8054 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:29.902693 12423 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:29.903347  8176 leader_election.cc:400] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14.
I20260709 13:14:29.907908  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2931) took 2160 ms. Trace:
I20260709 13:14:29.945586  8207 rpcz_store.cc:276] 0709 13:14:27.747961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.164149 (+416188us) service_pool.cc:224] Handling call
0709 13:14:28.176053 (+ 11904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.845658 (+669605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.881852 (+1036194us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b540019e6d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:29.907835 (+ 25983us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1518766336}
I20260709 13:14:29.771200 12159 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:29.883953 12166 consensus_queue.cc:1243] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:29.958938  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:29.959661  8063 raft_consensus.cc:2393] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:29.925772  8104 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:29.929555  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:29.871907 12464 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.066s	sys 0.159s
W20260709 13:14:29.938757  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 51596.1us
W20260709 13:14:29.963369  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 85890.3us
I20260709 13:14:29.963740  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:14:29.906121 12522 consensus_meta.cc:287] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.527s	user 0.001s	sys 0.007s
I20260709 13:14:29.941447  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:30.000247  8060 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
W20260709 13:14:29.941741  8724 consensus_meta.cc:287] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.532s	user 0.007s	sys 0.002s
I20260709 13:14:30.001147  8724 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:29.941695  8061 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 11. Ops: []
I20260709 13:14:29.955034  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:30.006990 12547 log.cc:927] Time spent T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.678s	user 0.010s	sys 0.000s
I20260709 13:14:30.007539 12426 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:30.010552 12554 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.011080 12554 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.920064  7960 catalog_manager.cc:5697] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 7 to 8, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 8 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:29.886127 12515 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:29.797904 12470 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.109s
I20260709 13:14:29.879679 12509 raft_consensus.cc:2764] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:14:30.021287 12509 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:30.021660 12509 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:30.022006 12509 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:30.030236 12253 raft_consensus.cc:3055] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:30.030692 12253 raft_consensus.cc:740] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:30.031550 12253 consensus_queue.cc:260] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:30.036559 12437 consensus_meta.cc:287] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.654s	user 0.006s	sys 0.000s
I20260709 13:14:30.036948 12437 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:29.963258 12464 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.067s	sys 0.159s
I20260709 13:14:30.055297  8104 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:29.960790  8062 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:29.964529  8102 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:29.965443  8056 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:29.918118 12258 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.917938 12373 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:29.940708 12369 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:14:30.063093  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 157333us
I20260709 13:14:29.958029 12557 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:30.016999  7960 catalog_manager.cc:5697] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 3 to 9, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 9 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:30.019761 12531 raft_consensus.cc:3055] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:29.960446  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:30.079955  8693 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:30.056906  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 153684us
I20260709 13:14:29.941159 12434 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:30.013733 12446 consensus_meta.cc:287] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.959s	user 0.008s	sys 0.000s
I20260709 13:14:30.087155  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:30.088466  8059 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:29.939043 12548 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:29.796962 12523 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.013329 12328 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:30.111713  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:30.013471 12191 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
W20260709 13:14:30.039055 12546 consensus_meta.cc:287] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.654s	user 0.009s	sys 0.000s
I20260709 13:14:30.082194  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:30.117892  8063 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:30.082993  7960 catalog_manager.cc:5697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 4 to 5, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 5 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:29.573203 12354 consensus_queue.cc:1059] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.340s
I20260709 13:14:29.963160 12544 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:30.122537 12544 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:30.124521 12544 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:30.091221  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:30.091483  8176 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:30.063800 12258 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:30.126045 12298 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:30.127027 12504 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:30.064216 12378 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:30.064272 12136 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:14:30.112032 12533 consensus_queue.cc:1059] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:14:30.092308 12369 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:30.133262 12369 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.112829  8056 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:30.135752 12558 raft_consensus.cc:3055] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:30.136185 12558 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:30.136961 12558 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:30.077044  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 113406us
I20260709 13:14:30.113775  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:30.015599 12522 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:30.087867 12407 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.090646 12551 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:30.094010 12557 raft_consensus.cc:697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:30.092981 12446 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.079914 12531 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:30.114131  8055 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:30.063902 12373 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.040661 12426 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:14:30.121652 12546 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.138942  8134 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:14:30.138702  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:30.139089  7960 catalog_manager.cc:5697] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 2 to 9, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 9 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:30.139671  8132 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:14:30.140321  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 48890.9us
I20260709 13:14:30.161152  8176 leader_election.cc:400] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10.
I20260709 13:14:30.161422  8176 leader_election.cc:403] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:14:30.143867  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1782) took 1028 ms. Trace:
I20260709 13:14:30.147223  8057 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:30.150985  8061 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:30.152459  8055 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:30.166175  8058 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:14:30.162936  8211 rpcz_store.cc:276] 0709 13:14:29.115864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.154196 (+ 38332us) service_pool.cc:224] Handling call
0709 13:14:30.143691 (+989495us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b5400189554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:30.143798 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":745659136}
W20260709 13:14:30.154858  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 277386us
I20260709 13:14:30.171027  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:30.171895  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:30.186164  8204 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:30.145268 12534 consensus_meta.cc:287] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.845s	user 0.003s	sys 0.007s
I20260709 13:14:30.172505  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:30.174134  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:30.142966  8063 raft_consensus.cc:2955] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 5 to 8, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:30.183475  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:30.181309  8056 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
W20260709 13:14:30.149116 12538 consensus_meta.cc:287] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.845s	user 0.011s	sys 0.000s
I20260709 13:14:30.237030  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:30.138320 12558 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:30.139369 12354 raft_consensus.cc:3055] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:30.148443 12446 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:30.186107  8203 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:30.236464  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:30.238847  8211 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:14:30.244721  8207 raft_consensus.cc:2410] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 13:14:30.244758  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 89221.2us
I20260709 13:14:30.245375  8211 pending_rounds.cc:85] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 7
I20260709 13:14:30.133793 12258 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:30.237886  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:30.151346 12373 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:30.153911 12378 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:30.239951  8060 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:30.140496 12428 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:30.078133 12301 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.091042 12434 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.255858  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:30.247367  8212 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:30.258757  8054 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:30.259887  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.257160  8057 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:30.144943 12407 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:30.240557 12545 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:30.148624 12533 raft_consensus.cc:3055] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:30.142405 12548 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:30.241750  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 103119us
I20260709 13:14:30.143148 12383 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:30.243130  8210 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:30.152240 12253 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:30.244724  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:30.296259  8203 raft_consensus.cc:2376] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13. Re-sending same reply.
I20260709 13:14:30.177153 12166 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:30.141166 12522 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:30.245812  8211 pending_rounds.cc:107] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:14:30.304545  8211 raft_consensus.cc:2889] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:30.092752 12506 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:30.182258 12468 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10.
I20260709 13:14:30.242024 12437 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:30.270154  8062 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:14:30.272423  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:30.271371 12446 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:30.279613 12556 log.cc:927] Time spent T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.148s	user 0.003s	sys 0.000s
I20260709 13:14:30.242311 12354 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:30.282027  8727 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 819) took 2052 ms (client timeout 2386 ms). Trace:
I20260709 13:14:30.298976  8204 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:30.241081  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:30.301302  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 55215.9us
I20260709 13:14:30.299919  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:30.273224  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:30.346050  8208 raft_consensus.cc:2376] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12. Re-sending same reply.
W20260709 13:14:30.346078  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 46232.9us
I20260709 13:14:30.317485  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:30.347106  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:30.286357 12533 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:30.289932 12136 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.325340  8054 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:30.327384 12545 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:30.362102  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:30.362129  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 397635us
I20260709 13:14:30.328483 12373 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.334180 12258 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:30.334923 12378 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.336426 12568 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.285458 12226 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:30.146302 12557 consensus_queue.cc:237] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:30.342206 12407 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:30.347507  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:30.347823  8205 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:30.285025 12428 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:30.344303  8727 rpcz_store.cc:269] 0709 13:14:28.229430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.230787 (+  1357us) service_pool.cc:224] Handling call
0709 13:14:29.559131 (+1328344us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b5800011798. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:14:30.281744 (+722613us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54007611d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:30.281937 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581251200}
I20260709 13:14:30.302187 12301 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:30.388398  8210 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:14:30.364872  8060 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:30.299008 12548 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:30.368067  8134 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:30.372902  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:30.373682  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:30.392433  8729 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
W20260709 13:14:30.401266  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.376361  8102 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:30.358673 12504 raft_consensus.cc:2764] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:14:30.390841  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:30.442986  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 52232.6us
I20260709 13:14:30.443735  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:30.445470  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:30.363032 12229 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:30.446354 12229 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.446718 12229 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:30.446952 12567 raft_consensus.cc:2764] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 13:14:30.447678 12567 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.448107 12567 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:30.451046 12468 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:30.451562 12468 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:30.451731  8130 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:14:30.441903  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:30.461946  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 105392us
I20260709 13:14:30.321079 12506 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.377591 12522 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.462860  8696 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:30.383241 12470 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.489836  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:30.493767  8054 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:30.359988 12446 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.496323  8101 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:30.497036 12509 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:30.440951 12166 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:30.374475 12136 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:30.505553  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 142462us
I20260709 13:14:30.508502  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:30.511739  8063 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:30.509114  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 46325.5us
I20260709 13:14:30.511876  8102 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:30.443470 12328 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:30.485631 12437 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.487133 12554 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:30.515508  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 70103.9us
I20260709 13:14:30.491094 12191 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:30.526582  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:30.529263  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 469709us
I20260709 13:14:30.441371 12548 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:30.486395 12534 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:14:30.534741  8063 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:30.452391 12468 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:30.541472  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.486310 12551 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:14:30.515969 12495 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:30.548637  8027 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:30.551911  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:30.552549  8730 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:14:30.554201 12554 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:30.554590 12554 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:30.554949 12515 consensus_meta.cc:287] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.000s
I20260709 13:14:30.555531 12355 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:30.555378 12515 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.552697  8054 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:30.501300 12522 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:30.531379 12369 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.504518 12470 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:30.563874  8104 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:30.492259 12354 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.392846 12568 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.540243 12546 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:30.529740 12437 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:30.535930 12533 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:31.243724  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 428218us
I20260709 13:14:31.280831 12509 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:30.565603 12307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:38710 (request call id 3) took 7521 ms. Trace:
I20260709 13:14:30.575757 12191 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.708s
W20260709 13:14:31.089187  8210 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.008s	sys 0.011s
I20260709 13:14:30.556756 12567 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:30.713088 12328 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.554952 12554 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:30.994141  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:30.635130  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 60663.3us
I20260709 13:14:31.332443  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1792) took 1159 ms. Trace:
I20260709 13:14:31.402493  8207 rpcz_store.cc:276] 0709 13:14:30.173483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.173588 (+   105us) service_pool.cc:224] Handling call
0709 13:14:31.332191 (+1158603us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b540010fed4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:31.332351 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15556352}
W20260709 13:14:31.242985 12558 consensus_meta.cc:287] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.999s	user 0.009s	sys 0.002s
W20260709 13:14:30.602334  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 104514us
W20260709 13:14:30.697710  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.712483 12166 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:30.635291  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.638303  8054 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
W20260709 13:14:30.578706  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 54903.6us
W20260709 13:14:31.321022 12538 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.006s	sys 0.006s
W20260709 13:14:31.419601  8210 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.885s	user 0.008s	sys 0.017s
I20260709 13:14:30.816169 12298 raft_consensus.cc:3055] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:30.815654 12423 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:30.618957 12430 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:30.817502 12253 consensus_meta.cc:287] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.515s	user 0.008s	sys 0.000s
W20260709 13:14:31.425168  8029 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.67:35745 timed out after 1.515s (SENT) [suppressed 13 similar messages]
W20260709 13:14:31.229741 12581 log.cc:927] Time spent T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:14:30.583163 12383 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:30.584473  8060 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:14:30.629976 12579 log.cc:927] Time spent T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:14:31.418741 12470 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:31.421689 12437 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:31.445039 12603 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.420953 12604 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.563592 12495 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:31.483212 12603 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.485117 12495 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:31.486851 12562 log.cc:927] Time spent T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 1.364s	user 0.013s	sys 0.003s
W20260709 13:14:31.328114 12588 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.584s	user 0.000s	sys 0.000s
I20260709 13:14:31.290684 12438 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.471218 12582 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.452113  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:30.582367 12522 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:31.327292 12589 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.454015  8204 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:31.495064  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:31.495558  8204 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:31.340376 12546 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:30.811913 12577 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:14:31.367388 12301 consensus_meta.cc:287] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.973s	user 0.010s	sys 0.000s
I20260709 13:14:30.809463 12585 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:14:30.810146 12571 log.cc:927] Time spent T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 13:14:31.228993 12580 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:31.510298 12568 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:31.494097 12582 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.323026 12354 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:31.343647 12533 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:31.312019 12432 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.458446  8212 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:14:31.519411 12432 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.521404 12432 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.313742 12226 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.618841 12434 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:30.697376 12136 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.399559  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1796) took 1224 ms. Trace:
I20260709 13:14:31.524483  8203 rpcz_store.cc:276] 0709 13:14:30.175206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.302868 (+127662us) service_pool.cc:224] Handling call
0709 13:14:30.303167 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:30.556790 (+253623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:30.556823 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:30.584387 (+ 27564us) log.cc:844] Serialized 26 byte log entry
0709 13:14:31.247898 (+663511us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:31.247953 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.248046 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:31.285408 (+ 37362us) raft_consensus.cc:1685] finished
0709 13:14:31.285436 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.399478 (+114042us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18480,"raft.queue_time_us":41508,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":18140,"thread_start_us":416185,"threads_started":2,"wal-append.queue_time_us":697493}
I20260709 13:14:30.809239 12587 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:31.460438 12423 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:31.463071  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:31.463177 12597 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.505267 12589 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.531291  8208 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:14:31.531507  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 733) took 1260 ms. Trace:
I20260709 13:14:31.531903  8724 rpcz_store.cc:276] 0709 13:14:30.271376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.271494 (+   118us) service_pool.cc:224] Handling call
0709 13:14:30.286921 (+ 15427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.287148 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:30.287184 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:30.287250 (+    66us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:30.287302 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.287371 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.531352 (+1243981us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b54007611d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:31.531452 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28207360}
I20260709 13:14:31.532642 12434 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.532701 12589 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:31.533386 12589 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:31.533262 12434 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:31.533910  8057 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2812) took 2535 ms (client timeout 2248 ms). Trace:
W20260709 13:14:31.534294  8057 rpcz_store.cc:269] 0709 13:14:28.998828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.998934 (+   106us) service_pool.cc:224] Handling call
0709 13:14:31.533717 (+2534783us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540043ce54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:31.533822 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2638962432}
I20260709 13:14:30.763629 12355 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:31.327109 12551 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.124s
I20260709 13:14:31.463596 12191 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:31.306322 12578 consensus_queue.cc:1059] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.161s
I20260709 13:14:30.698180 12229 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:31.456353 12538 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.006s	sys 0.006s
I20260709 13:14:31.458827 12298 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:31.464700 12378 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:31.464886  8101 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:31.465498 12426 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.466300 12598 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:31.470026  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:30.576102 12369 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.472035  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:31.554453  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 82461.3us
I20260709 13:14:31.556090  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:30.563297 12428 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:30.723417 12572 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.943s
I20260709 13:14:31.476457  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1465) took 1928 ms. Trace:
I20260709 13:14:31.461174 12567 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.479593  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 742) took 2262 ms. Trace:
I20260709 13:14:31.481374 12523 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:31.477295 12166 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.482586 12159 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:31.483230 12604 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.486730 12606 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:31.246568  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 734708us
I20260709 13:14:31.488258  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 746) took 1988 ms. Trace:
I20260709 13:14:30.580713 12373 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:30.811077 12515 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:31.492645  8205 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:31.568372 12567 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.571632 12567 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:31.567073 12159 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:31.573101 12159 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.573613 12369 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.490486 12438 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.574548 12159 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:31.575903 12438 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:31.502044  8204 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:31.503729 12607 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:31.471693 12430 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.541260 12603 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.534025 12589 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.523293 12226 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.567111 12572 consensus_queue.cc:1243] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:31.529361 12522 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:14:31.283871 12307 rpcz_store.cc:276] 0709 13:14:23.029938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.030055 (+   117us) service_pool.cc:224] Handling call
0709 13:14:23.680801 (+650746us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b5800700dd0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf720aa 00007f6dbaff4447 00007f6dbb01170a 00007f6dbb01276a 00007f6dbb01191b 00007f6dbb01d03c 00007f6dbb01c6b3 00007f6dbb0149fa 00007f6dbffdc4c9 00007f6db9558c54
0709 13:14:24.528113 (+847312us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b4c00235780. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db92e2a20 00007f6db92f3a29 00007f6db92b9644 00007f6db92e0f2d 00007f6db92e0b93 00007f6dbaf6f1af 00007f6dbaf6ef3e 00007f6dbaff4888 00007f6dbb01175c 00007f6dbb01276a 00007f6dbb01191b
0709 13:14:24.568309 (+ 40196us) log.cc:844] Serialized 494 byte log entry
0709 13:14:25.514290 (+945981us) spinlock_profiling.cc:243] Waited 6.78 ms on lock 0x7b18004dcd40. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ef72a6 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf96b26 00007f6dbaf9049a 00007f6dbaf8f714
0709 13:14:25.553168 (+ 38878us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b5400731020. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf8f142 00007f6dbaf6feaa 00007f6dbaf76d53 00007f6dbaf76cd9 00007f6dbaf76c69 00007f6dbaf76c31 00007f6dbaf75dfd 000000000038f354 000000000038f199 00007f6db7f29475
0709 13:14:25.608163 (+ 54995us) spinlock_profiling.cc:243] Waited 8.4 ms on lock 0x7b18004dcd40. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:14:25.730923 (+122760us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b5400731020. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf96b26 00007f6dbaf9049a 00007f6dbaf8f714 00007f6dbaf6feaa 00007f6dbaf76d53 00007f6dbaf76cd9 00007f6dbaf76c69 00007f6dbaf76c31 00007f6dbaf75dfd 000000000038f354
0709 13:14:30.537320 (+4806397us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33273,"raft.queue_time_us":12954,"raft.run_cpu_time_us":8134,"raft.run_wall_time_us":1970401,"spinlock_wait_cycles":465917312,"thread_start_us":96626,"threads_started":1,"wal-append.queue_time_us":105597,"wal-append.run_cpu_time_us":1687,"wal-append.run_wall_time_us":1001789}
I20260709 13:14:31.529930 12423 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.531008 12597 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:30.785753 12576 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.922s
W20260709 13:14:31.541069  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 115943us
W20260709 13:14:31.639405  8029 leader_election.cc:336] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Timed out: RequestConsensusVote RPC to 127.6.198.67:35745 timed out after 1.616s (SENT)
I20260709 13:14:31.511660 12546 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.527042 12587 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:31.553545  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 88760.9us
I20260709 13:14:31.643427  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:31.650732  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:31.652166 12603 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:31.652733 12603 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.653811 12226 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.654701 12603 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:31.654701 12604 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.655761 12434 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:31.558302 12598 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.658644  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:31.659443  8730 raft_consensus.cc:2368] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13.
I20260709 13:14:31.557478  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:31.536710 12432 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:31.523980 12582 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:31.559568  8206 rpcz_store.cc:276] 0709 13:14:29.216778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.216880 (+   102us) service_pool.cc:224] Handling call
0709 13:14:29.217231 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:29.233967 (+ 16736us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:29.233998 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:30.169083 (+935085us) spinlock_profiling.cc:243] Waited 5.64 ms on lock 0x7b2c00116ca8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db990fa11 00007f6db991f372 00007f6dbb051a01 00007f6dbb01643a 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2
0709 13:14:30.181875 (+ 12792us) log.cc:844] Serialized 26 byte log entry
0709 13:14:30.494012 (+312137us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:30.494045 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.494103 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:31.479242 (+985139us) raft_consensus.cc:1685] finished
0709 13:14:31.479264 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.479496 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":485,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"spinlock_wait_cycles":12962688,"thread_start_us":245534,"threads_started":1,"wal-append.queue_time_us":449376}
I20260709 13:14:31.559401  8132 rpcz_store.cc:276] 0709 13:14:29.548168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.548305 (+   137us) service_pool.cc:224] Handling call
0709 13:14:31.476290 (+1927985us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b540040a1d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:31.476410 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":596230528}
I20260709 13:14:31.564905  8208 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:14:31.584637  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.541834 12506 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.583635  8104 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:31.488910 12470 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.567492  8209 rpcz_store.cc:276] 0709 13:14:29.499973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.502458 (+  2485us) service_pool.cc:224] Handling call
0709 13:14:29.502723 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:29.504198 (+  1475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:29.504242 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:29.504755 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 13:14:29.566405 (+ 61650us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:29.566437 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.566498 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:31.478064 (+1911566us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:31.487966 (+  9902us) raft_consensus.cc:1685] finished
0709 13:14:31.487983 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.488210 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48862464,"thread_start_us":61185,"threads_started":1,"wal-append.queue_time_us":587567}
I20260709 13:14:31.563397 12301 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:31.685837 12383 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:14:31.686455 12301 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:31.683976 12191 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.686879 12301 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:31.687021 12191 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.687670 12589 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:31.689023 12191 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:31.690025  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 46701.6us
I20260709 13:14:31.690497 12520 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:14:31.691782 12522 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:31.692245 12522 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:31.693041  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:31.693853 12546 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.694557  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:31.528662  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:31.578022 12568 raft_consensus.cc:2764] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:14:31.696398 12598 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.553124 12426 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.697144  8130 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:31.698328 12426 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.698824  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:31.693133 12522 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.699339  7946 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b186be823643475982e269b73d886a42 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:14:31.700572  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:31.701408  8131 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:31.637976  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:31.594980 12606 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.651075 12587 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:31.655758 12430 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.678682 12572 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.641448 12355 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.685884 12307 raft_consensus.cc:3055] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:31.507172 12591 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.694433 12470 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.695401  8729 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:31.762126  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:31.696352  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:31.693125  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 109603us
I20260709 13:14:31.763198  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:31.763805 12470 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:31.764621  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:31.765213  8132 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:31.639941  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:31.769626 12470 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:14:31.769783  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 130404us
W20260709 13:14:31.770614  8029 leader_election.cc:336] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Timed out: RequestConsensusVote RPC to 127.6.198.67:35745 timed out after 1.657s (SENT)
I20260709 13:14:31.771054  8029 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:31.771910 12470 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:14:31.772356  8029 leader_election.cc:336] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Timed out: RequestConsensusVote RPC to 127.6.198.67:35745 timed out after 1.664s (SENT)
I20260709 13:14:31.773018  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:31.579249 12136 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:31.699584  8134 raft_consensus.cc:2393] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:31.600242 12229 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.679248 12328 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:31.703850 12432 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.652570 12593 consensus_queue.cc:1059] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.149s
I20260709 13:14:31.715047  8138 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:31.601107 12607 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.758291  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:31.759770  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:31.785502 12136 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:31.786275 12546 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:31.787447 12136 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.786653 12328 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.790303  8724 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 13: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:31.790495 12328 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:31.759459 12572 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:31.761744 12555 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 1.639s	user 0.006s	sys 0.010s
I20260709 13:14:31.762692  8131 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:31.762332 12591 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.786058  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:31.795861  8727 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:31.765228 12597 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.797400  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:31.722074 12554 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.800005 12594 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:31.800333  8207 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:14:31.800534 12594 raft_consensus.cc:697] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:31.801404 12594 consensus_queue.cc:237] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.760550 12307 raft_consensus.cc:740] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:31.803028 12307 consensus_queue.cc:260] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.804241 12307 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:31.804595 12591 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:31.804723  8211 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:31.779467 12191 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.846688  8138 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:14:31.850903  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:31.847041  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:31.847116  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:31.864660  8132 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:31.806383  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:31.864830  8727 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
W20260709 13:14:31.818112  8029 leader_election.cc:336] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Timed out: RequestConsensusVote RPC to 127.6.198.67:35745 timed out after 1.730s (SENT)
I20260709 13:14:31.870878  8134 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:31.871173  8137 raft_consensus.cc:2410] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:14:31.878057  8131 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:31.878756  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 749) took 1364 ms. Trace:
I20260709 13:14:31.880707  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:31.881019  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 62946.2us
I20260709 13:14:31.879153  8210 rpcz_store.cc:276] 0709 13:14:30.514734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.514915 (+   181us) service_pool.cc:224] Handling call
0709 13:14:30.533844 (+ 18929us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:31.456783 (+922939us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:31.456817 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:31.457376 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:14:31.873801 (+416425us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:31.873861 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.873968 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:31.876572 (+  2604us) raft_consensus.cc:1685] finished
0709 13:14:31.876596 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.878700 (+  2104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54439,"raft.queue_time_us":720853,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":41830,"thread_start_us":1329109,"threads_started":2,"wal-append.queue_time_us":406939}
I20260709 13:14:31.882144  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:31.882918  8130 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:31.871023 12434 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.845844 12572 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.850140 12554 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:31.889228  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:31.871937 12191 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.894678 12432 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:31.895505  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:31.896131  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:31.896862  8132 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:31.899123 12548 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:31.899267 12531 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.902872  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:31.880359 12229 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:31.845916 12587 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.905714  8134 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:31.908789  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:31.918591  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:31.927508 12301 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.928850 12531 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:31.959066 12520 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:31.959708 12520 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.959913 12301 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:31.920416  8138 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:31.930640  8131 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:31.961608  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:31.962162  8138 raft_consensus.cc:2376] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10. Re-sending same reply.
I20260709 13:14:31.889807 12495 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 13:14:31.963568 12539 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:31.963959 12539 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:31.964708 12539 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.884261 12434 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:31.935184  8130 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:31.989562  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:31.939575  8727 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 13:14:31.940301 12191 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:31.940634 12623 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:14:31.946038  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:31.952168 12423 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:31.952890 12229 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:31.933863 12464 raft_consensus.cc:3055] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:31.955116  8133 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:31.955482 12587 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:32.028120  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:32.027194  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:32.031951 12446 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:32.036059  8727 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 7.
I20260709 13:14:32.036857  8131 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:32.034301  8136 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:32.025362  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:32.014305  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.037779  8130 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 6.
I20260709 13:14:32.037819 12422 raft_consensus.cc:3055] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:32.044987 12422 raft_consensus.cc:740] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:32.045854 12422 consensus_queue.cc:260] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.048054  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:32.048770  8729 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:32.049106  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:32.049652  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:32.050213  8727 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:32.052830  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:32.053380  8057 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:32.044533 12504 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:32.058993  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 501529us
I20260709 13:14:32.062004  8175 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:32.059803 12582 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:32.063602  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:32.066620  8726 raft_consensus.cc:1813] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Handling vote request from an unknown peer d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:32.067225  8726 raft_consensus.cc:2410] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:14:32.059715 12520 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:32.059927  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:32.067580  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 120790us
I20260709 13:14:32.060624 12534 raft_consensus.cc:3055] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:32.039126  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:32.038156  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.039357  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.087258  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:32.087870  8132 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:32.087605  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:32.088652  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.087620  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:32.089352  8721 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:32.079865  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.067812  8727 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:32.091953  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.090723  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:32.094105  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:32.097622  8058 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:32.097441  8134 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:32.091766  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:32.099723  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:32.024996 12464 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:32.101755  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.102367  8129 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:32.088061 12534 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:32.104553  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:32.103857 12534 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.109092 12422 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:32.069701 12504 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.108645  8102 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:32.112629 12504 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:32.110702 12534 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:32.037842 12229 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:32.115214  8102 leader_election.cc:400] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13.
W20260709 13:14:32.122704  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.124307  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.126684  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.129762  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.130527  8205 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:32.070667 12446 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.121423  8026 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:32.133766  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.134132  8209 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:32.134693  8206 raft_consensus.cc:2410] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:14:32.134801  8204 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:14:32.070058 12539 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:32.137154  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 75226.6us
I20260709 13:14:32.144923  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.145704  8203 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:32.043196 12606 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:32.136664  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:32.144979  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:32.150790  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.136071  8204 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:32.147496  8027 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:32.151697  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.155572  8054 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:14:32.155961  8054 pending_rounds.cc:85] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 6
I20260709 13:14:32.156240  8054 pending_rounds.cc:107] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:14:32.156549  8054 raft_consensus.cc:2889] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:32.147818  8026 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:32.144042 12504 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:14:32.148751 12428 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.159583  8206 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:32.160471  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:32.160518  8027 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:32.162483  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:32.163113  8209 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:14:32.122816  8101 connection.cc:256] RPC call timeout handler was delayed by -1.0730954179998662s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22480000000000003s for a response
I20260709 13:14:32.163581  8029 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:32.162664  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.164971  8210 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:32.136114 12531 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:32.133841 12136 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.162786  8061 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:32.137142 12522 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:32.196671  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 81607.1us
I20260709 13:14:32.133929 12591 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.190151 12616 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:14:32.208137 12591 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.196962 12603 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:32.210530 12591 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:32.194851  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:32.196373  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:32.219890  8176 connection.cc:256] RPC call timeout handler was delayed by -1.2229754919999323s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
I20260709 13:14:32.150398 12568 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.167119 12506 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:32.203218  8101 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:32.204574 12533 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.196331 12226 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:32.205433 12548 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:32.205456 12582 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.150144 12301 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.206411 12522 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.158914 12354 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:32.207345  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.199797  8104 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:32.154470 12229 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:32.179960  8026 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:32.211241  8138 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
W20260709 13:14:32.212473  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.154799 12593 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:32.135996 12258 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:14:32.210038 12610 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:32.210757 12603 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.288707  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:32.197476  8102 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:32.167857  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:32.136421 12520 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:14:32.137713 12423 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
W20260709 13:14:32.224848  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 108416us
W20260709 13:14:32.242621  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 62726.3us
I20260709 13:14:32.243604  8101 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:32.167392 12407 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:14:32.166942 12307 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:14:32.155365 12446 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.226738 12591 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.189677 12428 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.229017 12506 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.227717 12226 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.286578  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:32.335201  8054 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:32.228422 12568 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.296321  8206 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 14: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:32.168537 12546 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:14:32.298988  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:32.212119 12136 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.244351 12301 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.300154  8026 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:32.301867  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 104462us
I20260709 13:14:32.243773 12582 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.305147 12597 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:32.244772 12383 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:32.285382 12554 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:32.299329 12603 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:32.231748 12533 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.287909  8721 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
W20260709 13:14:32.342507  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 43594.3us
I20260709 13:14:32.305774 12523 consensus_queue.cc:1059] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 10.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:14:32.359397  8027 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:32.297221 12593 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.244670 12531 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:32.347267  8057 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:32.343607  8026 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:32.344147  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:32.348356  8724 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:32.349047  8102 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:32.353150 12354 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:32.353335 12624 log.cc:927] Time spent T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:14:32.303182 12159 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:32.283943  8134 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:14:32.287679 12590 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:14:32.348696 12606 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:32.303967  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:32.288777 12572 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:14:32.339335 12506 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:32.300057 12355 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:14:32.387626  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:32.339108 12226 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:32.314824 12166 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.353772 12383 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.391757  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:32.360405 12422 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:32.390193  8727 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:14:32.351323 12616 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:32.383052 12629 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.411273  8176 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:32.418295 12298 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.418993 12572 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:32.419173  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:32.425609  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:32.389312 12159 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.412909 12430 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:32.420821 12515 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.383117 12554 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:32.435214  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:32.389220 12553 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:14:32.427119  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 183657us
I20260709 13:14:32.348016 12591 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:32.427649  8175 connection.cc:256] RPC call timeout handler was delayed by -1.2820116699999744s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
I20260709 13:14:32.428575  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.429282  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:32.425765  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.392280 12606 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:32.431911  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 250795us
I20260709 13:14:32.406008 12383 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:32.409581 12422 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.415091 12258 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:32.438375  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:32.458932  8175 connection.cc:256] RPC call timeout handler was delayed by -1.2010116699998434s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
W20260709 13:14:32.467715  8175 connection.cc:256] RPC call timeout handler was delayed by -1.0561116699998365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1759s for a response
I20260709 13:14:32.416105 12630 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.460177  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.462724  8204 raft_consensus.cc:2368] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14.
I20260709 13:14:32.468348  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:32.459306  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:32.433606 12159 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:32.426533 12582 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:32.462177  8209 raft_consensus.cc:2376] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10. Re-sending same reply.
I20260709 13:14:32.389385 12354 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.409703 12166 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.433836 12616 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:32.480108  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.490899  8205 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:32.481353  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:32.461655 12430 raft_consensus.cc:697] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:32.477999  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 134462us
I20260709 13:14:32.483666  8209 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:14:32.493813  8209 pending_rounds.cc:85] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 5
I20260709 13:14:32.494127  8209 pending_rounds.cc:107] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:14:32.494412  8209 raft_consensus.cc:2889] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:32.461817 12422 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:32.496595  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.489794  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:32.499423  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:32.501426  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:32.507356  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:32.508519 12637 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.509173 12637 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.512079  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.512802  8210 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:32.469309 12606 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:32.471519 12531 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:14:32.531309  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:32.539691  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 71435.8us
I20260709 13:14:32.540889 12136 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:32.540776 12422 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.544086  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:32.476408 12307 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:32.458464 12515 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.432989 12629 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.588284  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.477759 12636 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:32.511317 12522 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:32.597028  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.597097  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:32.589092  8209 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 12: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:14:32.615885  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 152117us
I20260709 13:14:32.620054  8210 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:32.570971 12438 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.514118 12582 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.098s
I20260709 13:14:32.630179  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.478576 12630 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.588959 12634 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:32.631018  8204 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:32.638813  8132 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:14:32.635210 12226 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.640293  8130 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: [7.5-11.6]
I20260709 13:14:32.641093 12226 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:32.642573 12301 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:32.645920 12430 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.649915 12616 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.651674 12629 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:32.653614 12515 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:32.589951 12307 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.654709 12354 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:32.633431  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 284451us
I20260709 13:14:32.659883  8102 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:32.661504  8102 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:32.662606  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 169598us
I20260709 13:14:32.663848  8026 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:32.624990 12533 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:32.678500  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.678870  8026 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:32.679365  8132 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:32.668399  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.678022  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:32.680945  8131 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:32.628602 12568 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:32.666079  8102 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:32.675377 12301 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:32.681399  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.684122  8212 raft_consensus.cc:2410] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:14:32.758673  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.759301  8204 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:32.657238 12634 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:32.761051  8029 leader_election.cc:336] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Timed out: RequestConsensusVote RPC to 127.6.198.67:35745 timed out after 1.857s (SENT)
W20260709 13:14:32.762305  8029 leader_election.cc:336] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Timed out: RequestConsensusVote RPC to 127.6.198.67:35745 timed out after 1.935s (SENT)
I20260709 13:14:32.637120 12422 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:32.656342 12428 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:32.636749 12572 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:32.635900 12355 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.665737 12639 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:32.679019 12522 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:32.675377 12531 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:32.632615 12438 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.678516 12629 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:32.624626 12554 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:32.774829  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 177796us
I20260709 13:14:32.770090  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.775750  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.786681  8137 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:32.775473  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:32.764845  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 98871.2us
I20260709 13:14:32.776665  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.777366  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.681121 12553 raft_consensus.cc:697] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:32.756735 12258 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:32.780912  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:32.782137  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.797128  8727 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:32.764729 12641 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:32.792022  8204 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:32.786684  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:32.789083  8102 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:32.802065  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 123788us
W20260709 13:14:32.803478  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.804397 12642 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.804864 12642 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.806632 12642 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:32.807821 12642 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.808382 12642 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.811240  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:32.813068  8723 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:32.814735  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:32.814877  8133 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:32.815315  8730 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:32.816834  8727 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:14:32.817924  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:32.819057 12422 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:32.819491 12422 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:32.819868 12422 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:32.768690 12634 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:32.779266 12446 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:32.784838 12591 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:32.831799  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:32.837845  8204 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:32.831799 12383 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:32.832571  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45998.4us
I20260709 13:14:32.827313 12652 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.834842 12166 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:32.785032 12307 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:32.814910 12633 log.cc:927] Time spent T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 13:14:32.848161 12616 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:14:32.848347 12638 log.cc:927] Time spent T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:14:32.828110 12629 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:32.850287 12438 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:32.851069 12637 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:32.851423  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:32.852017 12229 consensus_meta.cc:287] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.001s
W20260709 13:14:32.855751 12534 consensus_meta.cc:287] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.693s	user 0.005s	sys 0.003s
I20260709 13:14:32.855902 12553 consensus_queue.cc:237] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.857043 12630 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:32.801192 12641 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:32.861310 12642 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:32.864230  8130 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:32.777563 12572 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.847126  8102 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:32.871747  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.866611  8205 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:32.775420 12355 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.876408  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:32.863924 12383 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:32.875607 12652 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.887470  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.888407  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.841814 12649 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:32.874583 12616 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:32.892624  8133 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:32.859179 12422 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:32.875238 12632 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.402s	user 0.001s	sys 0.000s
I20260709 13:14:32.902637  8207 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:32.901243  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.887966 12594 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.904312 12636 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:32.915477  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 68468.3us
I20260709 13:14:32.953610  8102 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:32.951869 12616 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:32.951524 12642 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:32.898298 12229 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:32.955860  8137 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
I20260709 13:14:32.957345  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.950904 12647 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.902117 12258 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.970898  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.901500 12634 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:32.999497 12652 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:32.994545  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:33.001395  8133 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:33.001772 12647 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:33.002117 12634 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:33.002326  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:33.003229  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:33.003870 12422 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:32.973579  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:32.973802 12635 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:14:32.970706 12649 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.004882  8205 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:32.973065 12642 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.005878 12642 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:33.005155 12649 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:32.974404 12594 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:32.973256 12383 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:32.912662 12641 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:33.008292 12383 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.008708 12383 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:32.977911  8132 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:33.003919  8211 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:33.010648  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:33.011173  8211 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:33.011904  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:33.004508 12647 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:33.012669  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 136314us
I20260709 13:14:33.013403  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:33.013069  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:33.014686  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:33.016319  8211 raft_consensus.cc:2410] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:14:33.018555  8205 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:33.019590  8211 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:14:33.018116 12594 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:33.022053  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:33.027375  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:32.997103 12636 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.028017  8137 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 7.
I20260709 13:14:33.028788 12307 consensus_queue.cc:1059] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:33.029588 12307 raft_consensus.cc:3055] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:33.029891 12307 raft_consensus.cc:740] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:33.030679 12307 consensus_queue.cc:260] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.034282  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:33.040134  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:33.035609  8211 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:14:33.041993  8696 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:33.042644  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:33.044732  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:33.045861  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:33.046430 12656 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:33.046818  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:33.046923 12656 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.047865  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:33.048388  8137 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:33.048940 12656 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:33.049127  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:33.051847  8133 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:14:33.050136  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:33.056643  8723 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 7.
I20260709 13:14:33.037053 12307 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:33.042260  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:33.058621  8211 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:33.059806  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.030690 12634 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:33.060403  8063 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:33.061316  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:33.061779  8063 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:33.063072  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:33.063716  8133 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:33.053356 12534 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:33.087764  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:33.088373  8133 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:33.095269  8063 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:33.034691 12545 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 13:14:33.110124  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 73874.5us
I20260709 13:14:33.110176  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.074941 12520 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.132905  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.019551 12642 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:33.119494  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 77563.3us
I20260709 13:14:33.155849  8063 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:33.161275  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.162441  8063 raft_consensus.cc:2393] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:33.126844  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:33.164220  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:33.111061 12603 consensus_meta.cc:287] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.724s	user 0.010s	sys 0.000s
W20260709 13:14:33.156589  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.075627 12657 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:33.227366  8131 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 12: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:33.155983 12422 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.256404 12434 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:33.172828  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:33.205381 12603 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.255687  8063 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:33.166069  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:33.228309  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:33.228677  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 63963.5us
W20260709 13:14:33.258355  8060 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 663) took 2871 ms (client timeout 1644 ms). Trace:
W20260709 13:14:33.250566 12660 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:14:33.237188  8211 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
W20260709 13:14:33.262586  8060 rpcz_store.cc:269] 0709 13:14:30.386791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.441651 (+ 54860us) service_pool.cc:224] Handling call
0709 13:14:33.258146 (+2816495us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400219b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:33.258265 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":297999616}
I20260709 13:14:33.267984  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.268595  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:33.263729  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:33.259610  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:33.079833 12634 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:33.269214  8211 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:33.228530  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 58834.8us
I20260709 13:14:33.207060 12534 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:33.256919 12422 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.273231  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:33.252218 12642 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:33.274716  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1929) took 1374 ms. Trace:
I20260709 13:14:33.279538  8136 rpcz_store.cc:276] 0709 13:14:31.896068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.897338 (+  1270us) service_pool.cc:224] Handling call
0709 13:14:31.897690 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.037343 (+139653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.263350 (+1226007us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54001ebc54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:33.270407 (+  7057us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313318656}
I20260709 13:14:33.166314 12666 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:33.281491  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:33.155032 12591 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.140s
I20260709 13:14:33.277791  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:33.278585  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.232568 12636 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:14:33.165961 12649 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:33.154878 12567 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.118s
W20260709 13:14:33.270569 12539 consensus_meta.cc:287] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 1.125s	user 0.011s	sys 0.001s
I20260709 13:14:33.289575  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:33.282459 12534 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.291505 12534 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:33.293569  8063 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:33.295503  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:33.296670  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:33.297686  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:33.297948 12662 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:14:33.298489  8205 raft_consensus.cc:2393] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:33.299302  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:33.300040  8205 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:33.300768  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:33.284330 12665 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:33.302874  8136 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:33.303962 12607 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:33.305864 12506 consensus_meta.cc:287] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.888s	user 0.007s	sys 0.000s
W20260709 13:14:33.290712 12606 consensus_meta.cc:287] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.713s	user 0.003s	sys 0.008s
I20260709 13:14:33.307112 12606 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.309019 12606 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:33.309165 12641 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.311019  8721 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:14:33.311096  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:33.311705  8203 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:33.312264  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55942 (request call id 756) took 1144 ms. Trace:
I20260709 13:14:33.312636  8206 rpcz_store.cc:276] 0709 13:14:32.168256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.168402 (+   146us) service_pool.cc:224] Handling call
0709 13:14:33.312086 (+1143684us) spinlock_profiling.cc:243] Waited 49.2 ms on lock 0x7b54001531d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:33.312210 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113174784}
I20260709 13:14:33.313712  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:33.314765  8209 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:33.297775 12572 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:33.301321  8205 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:33.317369  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.317927  8060 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:33.314209  8206 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:14:33.318881  8211 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:33.296726  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.320437  8063 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:33.320621  8203 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:33.321496  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:33.322197  8063 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:33.165581 12520 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.326740  8055 raft_consensus.cc:2368] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
I20260709 13:14:33.286688 12666 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:33.276885 12634 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:33.314832 12523 raft_consensus.cc:3055] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:33.345649  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.341146 12671 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:33.338831 12666 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:33.326977 12665 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:33.343529  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 675) took 1293 ms. Trace:
I20260709 13:14:33.343691 12383 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:33.344857  8057 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 670) took 1236 ms (client timeout 1639 ms). Trace:
W20260709 13:14:33.382962  8057 rpcz_store.cc:269] 0709 13:14:32.108732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.137257 (+ 28525us) service_pool.cc:224] Handling call
0709 13:14:33.344699 (+1207442us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b540043e254. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:33.344799 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423561088}
I20260709 13:14:33.384109  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:33.384856 12672 log.cc:927] Time spent T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:14:33.385648  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:33.347824 12159 consensus_meta.cc:287] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.856s	user 0.005s	sys 0.004s
I20260709 13:14:33.347951  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:33.349751 12641 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:33.262125 12657 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:33.403429 12593 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:33.403573 12616 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.405304 12657 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:33.345460 12523 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:33.406286 12523 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:33.407390 12649 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:33.407862 12634 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:33.408687  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:33.360684  8724 rpcz_store.cc:276] 0709 13:14:32.050217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.050382 (+   165us) service_pool.cc:224] Handling call
0709 13:14:32.050614 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.381992 (+331378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.343279 (+961287us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b540060ead4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:33.343473 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":684238208}
I20260709 13:14:33.387187  8137 raft_consensus.cc:2393] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:33.350905 12671 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.409770 12671 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:33.358330 12520 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:33.399950  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 52078.3us
I20260709 13:14:33.408824  8203 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:14:33.402482 12666 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.420846  8102 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:33.415891  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:33.409288  8208 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:33.418284  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:33.421880  8724 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:33.419055  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:33.423456  8203 raft_consensus.cc:2410] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:14:33.415469  8063 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:33.428511  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:33.429108  8723 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:33.429910  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:33.428433  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:33.430574  8726 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:33.360608 12665 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.403518 12550 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:33.439237  8729 raft_consensus.cc:1813] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Handling vote request from an unknown peer 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:33.440156  8729 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 12: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:33.440249  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:33.424472 12422 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:33.435434  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.306280 12506 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.438012  8724 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:33.445704  8063 raft_consensus.cc:2393] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:33.433904  8029 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:33.443013  8723 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:33.434657  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:33.443472  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:33.432458 12641 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:33.444610  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:33.491482  8208 raft_consensus.cc:2376] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10. Re-sending same reply.
W20260709 13:14:33.490922  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 47500.4us
I20260709 13:14:33.429065 12603 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:33.493532  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:33.493468  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:33.494614  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 60785.7us
I20260709 13:14:33.492983 12572 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:33.498159  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:33.498741  8208 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:33.499660  8060 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:33.438643 12550 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.414814 12159 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:33.505360  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:33.505589  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:33.549561  8063 raft_consensus.cc:2393] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:33.500664  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.439594 12665 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:33.498498 12641 raft_consensus.cc:697] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:33.551144  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:33.557392  8729 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:33.557612  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 52348.1us
I20260709 13:14:33.553192  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:33.432153 12534 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:33.554750  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:33.555999  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.556272  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.571623  8063 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:33.572439  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.572940  8063 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:33.556708  8058 raft_consensus.cc:2368] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8.
I20260709 13:14:33.561981 12603 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:33.561421 12464 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.555125 12253 raft_consensus.cc:3055] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:33.562573  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:33.563334  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.559069  8029 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:33.582078  8029 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:33.583838  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:33.585012  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:33.586035  8029 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:33.565202  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:33.507313 12616 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:33.577236  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.577940  8134 raft_consensus.cc:606] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:33.596050  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:33.596922  8722 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:33.596992  8060 raft_consensus.cc:1813] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Handling vote request from an unknown peer d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:33.597637  8060 raft_consensus.cc:2410] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:14:33.505246 12572 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.581089  8055 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:33.598703  8134 raft_consensus.cc:614] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:33.578919  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.548611 12550 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:33.565924 12671 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:33.591257  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 327636us
I20260709 13:14:33.591321 12434 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:33.592029 12591 consensus_queue.cc:1243] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:33.621304  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:33.592911 12567 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.593600 12630 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:33.549960 12383 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:33.599442  8063 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:33.629619  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:33.589875  8208 raft_consensus.cc:2410] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:14:33.630203  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.595623 12553 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:33.619136  8176 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:33.622538  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.636773  8060 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:33.627393  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:33.637844  8055 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:14:33.624470  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:33.628281  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 155536us
I20260709 13:14:33.630741  8063 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:33.600320  8029 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:33.504640 12258 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:33.639725 12258 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.639935 12616 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:33.640129 12258 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:33.640667  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:33.642779  8029 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:33.601178 12649 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:33.656328  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1809) took 1117 ms. Trace:
I20260709 13:14:33.660715  8210 rpcz_store.cc:276] 0709 13:14:32.538661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.579951 (+ 41290us) service_pool.cc:224] Handling call
0709 13:14:32.618297 (+ 38346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.620579 (+  2282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.656039 (+1035460us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b5400110654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:33.656243 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3087744}
I20260709 13:14:33.624122 12434 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.595552 12470 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:33.555648 12646 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 13:14:33.620260 12589 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:33.621359 12464 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:33.602149 12504 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:33.601361 12572 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:33.580185 12253 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:33.629427 12591 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:33.675809  8728 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:14:33.667729  8726 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:33.632969 12567 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:33.660686 12649 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.655731 12657 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:33.660874 12656 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:33.691048 12641 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.662411 12434 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:33.684617  8055 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:35.258996  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2833) took 3152 ms. Trace:
W20260709 13:14:33.987084  8058 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2844) took 1791 ms (client timeout 2156 ms). Trace:
W20260709 13:14:35.263055  8058 rpcz_store.cc:269] 0709 13:14:32.195722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.425313 (+229591us) service_pool.cc:224] Handling call
0709 13:14:33.986858 (+1561545us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400334354. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:33.987012 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2581981440}
I20260709 13:14:33.868068  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2835) took 1761 ms. Trace:
I20260709 13:14:35.265223  8061 rpcz_store.cc:276] 0709 13:14:32.106885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.762740 (+1655855us) service_pool.cc:224] Handling call
0709 13:14:33.763054 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:33.763383 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:33.763413 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:33.763898 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:14:33.865946 (+102048us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:33.865984 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.866055 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:33.867856 (+  1801us) raft_consensus.cc:1685] finished
0709 13:14:33.867876 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:33.868002 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":676,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":28544,"thread_start_us":101654,"threads_started":1,"wal-append.queue_time_us":101965,"wal-append.run_cpu_time_us":1773,"wal-append.run_wall_time_us":1001918}
I20260709 13:14:35.267102  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:34.177289  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2834) took 2070 ms. Trace:
I20260709 13:14:34.173048  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2843) took 1964 ms. Trace:
I20260709 13:14:35.268782  8063 rpcz_store.cc:276] 0709 13:14:32.106637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.659832 (+1553195us) service_pool.cc:224] Handling call
0709 13:14:33.660120 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:34.028555 (+368435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:34.028585 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:34.028625 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:34.028659 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:34.177045 (+148386us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b540027b854. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:34.177102 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:34.177233 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":846823680}
I20260709 13:14:35.269380  8056 rpcz_store.cc:276] 0709 13:14:32.195435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.507088 (+311653us) service_pool.cc:224] Handling call
0709 13:14:34.160003 (+1652915us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b54001e82d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:34.160192 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2135155072}
I20260709 13:14:35.270298  8063 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:14:35.270789  8056 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:35.270900  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 672) took 3079 ms. Trace:
I20260709 13:14:35.271363  8063 rpcz_store.cc:276] 0709 13:14:32.191455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.269803 (+3078348us) service_pool.cc:224] Handling call
0709 13:14:35.270066 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:35.270667 (+   601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.270848 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:33.671519  8060 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:35.273353  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2832) took 3167 ms. Trace:
I20260709 13:14:35.273658  8060 rpcz_store.cc:276] 0709 13:14:32.106076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.660151 (+1554075us) service_pool.cc:224] Handling call
0709 13:14:33.660410 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.273126 (+1612716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.273298 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:34.213140  8054 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2842) took 2018 ms (client timeout 1743 ms). Trace:
I20260709 13:14:34.743069 12690 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.274770 12690 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.276889 12690 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:34.763557 12691 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.277561 12691 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.280298 12691 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:34.810702  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:34.830390 12693 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.832252  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:35.281242 12573 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:35.274501  8054 rpcz_store.cc:269] 0709 13:14:32.195127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.335938 (+140811us) service_pool.cc:224] Handling call
0709 13:14:34.212953 (+1877015us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b540021a054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:34.213085 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1314139008}
I20260709 13:14:35.281114 12693 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:34.172272  8132 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1477) took 1437 ms (client timeout 1783 ms). Trace:
W20260709 13:14:34.762583  8101 connection.cc:256] RPC call timeout handler was delayed by -1.0689738569999463s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
I20260709 13:14:35.293584  8101 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:34.193277  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:34.146417  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:33.860872  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:34.259207  8131 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 683) took 1364 ms (client timeout 1705 ms). Trace:
W20260709 13:14:35.302771  8131 rpcz_store.cc:269] 0709 13:14:32.894857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.957037 (+ 62180us) service_pool.cc:224] Handling call
0709 13:14:34.258974 (+1301937us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54004097d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:34.259139 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":504043392}
I20260709 13:14:33.940021  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:33946 (request call id 23) took 1776 ms. Trace:
I20260709 13:14:35.304864  8134 rpcz_store.cc:276] 0709 13:14:32.202217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.220554 (+ 18337us) service_pool.cc:224] Handling call
0709 13:14:33.939793 (+1719239us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54001bb054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb010853 00007f6dbffde60a 00007f6db95647d4 00007f6db9564762 00007f6db9564691 00007f6db956460c 00007f6db95638c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:33.939930 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2733053312}
W20260709 13:14:34.193650  8138 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 1936) took 2295 ms (client timeout 2106 ms). Trace:
W20260709 13:14:35.306368  8138 rpcz_store.cc:269] 0709 13:14:31.898110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.962779 (+ 64669us) service_pool.cc:224] Handling call
0709 13:14:34.193470 (+2230691us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54001aa254. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:34.193573 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262819712}
W20260709 13:14:34.023593  8207 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2968) took 1897 ms (client timeout 1826 ms). Trace:
W20260709 13:14:35.307600  8207 rpcz_store.cc:269] 0709 13:14:32.126146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.126390 (+   244us) service_pool.cc:224] Handling call
0709 13:14:34.023427 (+1897037us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b540010ca54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:34.023526 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1782764288}
I20260709 13:14:35.308740  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:35.309250  8207 raft_consensus.cc:2376] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12. Re-sending same reply.
W20260709 13:14:35.309698  8207 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1840) took 1869 ms (client timeout 1871 ms). Trace:
W20260709 13:14:35.309985  8207 rpcz_store.cc:269] 0709 13:14:33.441368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.308489 (+1867121us) service_pool.cc:224] Handling call
0709 13:14:35.309638 (+  1149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:33.947803  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:35.312829  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 1.36504e+06us
I20260709 13:14:35.313508  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:34.084336  8176 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:34.780225  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:35.315780  8203 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
W20260709 13:14:35.316224  8203 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1841) took 1874 ms (client timeout 1852 ms). Trace:
W20260709 13:14:35.316516  8203 rpcz_store.cc:269] 0709 13:14:33.441681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.780008 (+1338327us) service_pool.cc:224] Handling call
0709 13:14:35.316164 (+536156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.317304  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:35.317836  8203 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:34.159395  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:34.189764  8205 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.005s	sys 0.001s
W20260709 13:14:35.321385  8205 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.726s	user 0.006s	sys 0.002s
I20260709 13:14:35.322063  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2978) took 2457 ms. Trace:
W20260709 13:14:34.169566  8206 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.012s	sys 0.033s
W20260709 13:14:35.323252  8206 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.729s	user 0.012s	sys 0.033s
I20260709 13:14:35.323915  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2979) took 2459 ms. Trace:
I20260709 13:14:35.324235  8206 rpcz_store.cc:276] 0709 13:14:32.864574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.319343 (+454769us) service_pool.cc:224] Handling call
0709 13:14:33.345503 (+ 26160us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:33.593033 (+247530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:33.593066 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:33.593659 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 13:14:35.323599 (+1729940us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:35.323655 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.323723 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:35.323761 (+    38us) raft_consensus.cc:1685] finished
0709 13:14:35.323770 (+     9us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:35.323869 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":247083,"raft.queue_time_us":247982,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":64,"spinlock_wait_cycles":59264,"thread_start_us":1729541,"threads_started":1,"wal-append.queue_time_us":156838,"wal-append.run_cpu_time_us":1456,"wal-append.run_wall_time_us":1001476}
I20260709 13:14:33.666527  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:34.181586  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1811) took 1561 ms. Trace:
I20260709 13:14:35.328359 12690 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:35.328485  8209 rpcz_store.cc:276] 0709 13:14:32.620094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.620410 (+   316us) service_pool.cc:224] Handling call
0709 13:14:33.314600 (+694190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:33.315220 (+   620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:34.181385 (+866165us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b54001531d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:34.181526 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1576557440}
I20260709 13:14:35.329604  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:35.329808  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:35.330696  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2999) took 1244 ms. Trace:
I20260709 13:14:35.331053  8209 rpcz_store.cc:276] 0709 13:14:34.086593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.329557 (+1242964us) service_pool.cc:224] Handling call
0709 13:14:35.330637 (+  1080us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:34.206049  8211 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.016s	sys 0.024s
W20260709 13:14:35.332880  8211 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.740s	user 0.016s	sys 0.024s
W20260709 13:14:34.122313  8693 leader_election.cc:336] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.644s (SENT)
W20260709 13:14:33.775098  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.742771  8729 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
W20260709 13:14:35.338284  8729 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 846) took 1827 ms (client timeout 1902 ms). Trace:
W20260709 13:14:35.338665  8729 rpcz_store.cc:269] 0709 13:14:33.510525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.510646 (+   121us) service_pool.cc:224] Handling call
0709 13:14:35.338225 (+1827579us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:33.674566 12159 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:33.657311 12166 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.349264 12166 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:33.754714 12253 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.726761 12258 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:34.165946 12298 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:35.357926  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.776677 12432 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:35.359488 12166 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:35.359299 12432 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.360112 12166 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:35.360762 12166 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:35.361879 12432 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:33.782611 12434 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.367547 12432 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.368237 12432 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.368645 12434 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:34.015218 12506 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:34.797555 12520 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.370883 12520 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.373518 12520 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:34.001410 12523 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:33.710669 12534 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:34.819178 12546 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.376636 12546 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.778888 12550 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.380378 12550 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:33.724888 12553 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:35.381891 12553 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.382272 12553 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:33.756537 12567 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:33.778137 12572 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:34.747843 12590 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.388398 12590 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.389286 12572 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:33.727082 12591 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:35.390784 12567 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.391206 12591 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:35.391460 12567 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.658847 12593 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:33.942217 12610 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.395192 12593 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.395951 12610 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:35.397153 12593 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:33.753975 12616 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:33.774453 12617 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:35.400058  7946 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b578d88c1c1f4ca58b7291fed4909f09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 965e59f272f143449b18f65db4e61d98 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:35.400388 12617 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.990998 12630 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:35.403249 12630 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:33.660277 12634 consensus_queue.cc:1059] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:14:33.664399 12636 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.157416 12639 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:35.411424 12639 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:35.412204 12639 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.413489 12639 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:33.729940 12649 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:34.037475 12654 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 1.215s	user 0.000s	sys 0.003s
I20260709 13:14:34.837061 12355 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:34.798230 12629 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.755630 12665 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:34.426568 12515 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.494508 12515 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:33.716370 12671 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:35.500461 12515 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:35.485983 12617 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:34.834060 12423 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:33.750491 12657 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:35.060626  8130 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1479) took 2300 ms (client timeout 2342 ms). Trace:
I20260709 13:14:35.502678 12423 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.503265 12657 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.271533  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 673) took 3079 ms. Trace:
I20260709 13:14:35.504206  8056 rpcz_store.cc:276] 0709 13:14:32.191703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.270282 (+3078579us) service_pool.cc:224] Handling call
0709 13:14:35.270606 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.271291 (+   685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.271480 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.504595 12423 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:33.681154  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.282128  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 676) took 2897 ms. Trace:
I20260709 13:14:35.507056  8059 rpcz_store.cc:276] 0709 13:14:32.385222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.281472 (+2896250us) service_pool.cc:224] Handling call
0709 13:14:35.281707 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.281860 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:35.281884 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:35.281921 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:35.281955 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.282003 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:35.282086 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.508483  8059 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:14:35.508971  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2848) took 2992 ms. Trace:
I20260709 13:14:35.509265  8059 rpcz_store.cc:276] 0709 13:14:32.516687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.507938 (+2991251us) service_pool.cc:224] Handling call
0709 13:14:35.508185 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.508774 (+   589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.508923 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.281879 12573 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.282507  8054 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:14:35.277585  8062 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:35.281163  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 685) took 1170 ms. Trace:
I20260709 13:14:35.283351 12693 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:35.285975 12589 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.287516  8060 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:14:35.288043  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:35.288465  8055 rpcz_store.cc:276] 0709 13:14:32.106363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.660086 (+1553723us) service_pool.cc:224] Handling call
0709 13:14:33.683909 (+ 23823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.258695 (+1574786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.258943 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":686464}
W20260709 13:14:35.290180  8132 rpcz_store.cc:269] 0709 13:14:32.745820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.746001 (+   181us) service_pool.cc:224] Handling call
0709 13:14:34.172122 (+1426121us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400342bd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:34.172220 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436144384}
I20260709 13:14:35.291330  8061 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:14:35.516711  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 674) took 3324 ms. Trace:
I20260709 13:14:35.517302  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2836) took 3410 ms. Trace:
I20260709 13:14:35.517200  8060 rpcz_store.cc:276] 0709 13:14:32.192017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.274304 (+3082287us) service_pool.cc:224] Handling call
0709 13:14:35.287253 (+ 12949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.516408 (+229155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.516646 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:35.296054  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:35.518688  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.520751  8102 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:35.298015 12691 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:35.521904 12691 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.522464 12423 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:35.522943 12423 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.524855 12423 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:35.524693 12610 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.299626  8101 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:35.525272 12649 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:35.526029  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 226510us
I20260709 13:14:35.530408 12531 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.530685  8101 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:35.526602 12589 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:35.530892 12531 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.531967  8101 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:34.787431  8212 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1808) took 2304 ms (client timeout 2685 ms). Trace:
I20260709 13:14:35.327705  8208 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:14:35.331163 12301 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:34.092154  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:35.334322  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:35.541422  8210 raft_consensus.cc:2368] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14.
I20260709 13:14:35.541536  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2997) took 1455 ms. Trace:
W20260709 13:14:35.541842  8210 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1839) took 2102 ms (client timeout 1833 ms). Trace:
I20260709 13:14:35.541936  8209 rpcz_store.cc:276] 0709 13:14:34.086016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.334083 (+1248067us) service_pool.cc:224] Handling call
0709 13:14:35.541484 (+207401us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:35.542286  8210 rpcz_store.cc:269] 0709 13:14:33.439046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.091877 (+652831us) service_pool.cc:224] Handling call
0709 13:14:35.541777 (+1449900us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.542984  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:35.543176  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:35.543512  8209 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:35.543716  8210 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:35.336982  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 1.21462e+06us
I20260709 13:14:35.545399  8060 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:35.545949  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 680) took 2748 ms. Trace:
I20260709 13:14:35.546919 12301 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.547372 12301 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:35.339092  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:35.318217  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2996) took 1232 ms. Trace:
I20260709 13:14:35.549902  8203 rpcz_store.cc:276] 0709 13:14:34.085712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.317093 (+1231381us) service_pool.cc:224] Handling call
0709 13:14:35.318169 (+  1076us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.326102  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:35.550776  8206 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:35.352672 12253 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:35.554904  8207 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 14: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:35.555804  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:35.556336  8207 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:35.342144 12159 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:35.558624  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:35.558585 12159 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.559294  8203 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:35.338460  8205 rpcz_store.cc:276] 0709 13:14:32.864344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.317382 (+453038us) service_pool.cc:224] Handling call
0709 13:14:33.338861 (+ 21479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:33.594278 (+255417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:33.594318 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:33.594791 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 13:14:35.321770 (+1726979us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:35.321806 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.321868 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:35.321901 (+    33us) raft_consensus.cc:1685] finished
0709 13:14:35.321912 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:35.322017 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":179087,"raft.queue_time_us":180070,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":32912,"thread_start_us":1726604,"threads_started":1,"wal-append.queue_time_us":184424,"wal-append.run_cpu_time_us":1175,"wal-append.run_wall_time_us":1001202}
I20260709 13:14:35.363579 12258 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:35.398269 12567 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:35.398947 12590 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:35.407891 12591 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.419451  8204 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:35.409615 12636 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.560249  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:35.435915  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 1.35157e+06us
W20260709 13:14:35.436542  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 134522us
I20260709 13:14:35.561362  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:35.370448 12432 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:35.443905  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2981) took 2578 ms. Trace:
I20260709 13:14:35.571777  8211 rpcz_store.cc:276] 0709 13:14:32.865079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.319481 (+454402us) service_pool.cc:224] Handling call
0709 13:14:33.319831 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:35.333243 (+2013412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:35.333278 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:35.333800 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 13:14:35.441385 (+107585us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:35.441441 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.441534 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:35.443421 (+  1887us) raft_consensus.cc:1685] finished
0709 13:14:35.443447 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:35.443846 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":272375,"raft.queue_time_us":273491,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"thread_start_us":1847665,"threads_started":2,"wal-append.queue_time_us":107490}
I20260709 13:14:35.452116  8135 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:35.453698  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:35.454864 12166 raft_consensus.cc:3055] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:35.456094  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:35.576529  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:35.457801  8133 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:35.458264  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:35.458866  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:35.603098  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:35.603266  8136 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:35.462443 12695 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:35.463325  8134 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: [11.11-11.11]
W20260709 13:14:35.468773  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 139250us
I20260709 13:14:35.477552  8723 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:35.482744  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:35.607360  8722 raft_consensus.cc:2393] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 13: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:35.484191  8724 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: [9.11-9.11]
I20260709 13:14:35.486331  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:35.607465  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:35.437063 12690 raft_consensus.cc:697] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:35.491715 12629 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:34.013360 12661 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.970s	user 0.001s	sys 0.002s
I20260709 13:14:35.626650 12693 consensus_queue.cc:1059] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:14:35.644253 12693 raft_consensus.cc:3055] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:35.644577 12693 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:35.645273 12693 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.495857  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:35.646512  8729 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 8: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:35.497133  8727 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:35.499552 12550 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.485085 12630 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:33.717754 12666 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:33.696652 12674 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:14:35.501609 12546 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:35.654446  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:35.654723 12546 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:35.655246  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 2998) took 1569 ms. Trace:
I20260709 13:14:35.655613  8211 rpcz_store.cc:276] 0709 13:14:34.086310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.627101 (+1540791us) service_pool.cc:224] Handling call
0709 13:14:35.655189 (+ 28088us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:35.140745  8233 debug-util.cc:398] Leaking SignalData structure 0x7b08001a6280 after lost signal to thread 6937
I20260709 13:14:35.656685  8211 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:35.656744  8233 debug-util.cc:398] Leaking SignalData structure 0x7b0800082100 after lost signal to thread 6940
W20260709 13:14:35.658159  8233 debug-util.cc:398] Leaking SignalData structure 0x7b08001b8960 after lost signal to thread 7995
I20260709 13:14:35.658596 12550 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:35.659744  8233 debug-util.cc:398] Leaking SignalData structure 0x7b08001c29c0 after lost signal to thread 8087
W20260709 13:14:35.660109  8233 debug-util.cc:398] Leaking SignalData structure 0x7b08003e1ac0 after lost signal to thread 8088
I20260709 13:14:35.660755 12550 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:35.661461  8233 debug-util.cc:398] Leaking SignalData structure 0x7b08003c6ae0 after lost signal to thread 8162
W20260709 13:14:35.503274  8130 rpcz_store.cc:269] 0709 13:14:32.761412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.766244 (+  4832us) service_pool.cc:224] Handling call
0709 13:14:35.060386 (+2294142us) spinlock_profiling.cc:243] Waited 74.6 ms on lock 0x7b5400409cd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:35.060560 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171578624}
I20260709 13:14:35.505383 12657 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:35.505563  8056 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: [12.5-12.5]
I20260709 13:14:35.663172  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2847) took 3147 ms. Trace:
I20260709 13:14:35.663074  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:35.663607  8056 rpcz_store.cc:276] 0709 13:14:32.516314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.505030 (+2988716us) service_pool.cc:224] Handling call
0709 13:14:35.505330 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:35.662941 (+157611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.663126 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.663944  8130 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:35.664832  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:34.167819  8029 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.004s	sys 0.000s
I20260709 13:14:35.512750  8057 rpcz_store.cc:276] 0709 13:14:34.111169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.810426 (+699257us) service_pool.cc:224] Handling call
0709 13:14:35.281111 (+470685us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.514153  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 677) took 3128 ms. Trace:
I20260709 13:14:35.671311  8054 rpcz_store.cc:276] 0709 13:14:32.385534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.281978 (+2896444us) service_pool.cc:224] Handling call
0709 13:14:35.282212 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.511708 (+229496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.514083 (+  2375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.512281  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2845) took 3299 ms. Trace:
I20260709 13:14:35.515452  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:35.283352  8063 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:14:35.672992  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2846) took 3194 ms. Trace:
I20260709 13:14:35.673352  8063 rpcz_store.cc:276] 0709 13:14:32.478912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.282811 (+2803899us) service_pool.cc:224] Handling call
0709 13:14:35.283063 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.672769 (+389706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.672939 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.509191 12665 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:35.526528 12520 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:35.526700  8059 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:35.674678  8138 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:35.675943  8211 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:35.675968  8207 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:35.676656  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2984) took 2810 ms. Trace:
I20260709 13:14:35.527012  8102 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:35.676921 12589 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.677035  8211 rpcz_store.cc:276] 0709 13:14:32.865920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.656331 (+2790411us) service_pool.cc:224] Handling call
0709 13:14:35.656588 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.676439 (+ 19851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.676601 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:35.677474  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 150524us
I20260709 13:14:35.678519  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2985) took 2806 ms. Trace:
I20260709 13:14:35.528088  8055 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:35.528715  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:35.529824  8058 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:14:35.533209 12523 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:35.533589 12696 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:35.535050  8101 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:35.535627 12573 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:35.522326 12691 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.517715  8061 rpcz_store.cc:276] 0709 13:14:32.107111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.267992 (+3160881us) service_pool.cc:224] Handling call
0709 13:14:35.290979 (+ 22987us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:35.517062 (+226083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.517255 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.539804  7959 catalog_manager.cc:5697] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
W20260709 13:14:35.537065  8212 rpcz_store.cc:269] 0709 13:14:32.482106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.482752 (+   646us) service_pool.cc:224] Handling call
0709 13:14:34.786103 (+2303351us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b5400189f54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:34.786865 (+   762us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348727680}
I20260709 13:14:35.553830  8210 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:35.560259  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:35.546231  8060 rpcz_store.cc:276] 0709 13:14:32.797233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.544970 (+2747737us) service_pool.cc:224] Handling call
0709 13:14:35.545238 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.545746 (+   508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.545904 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.562120  8696 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:35.575708  8725 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:14:35.572002  8208 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:35.600229  8104 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:35.616027  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:35.560181 12610 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:35.616730  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:35.534333 12531 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:35.619558  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:35.688416  8029 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.045s	user 0.007s	sys 0.000s
W20260709 13:14:35.619163  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 110354us
I20260709 13:14:35.570492 12590 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:35.601840 12166 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:35.616554 12423 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:35.627892  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:35.630162 12629 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:35.713070 12546 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.713047 12640 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.721978 12546 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.722443 12640 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.723469 12694 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:35.723881 12694 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:35.724191 12694 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:35.616977 12639 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:35.725900 12639 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.727295 12166 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.728223 12166 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:35.630872 12258 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:35.729768 12258 raft_consensus.cc:3055] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:35.730221 12258 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:35.715366 12616 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:14:35.732702  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:35.733389  8725 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:35.618697 12698 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:35.632170  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:35.632375 12591 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:35.633194 12649 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:35.635321  8204 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:35.637773  8728 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:35.621414 12690 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.638594 12159 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:35.639340 12636 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:35.640065 12553 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.604198 12567 consensus_queue.cc:1059] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:14:35.601488 12515 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:14:35.620687 12695 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.648938  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:35.650756  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:35.708233 12523 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:35.751148  7960 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34134 (request call id 265) took 27099 ms (client timeout 15000 ms). Trace:
I20260709 13:14:35.647058  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:35.684434 12520 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:14:35.720058 12696 raft_consensus.cc:697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:35.668064  8233 debug-util.cc:398] Leaking SignalData structure 0x7b0800081ae0 after lost signal to thread 8236
I20260709 13:14:35.693114 12301 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.706117  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2853) took 2921 ms. Trace:
I20260709 13:14:35.767202  8058 rpcz_store.cc:276] 0709 13:14:32.784436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.529353 (+2744917us) service_pool.cc:224] Handling call
0709 13:14:35.529611 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:35.705877 (+176266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.706063 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.768523  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:35.769321  8058 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:35.714977 12657 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:14:35.706673  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2852) took 2930 ms. Trace:
I20260709 13:14:35.756323 12567 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:14:35.707059  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 144995us
W20260709 13:14:35.772796  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.773322 12423 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:35.672111  8062 rpcz_store.cc:276] 0709 13:14:32.213073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.277070 (+3063997us) service_pool.cc:224] Handling call
0709 13:14:35.277312 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.511996 (+234684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.512217 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.773929  8696 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:35.772303  8058 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: [7.5-11.6]
I20260709 13:14:35.774763  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1678) took 2739 ms. Trace:
I20260709 13:14:35.679350  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:14:35.689705  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.709530  8203 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:35.775097  8062 rpcz_store.cc:276] 0709 13:14:33.034977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.774045 (+2739068us) service_pool.cc:224] Handling call
0709 13:14:35.774317 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.774487 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:35.774510 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:35.774548 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:35.774581 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.774626 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:35.774716 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.775924  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2988) took 2871 ms. Trace:
I20260709 13:14:35.680989  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:35.619869  8206 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:35.776309  8203 rpcz_store.cc:276] 0709 13:14:32.904329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.708987 (+2804658us) service_pool.cc:224] Handling call
0709 13:14:35.709250 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.775706 (+ 66456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.775876 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.710304  8136 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
W20260709 13:14:35.777022  8206 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 3000) took 1690 ms (client timeout 1902 ms). Trace:
I20260709 13:14:35.710577  8726 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
W20260709 13:14:35.711854  8208 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1842) took 2269 ms (client timeout 1694 ms). Trace:
W20260709 13:14:35.712414  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 177408us
I20260709 13:14:35.776310  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1669) took 2864 ms. Trace:
I20260709 13:14:35.678735  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2986) took 2791 ms. Trace:
W20260709 13:14:35.713618  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 97660.5us
I20260709 13:14:35.778203  8058 rpcz_store.cc:276] 0709 13:14:32.911864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.770104 (+2858240us) service_pool.cc:224] Handling call
0709 13:14:35.771779 (+  1675us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:35.776083 (+  4304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.776257 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.779156  8101 leader_election.cc:400] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8.
I20260709 13:14:35.761418 12589 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:35.708319  8026 leader_election.cc:336] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.563s (SENT)
I20260709 13:14:35.716015  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:35.714021  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 143981us
I20260709 13:14:35.776628  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2858) took 2729 ms. Trace:
W20260709 13:14:35.777875  8208 rpcz_store.cc:269] 0709 13:14:33.441993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.632657 (+190664us) service_pool.cc:224] Handling call
0709 13:14:35.711760 (+2079103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.780591  8101 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:35.678848  8207 rpcz_store.cc:276] 0709 13:14:32.872166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.677517 (+2805351us) service_pool.cc:224] Handling call
0709 13:14:35.677751 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.677905 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:35.678240 (+   335us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:35.678281 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:35.678317 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.678385 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:35.678469 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.717351  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:35.783767  8062 rpcz_store.cc:276] 0709 13:14:33.047450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.775985 (+2728535us) service_pool.cc:224] Handling call
0709 13:14:35.776206 (+   221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.776367 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:35.776389 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:35.776424 (+    35us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:35.776451 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.776498 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:35.776584 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.719970  8205 raft_consensus.cc:2410] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:35.784760  8061 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:35.719358 12531 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:35.713200 12506 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:35.717862  7959 catalog_manager.cc:5697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 7 to 8, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 8 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:35.712312 12693 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:35.736693  8027 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:35.778519  8211 rpcz_store.cc:276] 0709 13:14:32.887493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.678002 (+2790509us) service_pool.cc:224] Handling call
0709 13:14:35.678266 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.678455 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:35.678477 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:35.678519 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:35.678557 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.678605 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:35.678687 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:35.786887  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 51441.3us
I20260709 13:14:35.787045  8101 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:35.679467 12298 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:35.652009  8209 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:35.789894  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 3003) took 1702 ms. Trace:
I20260709 13:14:35.790235  8209 rpcz_store.cc:276] 0709 13:14:34.087543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.542744 (+1455201us) service_pool.cc:224] Handling call
0709 13:14:35.789837 (+247093us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.790004  7959 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:35.750644  8137 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:35.653909 12630 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:35.715540 12550 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:35.756901  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:35.719743 12572 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:35.759091  8176 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:35.793017  8060 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:35.759179  8204 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1838) took 2321 ms (client timeout 1754 ms). Trace:
W20260709 13:14:35.762938  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 213716us
I20260709 13:14:35.754703  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 765) took 2731 ms. Trace:
W20260709 13:14:35.794224  8693 connection.cc:256] RPC call timeout handler was delayed by -1.2368706180002391s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1815s for a response
W20260709 13:14:35.794059  8204 rpcz_store.cc:269] 0709 13:14:33.438303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.159078 (+720775us) service_pool.cc:224] Handling call
0709 13:14:35.758980 (+1599902us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b5400110154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:35.759100 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230919936}
I20260709 13:14:35.794548  8212 rpcz_store.cc:276] 0709 13:14:33.023504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.732804 (+2709300us) service_pool.cc:224] Handling call
0709 13:14:35.754072 (+ 21268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.754283 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:35.754318 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:35.754395 (+    77us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:35.754447 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.754515 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:35.754639 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.763903  8131 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:35.665401  8056 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:14:35.795794  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 767) took 2632 ms. Trace:
I20260709 13:14:35.796393  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:35.796742  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:35.796954  8725 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:35.797353  8728 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:35.797545  8058 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:14:35.765545  8233 debug-util.cc:398] Leaking SignalData structure 0x7b080035db20 after lost signal to thread 8237
I20260709 13:14:35.799439  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 683) took 2296 ms. Trace:
I20260709 13:14:35.772626 12567 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:35.746061 12546 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:35.748667 12698 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:35.759682 12523 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:35.806313 12698 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:35.770929  8055 rpcz_store.cc:276] 0709 13:14:32.775949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.527520 (+2751571us) service_pool.cc:224] Handling call
0709 13:14:35.527801 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.706464 (+178663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.706625 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.687587  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2850) took 3022 ms. Trace:
W20260709 13:14:35.764258  7960 rpcz_store.cc:269] 0709 13:14:08.621471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:08.621725 (+   254us) service_pool.cc:224] Handling call
0709 13:14:35.720489 (+27098764us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:31.439490 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:31.440086 (+   596us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:31.440137 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:14:31.440170 (+    33us) tablet.cc:662] Decoding operations
0709 13:14:31.441504 (+  1334us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:31.441589 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:31.441613 (+    24us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:14:31.507606 (+ 65993us) tablet.cc:701] Row locks acquired
0709 13:14:31.507658 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:14:31.507813 (+   155us) write_op.cc:270] Start()
0709 13:14:31.507941 (+   128us) write_op.cc:276] Timestamp: P: 1783602871507745 usec, L: 0
0709 13:14:31.508096 (+   155us) op_driver.cc:348] REPLICATION: starting
0709 13:14:31.534401 (+ 26305us) log.cc:844] Serialized 6300 byte log entry
0709 13:14:32.036041 (+501640us) spinlock_profiling.cc:243] Waited 7.02 ms on lock 0x7b54001732a0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf8bdbe 00007f6dbaf96930 00007f6dbaf968d8 00007f6dbaf96858 00007f6dbaf96813 00007f6dbaf95a9c 00007f6dc0e81626 00007f6dc0e7bdd8 00007f6dbafd27cb 00007f6dbafd52c0
0709 13:14:32.434122 (+398081us) op_driver.cc:464] REPLICATION: finished
0709 13:14:33.030483 (+596361us) write_op.cc:301] APPLY: starting
0709 13:14:33.030580 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:33.343216 (+312636us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:33.343262 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:33.709419 (+366157us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:33.709688 (+   269us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 13:14:33.709717 (+    29us) write_op.cc:312] APPLY: finished
0709 13:14:33.709994 (+   277us) log.cc:844] Serialized 148 byte log entry
0709 13:14:33.771114 (+ 61120us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:33.771320 (+   206us) write_op.cc:454] Released schema lock
0709 13:14:33.771401 (+    81us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:33.771840 (+   439us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:33.771875 (+    35us) write_op.cc:454] Released schema lock
0709 13:14:34.148372 (+376497us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b5400172d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01332a 00007f6dbaf98f14 00007f6dbaf98ec3 00007f6dbaf98e38 00007f6dbaf98df3 00007f6dbaf980dc 00007f6dbafc1e16 00007f6dbafa9cbb 00007f6dbaf9439d 00007f6dbaf97d75
Metrics: {"mutex_wait_us":2380,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":532828,"apply.run_cpu_time_us":36953,"apply.run_wall_time_us":741327,"cfile_cache_hit":62,"cfile_cache_hit_bytes":64790,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6217,"cfile_init":1,"lbm_read_time_us":107423,"lbm_reads_10-100_ms":3,"lbm_reads_lt_1ms":5,"num_ops":16,"prepare.queue_time_us":652978,"prepare.run_cpu_time_us":12325,"prepare.run_wall_time_us":319822,"raft.queue_time_us":325437,"raft.run_cpu_time_us":22968,"raft.run_wall_time_us":1765251,"replication_time_us":901161,"spinlock_wait_cycles":22434432,"thread_start_us":1013465,"threads_started":5,"wal-append.queue_time_us":233748,"wal-append.run_cpu_time_us":66476,"wal-append.run_wall_time_us":2522258}]]}
I20260709 13:14:35.810948  8059 rpcz_store.cc:276] 0709 13:14:32.665194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.510018 (+2844824us) service_pool.cc:224] Handling call
0709 13:14:35.510318 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.674831 (+164513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.687387 (+ 12556us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b5400335754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:35.687537 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35342720}
I20260709 13:14:35.678741  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:35.708913  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 2.06608e+06us
I20260709 13:14:35.812386  8057 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:35.812341  8029 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:35.813156  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:35.813230 12546 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:35.813349  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:35.813840  8721 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:35.785142  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1829) took 2436 ms. Trace:
I20260709 13:14:35.814627  8029 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:35.814788  8205 rpcz_store.cc:276] 0709 13:14:33.348528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.620243 (+2271715us) service_pool.cc:224] Handling call
0709 13:14:35.785088 (+164845us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.814565 12589 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:35.815646 12589 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.815809  8029 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:35.777411  8206 rpcz_store.cc:269] 0709 13:14:34.086862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.325858 (+1238996us) service_pool.cc:224] Handling call
0709 13:14:35.776970 (+451112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.816416 12610 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:35.785789  8210 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:14:35.817183  8206 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:35.817394  8210 pending_rounds.cc:85] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 5
I20260709 13:14:35.817696  8210 pending_rounds.cc:107] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:14:35.817795  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1830) took 2469 ms. Trace:
I20260709 13:14:35.817938  8210 raft_consensus.cc:2889] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:35.815987 12589 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:35.819207 12695 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:35.819780 12695 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:35.820411 12616 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:35.820377 12695 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:35.821045 12616 raft_consensus.cc:697] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:35.779529  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:35.821919 12616 consensus_queue.cc:237] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.823170  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:35.823686 12355 raft_consensus.cc:3055] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:35.824091 12355 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:35.776707  8054 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:35.790763  8061 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:14:35.750259  8133 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:35.789165  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 684) took 2228 ms. Trace:
I20260709 13:14:35.826550  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:35.826941  8062 rpcz_store.cc:276] 0709 13:14:33.560698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.788510 (+2227812us) service_pool.cc:224] Handling call
0709 13:14:35.788753 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.788899 (+   146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:35.788920 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:35.788960 (+    40us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:35.788992 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.789039 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:35.789122 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.731377 12258 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.838917 12258 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:35.838836 12523 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.839913  8132 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:35.840276  8136 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: [7.5-11.6]
I20260709 13:14:35.841455  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:35.844070 12523 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:35.798280  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:35.798038 12515 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:35.799881  8058 rpcz_store.cc:276] 0709 13:14:33.503083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.788662 (+2285579us) service_pool.cc:224] Handling call
0709 13:14:35.788887 (+   225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.799233 (+ 10346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.799395 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:35.800163  8233 debug-util.cc:398] Leaking SignalData structure 0x7b0800287580 after lost signal to thread 8754
I20260709 13:14:35.678709  8102 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:35.862681  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 184028us
I20260709 13:14:35.801151  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:35.801488  8693 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:35.802069  7959 catalog_manager.cc:5697] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 7 to 9, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 9 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:35.783794 12665 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:35.793180 12572 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.805064 12640 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:35.813903 12546 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:35.815966  8205 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
W20260709 13:14:35.788057  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 79764us
W20260709 13:14:35.882505  8026 leader_election.cc:336] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.569s (SENT)
I20260709 13:14:35.802654 12573 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:35.827490  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:35.791353  8203 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:35.787830  8207 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:35.830039  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:35.836397  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:35.841552  8725 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:35.844521 12553 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:35.846486 12301 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:35.846463 12589 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:35.851178  8175 connection.cc:256] RPC call timeout handler was delayed by -2.0806787839999288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17830000000000001s for a response
I20260709 13:14:35.796110  8204 rpcz_store.cc:276] 0709 13:14:33.163731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.795105 (+2631374us) service_pool.cc:224] Handling call
0709 13:14:35.795354 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.795510 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:35.795534 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:35.795572 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:35.795607 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.795655 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:35.795748 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.825418  8027 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:35.842234  8136 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:35.854192  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:35.791373 12159 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:35.791435 12649 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:35.856798 12657 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:35.803464 12253 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:35.793450 12550 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.877137  8211 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:14:35.877807  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:35.880712  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:35.880586  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 79163.4us
W20260709 13:14:35.883577  8026 leader_election.cc:336] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.693s (SENT)
I20260709 13:14:35.818234  8206 rpcz_store.cc:276] 0709 13:14:33.348803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.816757 (+2467954us) service_pool.cc:224] Handling call
0709 13:14:35.817016 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.817571 (+   555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.817745 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.792200 12506 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.814525 12696 consensus_queue.cc:237] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.824865 12355 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:35.881598 12298 raft_consensus.cc:697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:36.034360  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08001e8d40 after lost signal to thread 6937
W20260709 13:14:36.084174  8084 debug-util.cc:398] Leaking SignalData structure 0x7b080014e3e0 after lost signal to thread 6940
I20260709 13:14:35.885839  7959 catalog_manager.cc:5697] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 8 to 11, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 11 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:35.903443  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.900926  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2990) took 2985 ms. Trace:
I20260709 13:14:36.085911  8210 rpcz_store.cc:276] 0709 13:14:32.915898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.711368 (+2795470us) service_pool.cc:224] Handling call
0709 13:14:35.785490 (+ 74122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.818467 (+ 32977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.900756 (+ 82289us) spinlock_profiling.cc:243] Waited 52.9 ms on lock 0x7b540019ebd0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:35.900875 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121629440}
I20260709 13:14:36.086604  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:36.087108  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08001d8bc0 after lost signal to thread 7995
W20260709 13:14:35.885214  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 98230.5us
I20260709 13:14:35.886268  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1828) took 2538 ms. Trace:
W20260709 13:14:36.071511  8175 connection.cc:256] RPC call timeout handler was delayed by -1.9951787840000179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
I20260709 13:14:36.089062  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:35.898916  8209 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:36.089846  8136 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:35.987185  8131 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:36.072402  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2995) took 2984 ms. Trace:
I20260709 13:14:35.901948  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.072907  8134 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
W20260709 13:14:35.911345 12709 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:14:36.074083  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 190517us
I20260709 13:14:35.855396 12515 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.074848 12649 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:36.075824  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.076120 12550 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:35.886312 12523 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:14:36.077276  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 251909us
I20260709 13:14:36.111174  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:36.111853  8725 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:36.112577  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:35.889503  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.114094  8138 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:35.885273  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 97512.2us
I20260709 13:14:36.115103  8104 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:14:36.087567  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08001f1c20 after lost signal to thread 8026
W20260709 13:14:36.116731  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08000e8e40 after lost signal to thread 8060
W20260709 13:14:36.117287  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08002ea5e0 after lost signal to thread 8087
W20260709 13:14:36.117527  8084 debug-util.cc:398] Leaking SignalData structure 0x7b080031dde0 after lost signal to thread 8088
W20260709 13:14:36.118548  8084 debug-util.cc:398] Leaking SignalData structure 0x7b080033c020 after lost signal to thread 8162
W20260709 13:14:36.119240  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08003a9660 after lost signal to thread 8208
W20260709 13:14:36.119803  8084 debug-util.cc:398] Leaking SignalData structure 0x7b080037c780 after lost signal to thread 8236
W20260709 13:14:36.120039  8084 debug-util.cc:398] Leaking SignalData structure 0x7b080037c720 after lost signal to thread 8237
W20260709 13:14:36.120357  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08001e9d60 after lost signal to thread 8695
W20260709 13:14:36.120985  8084 debug-util.cc:398] Leaking SignalData structure 0x7b080016b9c0 after lost signal to thread 8729
I20260709 13:14:36.073503  8133 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:36.098727  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.099076 12506 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:36.099259  8726 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:14:35.888166 12629 raft_consensus.cc:3055] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:35.899057 12567 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:36.126569  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.137605  8204 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:14:36.138219  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 770) took 2545 ms. Trace:
I20260709 13:14:36.138583  8204 rpcz_store.cc:276] 0709 13:14:33.593273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.089092 (+2495819us) service_pool.cc:224] Handling call
0709 13:14:36.107370 (+ 18278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:36.137997 (+ 30627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.138170 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.888581 12665 raft_consensus.cc:697] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:36.099607 12298 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.100215 12355 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:36.083760 12640 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:36.096460  7959 catalog_manager.cc:5697] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 7 to 8, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 8 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:36.101766  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.101922  8136 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: [10.7-11.8]
I20260709 13:14:36.102363  8132 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:36.090031  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1833) took 2740 ms. Trace:
I20260709 13:14:36.155220  8209 rpcz_store.cc:276] 0709 13:14:33.349573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.898377 (+2548804us) service_pool.cc:224] Handling call
0709 13:14:35.898629 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:36.089802 (+191173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.089984 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:36.089399  8175 connection.cc:256] RPC call timeout handler was delayed by -1.8466787840000052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
W20260709 13:14:36.156675  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7395787840000594s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2162s for a response
W20260709 13:14:36.157145  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6342787840001165s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2279s for a response
W20260709 13:14:36.157509  8175 connection.cc:256] RPC call timeout handler was delayed by -1.5775787840000248s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23420000000000002s for a response
W20260709 13:14:36.157912  8175 connection.cc:256] RPC call timeout handler was delayed by -1.5775787840000248s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23420000000000002s for a response
W20260709 13:14:36.158269  8175 connection.cc:256] RPC call timeout handler was delayed by -1.277878783999995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2675s for a response
I20260709 13:14:36.088594  8205 rpcz_store.cc:276] 0709 13:14:33.348202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.815531 (+2467329us) service_pool.cc:224] Handling call
0709 13:14:35.815803 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.886024 (+ 70221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.886217 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:35.971645 12666 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:35.883217 12572 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:36.104694  8207 rpcz_store.cc:276] 0709 13:14:33.087906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.785950 (+2698044us) service_pool.cc:224] Handling call
0709 13:14:35.786190 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:36.072196 (+286006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.072355 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:36.105223  8026 leader_election.cc:336] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.748s (SENT)
I20260709 13:14:36.162528  8026 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:35.913458 12695 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.139261 12629 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:35.898495  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.855204 12630 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:14:36.121501  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08002178c0 after lost signal to thread 8754
I20260709 13:14:35.910188 12691 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:36.164417  7959 catalog_manager.cc:5697] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 8 to 11, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 11 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
W20260709 13:14:36.145849  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 291720us
I20260709 13:14:36.169431  8207 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:36.168635  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3002) took 2081 ms. Trace:
I20260709 13:14:36.172310  8029 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:35.891296 12639 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:36.017198 12694 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.122311 12590 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:14:36.127416 12520 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:14:36.122730 12649 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:36.162704 12666 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.163332  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:36.071352  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1827) took 2775 ms. Trace:
I20260709 13:14:36.143993  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1821) took 3111 ms. Trace:
I20260709 13:14:36.214576  8138 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:36.126384 12713 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
W20260709 13:14:36.170158  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08003f83e0 after lost signal to thread 12159
W20260709 13:14:36.217389  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08003c0400 after lost signal to thread 12166
W20260709 13:14:36.217753  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08003aeea0 after lost signal to thread 12253
W20260709 13:14:36.218021  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08003f10c0 after lost signal to thread 12258
W20260709 13:14:36.218355  8084 debug-util.cc:398] Leaking SignalData structure 0x7b0800285780 after lost signal to thread 12432
W20260709 13:14:36.218603  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08003f0fe0 after lost signal to thread 12434
W20260709 13:14:36.218885  8084 debug-util.cc:398] Leaking SignalData structure 0x7b0800385bc0 after lost signal to thread 12506
I20260709 13:14:36.213755  8211 rpcz_store.cc:276] 0709 13:14:33.295582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.829107 (+2533525us) service_pool.cc:224] Handling call
0709 13:14:35.876830 (+ 47723us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:36.071079 (+194249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.071301 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:36.219172  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08003a1f40 after lost signal to thread 12523
W20260709 13:14:36.170756  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 65553.8us
I20260709 13:14:36.220125 12616 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:14:36.220530  8026 leader_election.cc:336] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.908s (SENT)
I20260709 13:14:36.221077  8026 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:36.224140 12258 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:36.166283  8210 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: [12.5-12.5]
I20260709 13:14:36.173357 12253 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:14:36.225086  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3001) took 2138 ms. Trace:
I20260709 13:14:36.225435  8210 rpcz_store.cc:276] 0709 13:14:34.087099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.153461 (+2066362us) service_pool.cc:224] Handling call
0709 13:14:36.166011 (+ 12550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:36.224858 (+ 58847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.225038 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:36.172351  8204 rpcz_store.cc:276] 0709 13:14:34.087317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.153727 (+2066410us) service_pool.cc:224] Handling call
0709 13:14:36.168067 (+ 14340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:36.168267 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:36.168302 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:36.168354 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:36.168395 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.168460 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:36.168572 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:36.174053  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.226229 12629 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.150993 12693 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:36.227246 12629 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:36.014039 12698 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.227813 12520 raft_consensus.cc:2764] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:14:36.229382 12698 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:36.208521 12553 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:36.234855  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08002e79a0 after lost signal to thread 12534
I20260709 13:14:36.235029 12567 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:36.235256  8084 debug-util.cc:398] Leaking SignalData structure 0x7b0800227600 after lost signal to thread 12550
W20260709 13:14:36.235567  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08003f6720 after lost signal to thread 12553
W20260709 13:14:36.235829  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08003c7080 after lost signal to thread 12567
W20260709 13:14:36.236150  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08002924c0 after lost signal to thread 12572
W20260709 13:14:36.236408  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08003cea60 after lost signal to thread 12591
W20260709 13:14:36.236603  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08003cf4a0 after lost signal to thread 12593
W20260709 13:14:36.236851  8084 debug-util.cc:398] Leaking SignalData structure 0x7b0800360e40 after lost signal to thread 12610
W20260709 13:14:36.237044  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08001aaf20 after lost signal to thread 12616
W20260709 13:14:36.237231  8084 debug-util.cc:398] Leaking SignalData structure 0x7b080034c4a0 after lost signal to thread 12617
I20260709 13:14:36.160669 12166 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
W20260709 13:14:36.237888  8026 leader_election.cc:336] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 2.132s (SENT)
I20260709 13:14:36.167474 12695 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.215543 12666 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:36.238623 12630 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:36.177304  7959 catalog_manager.cc:5697] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 6 to 9, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 9 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:36.080202 12423 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.187853 12589 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:36.187932  8726 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:14:36.204674 12665 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:35.885918 12546 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.124204 12710 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:14:36.122833 12515 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:36.124483 12712 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:14:36.131023 12714 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:14:36.214668  8203 rpcz_store.cc:276] 0709 13:14:33.033018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.777213 (+2744195us) service_pool.cc:224] Handling call
0709 13:14:35.791184 (+ 13971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:36.071764 (+280580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.143771 (+ 72007us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b54002bdc50. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:36.143927 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31541632}
I20260709 13:14:36.084745 12573 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:36.216288  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 53016.7us
I20260709 13:14:36.225943 12159 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:14:36.237442  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08002880a0 after lost signal to thread 12630
I20260709 13:14:36.209416 12691 consensus_queue.cc:1059] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:14:36.154094 12640 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.240504  8137 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:36.179744 12694 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:36.241146  8026 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:36.240032  7959 catalog_manager.cc:5697] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:36.244928 12253 consensus_queue.cc:1059] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:14:36.265527 12531 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:36.311293 12531 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.311875 12515 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.315634  8212 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 12. Ops: []
I20260709 13:14:36.316718 12515 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:36.313597 12666 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.317706  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 768) took 2725 ms. Trace:
I20260709 13:14:36.317899  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:36.318611 12666 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:36.318712  8726 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:36.269838  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:36.323073  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 53331.7us
I20260709 13:14:36.325192  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:36.270114  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08001b47e0 after lost signal to thread 12634
I20260709 13:14:36.273820 12640 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:36.326848  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08000ca1a0 after lost signal to thread 12636
W20260709 13:14:36.327189  8084 debug-util.cc:398] Leaking SignalData structure 0x7b080031dd80 after lost signal to thread 12649
W20260709 13:14:36.327415  8084 debug-util.cc:398] Leaking SignalData structure 0x7b080037baa0 after lost signal to thread 12654
W20260709 13:14:36.327641  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08002d93e0 after lost signal to thread 12657
W20260709 13:14:36.327873  8084 debug-util.cc:398] Leaking SignalData structure 0x7b0800392e40 after lost signal to thread 12661
W20260709 13:14:36.328099  8084 debug-util.cc:398] Leaking SignalData structure 0x7b080037b140 after lost signal to thread 12665
W20260709 13:14:36.328286  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08003cbd80 after lost signal to thread 12666
W20260709 13:14:36.328497  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08003a8660 after lost signal to thread 12671
W20260709 13:14:36.328702  8084 debug-util.cc:398] Leaking SignalData structure 0x7b08002cefa0 after lost signal to thread 12674
I20260709 13:14:36.273859 12690 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
W20260709 13:14:36.266752  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 140828us
I20260709 13:14:36.276825 12573 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:36.331667 12690 consensus_queue.cc:1243] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:14:36.332677 12690 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:36.332895  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.275903  7959 catalog_manager.cc:5697] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 8 to 9, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 9 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:36.335150  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.335244  7959 catalog_manager.cc:5697] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 10 to 12, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 12 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:36.335984  8726 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:36.335922  8132 raft_consensus.cc:2376] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10. Re-sending same reply.
I20260709 13:14:36.336694  7959 catalog_manager.cc:5697] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:36.337845  8136 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:14:36.282598 12589 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:36.338567  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.339282  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.339389 12298 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:36.340096  8131 raft_consensus.cc:2410] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:36.337179  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.340723  8132 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:36.340960  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.341508  8133 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:36.341624  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.275393 12691 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:36.342733 12691 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:36.343787 12691 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.290143 12710 raft_consensus.cc:3055] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:36.345101 12710 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:36.345942  8138 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:14:36.345856 12710 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.346697  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:36.347152 12710 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:36.347524  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:36.348265 12166 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:36.348381  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:36.348639 12166 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.349049 12166 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:36.349193 12698 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:36.349617 12698 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.349957 12698 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:36.350484 12629 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:36.292253 12553 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:36.351256 12553 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.353291 12553 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:36.279969 12695 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:36.282784 12301 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:36.355064 12301 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:36.356189 12166 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.356014 12301 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.356941 12698 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.302824 12550 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.357620 12301 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:36.266429  8208 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:36.358866  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 186617us
I20260709 13:14:36.359443  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 2994) took 3319 ms. Trace:
I20260709 13:14:36.359795  8029 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:36.359764  8208 rpcz_store.cc:276] 0709 13:14:33.039567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.784325 (+2744758us) service_pool.cc:224] Handling call
0709 13:14:35.784584 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:36.358744 (+574160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.359245 (+   501us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b54001536d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:36.359395 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1107709696}
I20260709 13:14:36.326273 12694 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:36.361774 12694 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.362128 12694 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:36.363739 12723 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:36.365027 12723 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:14:36.362876  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 282792us
I20260709 13:14:36.366200  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:36.367245 12550 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:36.367492  8176 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:36.339300  8134 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:36.300384 12572 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.369406 12523 raft_consensus.cc:697] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:36.331002 12567 consensus_queue.cc:1059] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:14:36.369902  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:36.370218 12630 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:36.370523  8725 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:36.370688 12572 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:36.370746 12630 raft_consensus.cc:697] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:36.371696  8135 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:36.371850 12630 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.332985 12690 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.350070  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:36.375061 12690 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:36.375454 12567 consensus_queue.cc:1243] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:14:36.375744 12432 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:36.376973  8725 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:36.377308  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:36.378295 12166 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:36.283934 12649 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.380789 12657 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:36.381229 12657 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.318071  8212 rpcz_store.cc:276] 0709 13:14:33.592788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.899702 (+2306914us) service_pool.cc:224] Handling call
0709 13:14:35.900016 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:36.316019 (+416003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.317488 (+  1469us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b54002c9554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:36.317640 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":955415808}
I20260709 13:14:36.381630 12657 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:36.382879 12649 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:36.383517 12691 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:36.383719  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:36.384399  8725 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:36.282874  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45118.9us
I20260709 13:14:36.388105 12553 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.388437 12657 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.388571 12553 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.379549 12515 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:36.390285 12657 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:36.390724  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:36.365355 12723 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:36.393467 12723 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:36.393540 12657 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:36.394240 12657 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:36.395113 12657 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.397671  8693 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:36.395498  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.395533  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.398667  8135 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:36.338044  7959 catalog_manager.cc:5697] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 4 to 9, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 9 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:36.400406  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.400483  7959 catalog_manager.cc:5697] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index 4 to 9, term changed from 10 to 11, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65), VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted, no longer has a pending config: VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65), VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67), VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 11 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:36.402002  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 184492us
I20260709 13:14:36.396986 12550 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:36.403241  7959 catalog_manager.cc:5697] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
W20260709 13:14:36.403967  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.404595  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:36.405138  8203 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:36.405436  8725 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:36.406378  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:36.406594  8027 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:36.406903  8204 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:36.407366 12589 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.407472  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.407980  8136 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:36.407852 12589 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.409969 12589 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:36.411233  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.411502  8203 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
I20260709 13:14:36.411791  8133 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:36.411834 12589 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:36.412642 12589 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:36.412505  7959 catalog_manager.cc:5697] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 7 to 8, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 8 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:36.413228  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:36.413961  8208 raft_consensus.cc:2393] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 15: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:36.413736 12589 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.414215  7959 catalog_manager.cc:5697] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 12 to 13, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 13 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:36.418879 12553 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:36.419334  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 769) took 2826 ms. Trace:
I20260709 13:14:36.419940  8206 rpcz_store.cc:276] 0709 13:14:33.593054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.071672 (+2478618us) service_pool.cc:224] Handling call
0709 13:14:36.072433 (+   761us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:36.072822 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:36.072849 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:36.073274 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 13:14:36.418880 (+345606us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:36.418941 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.419026 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:36.419079 (+    53us) raft_consensus.cc:1685] finished
0709 13:14:36.419100 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:36.419259 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":9370,"thread_start_us":310513,"threads_started":1,"wal-append.queue_time_us":315357}
W20260709 13:14:36.421975  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 280418us
I20260709 13:14:36.390733 12515 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:36.423764 12432 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:36.424209 12432 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.424039  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:36.424654 12432 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:36.425791 12698 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:36.431879 12432 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.434018 12432 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:36.434440  8204 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
I20260709 13:14:36.434667 12550 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:36.435554 12550 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.436661  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.437655  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.438624  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.439289  8131 raft_consensus.cc:2393] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:36.439440  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.439932  8132 raft_consensus.cc:2376] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10. Re-sending same reply.
I20260709 13:14:36.440951  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.439942  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.443087  8131 raft_consensus.cc:2393] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:36.443427 12640 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.442183  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.445197  8132 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:36.446374 12640 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:36.446630  8132 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:36.448527  8134 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:36.448591  8136 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:36.481415  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.481899  8136 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:36.482475  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.416154  7959 catalog_manager.cc:5697] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
W20260709 13:14:36.484081  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.483774  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.485140  8134 raft_consensus.cc:2410] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 14 because replica has last-logged OpId of term: 12 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:14:36.484977  7959 catalog_manager.cc:5697] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 8 to 10, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 10 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:36.486328  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.449851  8137 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:36.486529  7959 catalog_manager.cc:5697] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 7 to 9, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 9 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:36.488139  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.487836  7959 catalog_manager.cc:5697] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 8 to 10, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 10 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:36.488868  8137 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:36.489028  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.489616  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.489449  7959 catalog_manager.cc:5697] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 4 to 12, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 12 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:36.490036  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.450817  8133 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:36.451025  8135 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:36.491102  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.491365  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.492090  8133 raft_consensus.cc:2368] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12.
I20260709 13:14:36.492803  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.493497  8133 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:36.491371  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.453588 12694 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.458042  8696 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:36.556545  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 98567.7us
I20260709 13:14:36.557874 12730 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:36.557930  8696 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:36.558598 12730 raft_consensus.cc:697] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:36.458540  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 60964.1us
I20260709 13:14:36.435323 12523 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.560549  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:36.560839 12665 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:36.561569  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:36.562625  8693 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:36.563308 12573 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:36.563910 12573 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:36.564384 12573 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:36.565080  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:36.459285  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 48407.3us
I20260709 13:14:36.566675 12694 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:36.460546 12515 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.464231  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:36.465004 12520 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:36.468479  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.579062  8129 raft_consensus.cc:2393] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 13: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:36.475657 12690 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.478567 12696 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:36.479349  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:36.480129 12691 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:36.612154  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:36.541518  8133 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: [9.11-9.11]
I20260709 13:14:36.541667 12723 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:36.542627  8131 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:14:36.544075  8137 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:36.617791  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.545339  8136 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:36.502435  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:36.546398  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.618868  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.619306  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.619638  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:36.616282  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 51266.2us
I20260709 13:14:36.361488  8029 leader_election.cc:400] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14.
I20260709 13:14:36.620528  8029 leader_election.cc:403] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:14:36.554874  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:36.620646  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.621272  8132 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:36.621807  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:36.546939  8026 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 2.699s (SENT) [suppressed 29 similar messages]
I20260709 13:14:36.622541  8063 raft_consensus.cc:2393] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 12: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:14:36.622763  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 75854.2us
I20260709 13:14:36.559463 12730 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:36.574314  8138 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:36.625519  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.625891  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.626653  8137 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:36.626893  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.627796  8133 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:36.470726 12666 raft_consensus.cc:3055] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:14:36.580660  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.542647  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 772) took 1346 ms. Trace:
I20260709 13:14:36.634399  8209 rpcz_store.cc:276] 0709 13:14:35.196580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.167621 (+971041us) service_pool.cc:224] Handling call
0709 13:14:36.177952 (+ 10331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:36.178142 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:14:36.361859 (+183717us) log.cc:844] Serialized 14 byte log entry
0709 13:14:36.540144 (+178285us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:36.542322 (+  2178us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:36.542431 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:36.542496 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:36.542598 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":180043,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":1513,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":96,"thread_start_us":359464,"threads_started":2,"wal-append.queue_time_us":189672}
W20260709 13:14:36.585146  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.541286  8134 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:36.614133  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.615139  8728 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:36.612792  8061 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:36.619865  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.621218  8058 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:36.571405  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:36.640535  8054 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:36.573964  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:36.620550  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.642257  8130 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:36.616268 12550 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:36.643918  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:36.622236  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:36.624120  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 141802us
W20260709 13:14:36.628998  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 267650us
I20260709 13:14:36.616580  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:36.648149  8059 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:36.648233  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:36.632985  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:36.641373  8062 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:36.651449  8132 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:14:36.653213  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:36.653882  8054 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:14:36.654352  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:36.655328  8029 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:36.655905  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:36.656416  8061 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:36.656574  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:36.657482  8026 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:36.657676  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.657402  7959 catalog_manager.cc:5697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:36.643615 12750 raft_consensus.cc:2764] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:14:36.652773  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:36.610917 12520 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:36.660084  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.670960  8056 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:36.668170  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:36.671975  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:36.610639 12515 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:36.633035 12710 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:36.660534  8135 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:36.673722  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.790071  8138 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:36.670480 12589 consensus_queue.cc:1059] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:14:36.674906  8055 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:36.662024  8058 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:36.629822 12253 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:14:36.676707  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 53752.1us
I20260709 13:14:36.672979  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:36.948135  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 275230us
I20260709 13:14:36.674594 12567 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:36.949527  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:36.644174 12610 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14.
I20260709 13:14:36.633729 12666 raft_consensus.cc:740] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:36.672782 12520 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:36.715879  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:36.787762  8209 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:36.715267  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:36.659448 12727 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:36.667570 12629 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:36.788955  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:36.973487  8210 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:36.715912  8208 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:36.718257  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:36.975615  8207 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:36.965338  8693 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:36.792284  8132 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:14:36.978622  8132 raft_consensus.cc:606] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:36.979008  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:36.793183  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:36.890674  8061 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:36.979782  8211 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:36.980167  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:36.981372  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:36.981936  8061 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:36.806277  8212 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:14:36.927852  8138 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:36.854616 12506 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:36.825923  7959 catalog_manager.cc:5697] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 5 to 10, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 10 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
W20260709 13:14:36.825747  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 168319us
I20260709 13:14:36.986284  8026 leader_election.cc:400] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12.
I20260709 13:14:36.987159  8026 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:36.988507  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:36.989205  8207 raft_consensus.cc:2410] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:14:36.991355  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.993635  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:36.994158  8129 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:36.994107  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:36.995105  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:36.995998  8726 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 13:14:36.921445 12745 log.cc:927] Time spent T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:14:36.805387  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 145136us
W20260709 13:14:36.813669  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 235551us
I20260709 13:14:36.895617  8057 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:14:36.909247  8062 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:14:36.792160  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:36.681324 12698 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:36.716977 12301 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:36.896718  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:36.674329 12553 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.118s
I20260709 13:14:36.894438 12166 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.846998 12710 raft_consensus.cc:697] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:36.672564 12572 consensus_queue.cc:1059] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.120s
I20260709 13:14:36.680537 12640 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.161s
I20260709 13:14:36.798803 12258 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:14:36.672134 12716 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.173s
I20260709 13:14:36.791766 12665 raft_consensus.cc:697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:36.948704  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:36.970137  8204 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:36.969697  8203 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:36.962659  8058 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:36.931617 12723 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:36.922219 12738 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:14:36.990602  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:37.151129  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.004863  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:36.997777 12550 raft_consensus.cc:697] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:36.969084 12432 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:14:37.022960  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:37.024438  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:37.158317  8054 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:36.962708 12649 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:36.972854 12657 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:36.962572 12159 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:37.030315  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:37.036527  8210 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:37.165179  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:37.038843 12698 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:36.962384 12755 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:37.058647  8101 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:37.094223 12740 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:14:36.973995 12751 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:14:36.977846 12702 consensus_queue.cc:1059] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:14:36.997465 12729 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:37.186503  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:37.187199  8207 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:37.134171  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 185515us
I20260709 13:14:37.135474  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:37.136850  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:37.015323 12589 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:37.199270  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:37.096930 12758 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:14:37.160019  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:37.168386 12629 raft_consensus.cc:697] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:37.165977  8210 raft_consensus.cc:2410] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:14:36.984002 12718 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:37.004465 12693 consensus_queue.cc:1059] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.150s
I20260709 13:14:37.168890  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:37.170972  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:37.171941  8209 raft_consensus.cc:2393] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:14:37.172981  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 114450us
I20260709 13:14:37.220834  8208 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:37.221513  8101 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:37.222085  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:37.222609  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:37.139441 12523 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:14:37.159050 12709 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:14:37.052654 12616 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.115s
I20260709 13:14:37.189486  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:37.193275 12772 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:37.049329 12760 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.143s
I20260709 13:14:37.178877 12159 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.201531  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:37.245846  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 44401us
I20260709 13:14:37.162114 12762 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:37.204425  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:37.051132 12691 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.097163 12716 consensus_queue.cc:1243] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:37.235008 12523 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:37.211761  8207 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:37.180239 12694 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:37.178673 12649 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:37.180774 12657 raft_consensus.cc:697] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:37.261191  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:37.263492  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:37.231308 12728 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.003s	sys 0.006s
I20260709 13:14:37.234571  8206 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:37.139158 12666 consensus_queue.cc:260] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.202793 12553 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:37.219461  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:37.266670  8212 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:37.240939  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:37.267122  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:37.267772  8206 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:37.209843 12764 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.049938 12166 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:37.243306  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 203949us
I20260709 13:14:37.241839  8204 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 13: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:37.221129 12718 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:37.234877 12696 raft_consensus.cc:2764] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:14:37.204391 12710 consensus_queue.cc:237] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:37.242324 12753 log.cc:927] Time spent T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.319s	user 0.000s	sys 0.001s
I20260709 13:14:37.179899 12713 raft_consensus.cc:2764] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:14:37.252651 12629 consensus_queue.cc:237] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.216174 12665 consensus_queue.cc:237] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.248258 12698 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:37.259121 12573 consensus_meta.cc:287] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.694s	user 0.011s	sys 0.000s
I20260709 13:14:37.287822 12649 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.309199 12573 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.290421 12666 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:37.310498  8725 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:37.286486 12718 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:37.311311 12573 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:37.264621 12756 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
I20260709 13:14:37.253454 12691 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.314280 12691 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:37.248922 12762 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.292490 12567 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:37.315613 12567 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.266433 12553 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:37.292789 12166 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:37.307775  8728 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:14:37.308192 12763 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 13:14:37.294540 12759 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:14:37.320741 12690 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:37.321697  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:37.322017  8727 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:37.308529 12159 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:37.319319  8726 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:14:37.316320 12523 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.323877 12523 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:37.324198 12515 consensus_meta.cc:287] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.611s	user 0.008s	sys 0.000s
I20260709 13:14:37.321794 12520 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.316159 12762 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:37.265733 12728 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.003s	sys 0.006s
I20260709 13:14:37.262946 12301 raft_consensus.cc:697] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:37.328810 12729 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:37.249446 12550 consensus_queue.cc:237] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.329627 12610 raft_consensus.cc:3055] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:37.329802 12657 consensus_queue.cc:237] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.291224 12764 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.317687 12553 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.336966 12729 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:37.342406 12757 raft_consensus.cc:3055] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:37.349874 12757 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:37.350554 12764 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:37.348949 12694 raft_consensus.cc:697] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:37.350697 12757 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:37.191099  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 186311us
I20260709 13:14:37.399575  8026 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:37.402918  8026 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:37.202577  8055 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:37.408787  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:37.409078  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:37.409581  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.177788  8054 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:37.411859  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:37.412178  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:37.413134  8054 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:37.209434  8061 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:37.415882  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:37.416205  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:37.416777  8061 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
I20260709 13:14:37.414099  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:37.201697  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:37.422094  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:37.422992  8061 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:37.418951  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:37.413962  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:37.423069  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:37.209651  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:37.420825  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:37.251758  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.251101  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:37.264240  8208 raft_consensus.cc:2393] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:37.491488  8208 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:37.493505  8208 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:14:37.251716  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:37.495002  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:37.495361  8209 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 13: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:37.261243  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:37.498667 12506 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:37.499586  8211 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:37.428966  8133 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:37.417171  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:37.405812  8026 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:37.429001  8057 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:37.465044 12758 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:14:37.468405  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 146642us
I20260709 13:14:37.469305 12730 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:37.470088  8062 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:37.472432 12640 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:37.217245 12766 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:14:37.509979 12771 consensus_queue.cc:1059] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 13:14:37.475338  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 265675us
I20260709 13:14:37.476289 12691 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:14:37.467388  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 46923.8us
I20260709 13:14:37.339751 12610 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:37.328109 12520 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:37.501853  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:37.461297 12755 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:14:37.515002  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:37.519769 12691 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:37.519977 12696 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:37.521591 12696 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:37.522473 12762 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:37.523029 12762 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.523394 12762 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:37.352589 12694 consensus_queue.cc:237] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.526470  8203 raft_consensus.cc:2410] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:14:37.527333 12301 consensus_queue.cc:237] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.529821 12589 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:37.530247 12589 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:37.530468 12693 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:37.521530 12691 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.534224 12589 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.535399 12727 consensus_queue.cc:1059] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:37.536494 12727 raft_consensus.cc:3055] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:37.536906 12727 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:37.538244 12589 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:37.533315  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 506001us
I20260709 13:14:37.540488  8211 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:37.530251  8203 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
W20260709 13:14:37.552636  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.555680 12778 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.555604 12610 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.556630 12778 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.557935 12610 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:37.559101 12764 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.559347 12757 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:37.560266 12766 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.560681  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:37.561854  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:37.547088  8203 pending_rounds.cc:85] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 7
I20260709 13:14:37.565851  8203 pending_rounds.cc:107] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:14:37.567291  8203 raft_consensus.cc:2889] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:37.522142 12696 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:37.538249 12727 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.570937 12764 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.572088  8063 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:37.572278  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:37.573486 12729 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.578434  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:37.579878 12772 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:37.580381  8206 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:14:37.581238 12553 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:37.586843  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:37.587033 12698 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.590684 12773 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:14:37.610425  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 387737us
I20260709 13:14:37.610700  8055 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:37.611840  8101 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:37.614418  8137 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.008s	sys 0.000s
W20260709 13:14:37.614508  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 208942us
W20260709 13:14:37.614954  8137 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.008s	sys 0.000s
I20260709 13:14:37.619812  8726 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:37.620330  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:37.621281 12713 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:37.621778  8054 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:37.624349  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:37.627105  8059 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:37.628361  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:37.629158  8060 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.016s	sys 0.011s
W20260709 13:14:37.629593  8060 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.016s	sys 0.011s
I20260709 13:14:37.630481  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2873) took 1007 ms. Trace:
I20260709 13:14:37.630681 12729 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:37.630878  8060 rpcz_store.cc:276] 0709 13:14:36.623023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.623130 (+   107us) service_pool.cc:224] Handling call
0709 13:14:36.623385 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:36.623568 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:36.652717 (+ 29149us) log.cc:844] Serialized 14 byte log entry
0709 13:14:37.629938 (+977221us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:37.630047 (+   109us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:37.630142 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:37.630245 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:37.630412 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11213,"prepare.queue_time_us":35646,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":189395,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":965552,"threads_started":2,"wal-append.queue_time_us":245819}
I20260709 13:14:37.634215  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:37.634811  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.639938  8725 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 14: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:37.642967  8203 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:37.643249  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:37.642987  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:37.644119  8207 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:37.644649  8208 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:37.645229  8727 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:37.645488 12721 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:37.608799 12693 raft_consensus.cc:2764] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:14:37.646628  8726 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:14:37.643290  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:37.636938 12698 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.653769 12698 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:37.571996 12766 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.662478  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:37.647560 12773 raft_consensus.cc:697] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:37.672024  8054 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:37.664724  8055 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:14:37.675103  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:37.675853  8062 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:37.681483  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 64863us
I20260709 13:14:37.680763  8208 pending_rounds.cc:85] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 4
I20260709 13:14:37.686964  8208 pending_rounds.cc:107] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:14:37.692948  8063 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:37.692795  8061 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:37.687280  8208 raft_consensus.cc:2889] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:37.679643 12710 consensus_queue.cc:1059] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:14:37.702512 12779 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:37.702739 12710 raft_consensus.cc:3055] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:37.703145 12779 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.698486 12773 consensus_queue.cc:237] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.710529 12546 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.711315 12546 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.713698 12730 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:37.714643  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 55642.3us
I20260709 13:14:37.738583 12531 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:37.715902 12718 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.740674 12531 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:37.741124 12531 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:37.741479 12718 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:37.742712  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1857) took 1018 ms. Trace:
I20260709 13:14:37.743105  8210 rpcz_store.cc:276] 0709 13:14:36.724185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.267979 (+543794us) service_pool.cc:224] Handling call
0709 13:14:37.268264 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:37.468397 (+200133us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:37.468439 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:37.469354 (+   915us) log.cc:844] Serialized 26 byte log entry
0709 13:14:37.604868 (+135514us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:37.604902 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:37.637857 (+ 32955us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:37.742394 (+104537us) raft_consensus.cc:1685] finished
0709 13:14:37.742417 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:37.742556 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24343,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":53,"thread_start_us":327086,"threads_started":2,"wal-append.queue_time_us":267978}
I20260709 13:14:37.745334 12716 consensus_queue.cc:1059] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:37.719280  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:37.723944 12782 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:37.750885  8101 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:37.751194 12782 raft_consensus.cc:3055] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:14:37.724416  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:37.751559  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 150301us
I20260709 13:14:37.724803 12728 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:37.725391 12789 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.135s	user 0.004s	sys 0.000s
I20260709 13:14:37.727768  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1850) took 1011 ms. Trace:
I20260709 13:14:37.728073 12567 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:14:37.703204 12710 raft_consensus.cc:740] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:37.729285 12572 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:37.730079 12520 consensus_queue.cc:1059] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:37.735736 12766 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:37.751849  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:37.688719 12772 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:37.692306 12713 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:37.757241  8205 rpcz_store.cc:276] 0709 13:14:36.717405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.717529 (+   124us) service_pool.cc:224] Handling call
0709 13:14:37.727358 (+1009829us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b54001545d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:37.727531 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1414874496}
I20260709 13:14:37.758730 12698 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:37.751685 12782 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:37.768963  8062 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:37.758271 12546 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:37.772892  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:37.764174 12772 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.762746 12779 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:37.765632 12713 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:37.774286  8058 consensus_meta.cc:287] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.006s
I20260709 13:14:37.780118  8026 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:37.785820  8058 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:37.772739 12696 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.787338  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1975) took 1297 ms. Trace:
I20260709 13:14:37.787789 12696 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:37.790419 12159 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
W20260709 13:14:37.790580 12423 consensus_queue.cc:1175] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "965e59f272f143449b18f65db4e61d98" responder_term: 14 status { last_received { term: 12 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: [13.12-13.12]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:37.792078 12159 consensus_queue.cc:1059] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:14:37.792982  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.794728  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:37.795619  8726 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:37.787768  8137 rpcz_store.cc:276] 0709 13:14:36.489719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.627096 (+137377us) service_pool.cc:224] Handling call
0709 13:14:36.627351 (+   255us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:36.627673 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:36.627697 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:36.643957 (+ 16260us) log.cc:844] Serialized 26 byte log entry
0709 13:14:37.615698 (+971741us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:37.615830 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:37.615963 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:37.663073 (+ 47110us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:37.786990 (+123917us) raft_consensus.cc:1685] finished
0709 13:14:37.787008 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:37.787252 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11349,"raft.queue_time_us":465,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":18389,"spinlock_wait_cycles":6224768,"thread_start_us":826415,"threads_started":1,"wal-append.queue_time_us":914832}
I20260709 13:14:37.796701  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:37.803617 12764 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:37.804831 12572 raft_consensus.cc:2764] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:14:37.811024  8029 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:37.812788 12710 consensus_queue.cc:260] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.814321 12710 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:37.762203 12762 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.816179 12762 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:37.817085  8057 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:37.779963 12772 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:37.819070  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 67423.9us
I20260709 13:14:37.821705 12690 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.822351 12690 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.824003 12779 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:37.825011 12690 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:37.782701 12728 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.826726  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:37.827324  8725 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:37.828548 12728 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:37.831717 12758 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:37.832839 12758 consensus_queue.cc:1059] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:37.834110  8056 raft_consensus.cc:2410] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:14:37.834203 12762 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:14:37.834579  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 64662.9us
I20260709 13:14:37.835057 12782 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.835446  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:37.769119 12778 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:37.836501 12782 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:37.837380  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:37.838560 12610 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:37.839267 12610 raft_consensus.cc:697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:37.841429 12616 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:37.841713  8728 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:37.840554 12610 consensus_queue.cc:237] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.850606  8131 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:14:37.850301  8205 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:37.859793  8210 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:14:37.785415 12713 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:37.857718  8027 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:37.791679 12727 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:37.865890  8212 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:37.873056  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 136136us
I20260709 13:14:37.875730  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:37.882036 12757 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:37.882423  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:37.892473  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:37.899283  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:37.895569  8129 consensus_meta.cc:287] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.764s	user 0.011s	sys 0.007s
I20260709 13:14:37.886757 12757 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.878693 12762 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:37.908126  8131 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:37.904543  8211 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:14:37.916254  8203 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 14: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:37.914664  8129 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:14:37.911332 12757 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:37.915459 12531 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.924875  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:37.925560  8138 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:37.926801 12531 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:37.931483  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:37.934373  8176 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:37.935065  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 80644.7us
I20260709 13:14:37.935145 12301 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:37.935597 12301 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.935940 12301 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:37.935981  8176 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:37.937084 12775 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.937685 12775 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.938361 12762 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.939874 12775 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:37.940930 12762 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:37.946796 12694 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:37.947214 12694 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.947525 12694 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:37.947645 12764 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:37.948395 12764 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.951406  8133 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:14:37.953802  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.954936 12694 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:37.956337  8729 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:14:37.956919 12764 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:37.912906 12616 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:37.958118 12616 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:37.958637 12616 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:37.958667  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:37.910986 12716 raft_consensus.cc:3055] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:37.960954 12716 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:37.961709  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:37.961833 12716 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:37.962281  8729 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:37.962983 12716 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:37.944635 12301 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:37.968444 12523 consensus_meta.cc:287] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.644s	user 0.014s	sys 0.003s
I20260709 13:14:37.968809 12301 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:37.969408  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:14:37.970418  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.971696  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:37.972378  8725 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:37.970281  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:37.978652  8729 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:14:37.980718  8176 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:37.981542 12629 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:37.982030 12629 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:37.912209 12520 consensus_queue.cc:1243] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:37.973706 12694 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:37.991863  8104 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:37.992743  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:37.994861  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:38.005897  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:37.979110 12589 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:14:38.006870  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:37.968463 12798 consensus_queue.cc:1059] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:14:37.996394 12531 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:38.010975  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:38.021927  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:37.987528 12696 consensus_queue.cc:1059] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:14:38.023483  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:38.025497  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.013214 12762 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:37.983067 12629 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:38.058439  8058 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 11: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:14:38.058940  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 53307.7us
W20260709 13:14:38.058383  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 307636us
I20260709 13:14:38.023164 12523 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:38.061615  8056 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 8: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:38.061518  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:38.062179 12520 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:38.062917  8057 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:38.065037  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:38.065512  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3009) took 1022 ms. Trace:
I20260709 13:14:38.066677  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:38.070670  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:38.022464 12690 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.079463  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:38.082540 12730 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:14:38.082719  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:38.082800  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 96630.6us
I20260709 13:14:38.083814  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:38.086553 12800 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:38.087273 12800 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:38.082744 12803 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:14:38.089480 12800 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:38.090371  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:38.078209 12728 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:14:38.091550  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:38.092259  8138 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:38.092670  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:38.093104  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:38.093431 12801 log.cc:927] Time spent T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Append to log took a long time: real 0.099s	user 0.002s	sys 0.000s
I20260709 13:14:38.094470  8138 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:38.095052  8101 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:38.095697 12572 raft_consensus.cc:3055] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:38.096159 12572 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:38.096618  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:38.097133 12572 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:38.098379  8101 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:38.098924 12727 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:38.129518  8135 raft_consensus.cc:2410] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:38.130504  8061 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:38.131790  8693 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:38.138070 12805 raft_consensus.cc:3055] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:38.138499 12805 raft_consensus.cc:740] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:38.138612 12727 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:38.139482 12805 consensus_queue.cc:260] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.139662 12727 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.140794 12805 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:38.141605 12772 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:38.142505  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 46216.7us
I20260709 13:14:38.143527 12666 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:38.143785 12772 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:38.146620 12728 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:38.147127 12728 raft_consensus.cc:697] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:38.148191  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:38.148016 12728 consensus_queue.cc:237] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:38.148960 12616 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.149469  8203 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:38.150552  8132 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:38.150453 12666 consensus_queue.cc:1059] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:38.075937  8204 rpcz_store.cc:276] 0709 13:14:37.011835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.442432 (+430597us) service_pool.cc:224] Handling call
0709 13:14:37.559784 (+117352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:37.560339 (+   555us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:37.851441 (+291102us) log.cc:844] Serialized 14 byte log entry
0709 13:14:38.024025 (+172584us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:38.024100 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:38.034235 (+ 10135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:38.034331 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:38.034488 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32749,"prepare.queue_time_us":357479,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":1277,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":23015,"thread_start_us":322880,"threads_started":2}
I20260709 13:14:38.074407 12523 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.152667 12523 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:38.152727 12690 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:38.155032  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1855) took 1431 ms. Trace:
I20260709 13:14:38.155448  8212 rpcz_store.cc:276] 0709 13:14:36.723534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.267259 (+543725us) service_pool.cc:224] Handling call
0709 13:14:37.286832 (+ 19573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:37.882036 (+595204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:38.154772 (+272736us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b54000ba9d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:38.154970 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1330647424}
I20260709 13:14:38.080157  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:38.156463  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:38.156715  8205 raft_consensus.cc:2410] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:14:38.157101  8721 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:38.101497 12716 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:38.101951 12806 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:38.101994 12798 consensus_queue.cc:1059] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:14:38.103513  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.106693  8062 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:38.109148  8209 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:38.086014  8029 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:38.081386  8134 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:14:38.094198  8133 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:38.075037  8725 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:14:38.013782 12531 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:38.098170  8138 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:38.110538 12800 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:38.125072  8131 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:38.148973  8726 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:38.067179 12775 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 13:14:38.156728 12423 raft_consensus.cc:3055] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:14:38.189877  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.160805 12690 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:38.195390  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 225038us
I20260709 13:14:38.204840  8027 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:38.205130  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 106834us
I20260709 13:14:38.199932  8696 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:38.170763 12520 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:38.192174 12531 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:38.176358 12691 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:38.212976  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 127016us
I20260709 13:14:38.192416 12423 raft_consensus.cc:740] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:38.199095 12546 raft_consensus.cc:3055] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:38.219977 12546 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:38.220464 12423 consensus_queue.cc:260] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.221211  8029 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:38.221107 12546 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.221462 12589 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:38.222175 12546 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:38.222851 12772 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:38.224049 12589 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:38.199332 12506 raft_consensus.cc:3055] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:38.224821 12506 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:14:38.217358  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 69070.5us
I20260709 13:14:38.203531 12693 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:38.218585  8061 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:14:38.210126 12610 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:38.228037  8209 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:38.225713 12506 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.209450 12716 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:38.228879 12159 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.244553 12573 raft_consensus.cc:3055] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:38.254061 12573 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:38.241056 12616 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:38.241246 12572 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:38.264578  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:38.242406 12610 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:38.271670  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:38.272740  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.260344 12573 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:38.275018  8055 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:38.260686 12159 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.284384 12159 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:38.246402 12506 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:38.263417  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:38.282503 12301 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:38.285909  8101 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:38.291551  8057 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
I20260709 13:14:38.281129 12531 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:38.238090 12691 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.300814  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:38.275301 12573 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:38.247412 12757 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:38.255179 12718 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:14:38.307623  8061 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:38.243186 12520 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.285216 12550 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:38.297185 12721 consensus_meta.cc:287] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.648s	user 0.007s	sys 0.004s
I20260709 13:14:38.291042 12301 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.337941  8129 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
W20260709 13:14:38.345759  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 72506.4us
I20260709 13:14:38.359617 12520 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:38.360265 12520 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:38.360749 12520 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:38.361573 12520 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:38.364459 12812 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:14:38.368597  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.383690 12757 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:38.385077 12721 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:38.385499  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:38.385808 12721 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.386400 12721 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.386731  8130 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:38.388391 12721 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:38.397022 12610 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:38.400964  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.298187 12523 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.343573 12301 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:38.432349  8131 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:38.415362  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:38.432674  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:38.435294  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 177795us
I20260709 13:14:38.447515  8131 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:38.449213  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:38.455696  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:38.456435  8061 raft_consensus.cc:2393] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 11: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:38.414188 12730 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:14:38.441556 12757 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:38.465531  8133 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:38.438164 12764 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.452979 12694 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:14:38.471196 12816 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:14:38.474001 12523 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:38.475116 12817 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:14:38.475325 12691 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:38.480165 12713 consensus_meta.cc:287] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.607s	user 0.006s	sys 0.002s
I20260709 13:14:38.484156  8061 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:14:38.486573 12757 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.486984 12764 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:38.466732 12531 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:38.481076 12694 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:38.503120  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:38.504737  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.511435  8101 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:38.511229 12523 consensus_queue.cc:1059] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:14:38.517098 12757 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:38.518940  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.499250 12694 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:38.524128  8727 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:38.483347 12713 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.566751  8061 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:38.504335 12773 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:14:38.498742 12828 consensus_queue.cc:1059] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
W20260709 13:14:38.611383  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 102502us
I20260709 13:14:38.617383  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:38.582727 12764 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:38.635042 12764 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:38.638495 12713 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:38.577262 12825 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:38.592561 12828 consensus_queue.cc:1243] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:38.614188  8101 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:38.675228  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.688759  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:38.689544  8209 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:38.742067  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 295307us
I20260709 13:14:38.742918  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:38.743814  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:38.744447  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:38.755129  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:38.755828  8207 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:38.757134  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 143047us
I20260709 13:14:38.692451 12757 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.741412  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:38.766371  8204 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:14:38.768290  8057 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:14:38.770408  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.771252 12730 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.771790 12730 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:38.733788 12798 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:38.765889  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:38.737893 12546 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:38.803397  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 339734us
I20260709 13:14:38.774205 12730 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:38.813179  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:38.815969  8134 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:14:38.816460  8134 pending_rounds.cc:85] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 10
I20260709 13:14:38.821590  8134 pending_rounds.cc:107] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:14:38.821872  8134 raft_consensus.cc:2889] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:38.825659  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.827126 12690 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:38.827569 12690 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.827965 12690 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:38.828855 12546 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.775902 12757 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:38.830654  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:38.831008 12546 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:38.831362  8063 raft_consensus.cc:2368] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
I20260709 13:14:38.831990 12757 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:38.832304  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:38.832918  8058 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:14:38.833333  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:38.833810  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:38.835980 12805 consensus_meta.cc:287] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.695s	user 0.008s	sys 0.006s
I20260709 13:14:38.836786  8056 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:14:38.810998 12798 raft_consensus.cc:697] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:38.838411 12798 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.827137  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:38.840104  8055 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:38.842423  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.845899  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:38.846014  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:38.846626  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:38.846822  8057 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:38.847350  8054 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:38.813433 12553 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:38.849988  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.855865  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:38.862922 12752 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:38.867904 12752 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:38.869331  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:38.870451  8057 raft_consensus.cc:2410] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:14:38.871320  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:38.872004  8057 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:38.863279 12553 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.873512 12694 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:38.824352 12572 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:14:38.875869 12694 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:38.876093  8058 raft_consensus.cc:2410] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:14:38.877259  8204 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:14:38.852365  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:38.816164 12764 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:38.903080  8059 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:38.960733  8207 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
W20260709 13:14:38.975946  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.986644  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:39.019088  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 370142us
I20260709 13:14:39.027942  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.090153  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.091197  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:38.991719 12757 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:39.097105  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.092608  8063 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:14:39.131249  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.130327  8055 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:39.133239  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.148130  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.032209 12696 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.075150 12775 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.227236 12610 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:39.227679 12301 consensus_meta.cc:287] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.788s	user 0.010s	sys 0.000s
I20260709 13:14:39.227847 12693 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:39.229239 12835 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.135s	user 0.007s	sys 0.003s
I20260709 13:14:39.134933 12772 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.232553 12828 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.233407 12757 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:39.239989  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.234261  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:39.277292 12841 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:14:39.340376  8137 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:39.234740  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:39.285317  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:39.288094  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:39.292635  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:39.120587 12808 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:14:39.281527  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:39.301817  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.302618  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:39.375593  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 73085.2us
W20260709 13:14:39.315702  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 177699us
W20260709 13:14:39.259796 12506 consensus_meta.cc:287] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.967s	user 0.006s	sys 0.005s
I20260709 13:14:39.311398  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1863) took 1055 ms. Trace:
I20260709 13:14:39.134737 12616 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:39.337224  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 43889.9us
W20260709 13:14:39.238881 12837 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.361s	user 0.000s	sys 0.001s
W20260709 13:14:39.355532  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.373327  8133 raft_consensus.cc:2410] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:14:39.376770  8209 consensus_meta.cc:287] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.685s	user 0.010s	sys 0.000s
I20260709 13:14:39.403334  8061 raft_consensus.cc:2410] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:14:39.403628  8209 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:14:39.238524 12523 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:39.378473 12690 consensus_meta.cc:287] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.550s	user 0.009s	sys 0.000s
I20260709 13:14:39.378554  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:39.339382 12610 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:39.334769 12693 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:39.384224  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:39.384864  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:39.388715  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3031) took 1188 ms. Trace:
I20260709 13:14:39.399021  8057 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:39.399276  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:39.400252  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:39.411548  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:39.405892  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:39.312296 12696 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.412487  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:39.420599  8135 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:39.483942  8209 raft_consensus.cc:2393] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:39.480391  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:39.402146 12710 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:39.374575 12721 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.330654 12301 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.410352  8208 rpcz_store.cc:276] 0709 13:14:38.200364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.200485 (+   121us) service_pool.cc:224] Handling call
0709 13:14:38.212058 (+ 11573us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:38.231428 (+ 19370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:38.231457 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:38.240365 (+  8908us) log.cc:844] Serialized 52 byte log entry
0709 13:14:38.743206 (+502841us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:38.743283 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:38.743367 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:39.388451 (+645084us) raft_consensus.cc:1685] finished
0709 13:14:39.388479 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.388649 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12389,"raft.queue_time_us":11810,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":4681,"thread_start_us":451229,"threads_started":1,"wal-append.queue_time_us":602949}
I20260709 13:14:39.480119  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:39.501309  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:39.481889  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.339959 12772 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:39.502386  8054 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:39.503448  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.505417  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.479720  8136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:14:39.482250 12728 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:39.409844  8175 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.048s	sys 0.094s
I20260709 13:14:39.480226 12820 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.424993  8026 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:39.478118  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:39.511262  8054 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:39.492662  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:39.389561  8203 rpcz_store.cc:276] 0709 13:14:38.231659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.231872 (+   213us) service_pool.cc:224] Handling call
0709 13:14:38.241402 (+  9530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:38.265560 (+ 24158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:38.265604 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:38.334678 (+ 69074us) log.cc:844] Serialized 26 byte log entry
0709 13:14:38.693436 (+358758us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:38.693471 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:38.693528 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:39.287032 (+593504us) raft_consensus.cc:1685] finished
0709 13:14:39.287060 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.287217 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1649,"raft.queue_time_us":5476,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":15111,"thread_start_us":320044,"threads_started":1,"wal-append.queue_time_us":781542}
W20260709 13:14:39.512245  8175 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.050s	sys 0.104s
W20260709 13:14:39.532011  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 766180us
I20260709 13:14:39.532298  8057 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:39.533672  8209 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:39.533581  8175 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:39.490748 12710 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.404424 12616 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:39.556155 12616 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.558207 12853 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.558786 12853 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:39.560748 12853 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.404194 12828 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:39.564080 12828 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.550127  8135 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:14:39.565464  8135 pending_rounds.cc:85] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 6
I20260709 13:14:39.565699  8135 pending_rounds.cc:107] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:14:39.565920  8135 raft_consensus.cc:2889] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:39.479848 12523 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:39.568064 12523 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.534634 12728 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.569228 12553 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.570071 12728 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:39.552309 12710 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:39.570415 12721 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.571128  8138 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:14:39.571492  8138 pending_rounds.cc:85] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 11
I20260709 13:14:39.571728  8138 pending_rounds.cc:107] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:14:39.571956  8138 raft_consensus.cc:2889] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:39.573218 12721 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:39.573525  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 721) took 1157 ms. Trace:
I20260709 13:14:39.573922  8134 rpcz_store.cc:276] 0709 13:14:38.416230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.434722 (+ 18492us) service_pool.cc:224] Handling call
0709 13:14:38.449028 (+ 14306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:38.825358 (+376330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.573315 (+747957us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b5400062054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:39.573475 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":751331072}
I20260709 13:14:39.574689 12800 raft_consensus.cc:3055] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:39.575112 12800 raft_consensus.cc:740] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:39.575918 12800 consensus_queue.cc:260] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:39.577152 12800 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:39.574724 12610 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.573544  8135 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:14:39.580390 12716 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:39.580832  8135 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:14:39.586616  8054 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:14:39.586984  8054 pending_rounds.cc:85] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 5
I20260709 13:14:39.587193  8054 pending_rounds.cc:107] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:14:39.587153  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 713) took 2000 ms. Trace:
I20260709 13:14:39.587576  8054 raft_consensus.cc:2889] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:39.587651  8130 rpcz_store.cc:276] 0709 13:14:37.586567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.597569 (+ 11002us) service_pool.cc:224] Handling call
0709 13:14:37.636004 (+ 38435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:38.387272 (+751268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.586955 (+1199683us) spinlock_profiling.cc:243] Waited 750 ms on lock 0x7b54003c87d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:39.587104 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1725638528}
I20260709 13:14:39.589105  8130 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: [8.5-10.6]
I20260709 13:14:39.589737  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.580828 12716 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:39.592540 12772 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.479441 12730 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:14:39.503245 12423 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:39.570919  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1536) took 1519 ms. Trace:
I20260709 13:14:39.573307  8138 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:14:39.601485  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:39.601886  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:39.602231  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.564929  8204 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:39.531875  8132 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:39.409561 12690 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:39.404449 12775 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.599881  8129 rpcz_store.cc:276] 0709 13:14:38.051310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.058656 (+  7346us) service_pool.cc:224] Handling call
0709 13:14:39.570759 (+1512103us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54000d20d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:39.570845 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":587450752}
I20260709 13:14:39.610430  8057 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:39.616562  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.618299  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.619516  8056 raft_consensus.cc:1813] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Handling vote request from an unknown peer d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:39.620090  8056 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:39.631812  8138 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 5. (term mismatch)
I20260709 13:14:39.632304  8138 pending_rounds.cc:85] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 4
I20260709 13:14:39.632575  8138 pending_rounds.cc:107] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 11.5
I20260709 13:14:39.632866  8138 raft_consensus.cc:2889] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:39.634260  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:39.600626 12506 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:39.635927  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:39.636664  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:39.638392  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.639073  8728 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:14:39.509137 12858 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.137s
I20260709 13:14:39.641597  8101 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:39.645617  8057 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:14:39.646136  8057 pending_rounds.cc:85] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 7
I20260709 13:14:39.646449  8057 pending_rounds.cc:107] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:14:39.646780  8057 raft_consensus.cc:2889] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:39.648461  8056 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:39.626200 12616 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.535893 12820 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:39.709335  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 303461us
W20260709 13:14:39.719728  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 60231.3us
W20260709 13:14:39.661319  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 280292us
I20260709 13:14:39.764598  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:39.662902  8059 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:39.715358  8056 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:14:39.768337  8056 pending_rounds.cc:85] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 5
I20260709 13:14:39.769359  8056 pending_rounds.cc:107] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:14:39.643440 12716 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.770603  8056 raft_consensus.cc:2889] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:39.647748 12772 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.647996 12610 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:39.766264  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 75745.2us
W20260709 13:14:39.705205 12837 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:14:39.652174 12520 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:39.788511  8057 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:39.725641 12553 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.661662 12800 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:39.738202 12757 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.726823 12546 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.799152  8104 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:39.802011  8059 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:39.803311 12506 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:39.788980 12867 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.804184 12867 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.654692 12710 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:39.807332 12696 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:39.808058 12871 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:39.808560 12871 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.808830 12710 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.809826 12775 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.811218 12820 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.811471  8693 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:39.800499 12610 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:39.808880  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.815912  8056 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:39.816831  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.817289  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:39.817436  8057 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:39.780584 12805 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:39.681074 12572 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.135s
I20260709 13:14:39.789469 12301 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:39.828351  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:39.829100  8059 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:39.831053  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:39.820889  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 56407.6us
I20260709 13:14:39.790307 12716 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:39.822415  8130 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:14:39.688571 12721 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.114s
I20260709 13:14:39.806288 12867 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:39.681676 12616 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.836846  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 701) took 1620 ms. Trace:
I20260709 13:14:39.814319 12610 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:39.837424  8062 rpcz_store.cc:276] 0709 13:14:38.210310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.216544 (+  6234us) service_pool.cc:224] Handling call
0709 13:14:38.231121 (+ 14577us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:38.273027 (+ 41906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:38.273065 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:38.296469 (+ 23404us) log.cc:844] Serialized 52 byte log entry
0709 13:14:38.512451 (+215982us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:38.514076 (+  1625us) log.cc:844] Serialized 14 byte log entry
0709 13:14:38.516853 (+  2777us) log.cc:844] Serialized 14 byte log entry
0709 13:14:38.518076 (+  1223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:38.519103 (+  1027us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:39.830585 (+1311482us) raft_consensus.cc:1685] finished
0709 13:14:39.830612 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.830787 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2462,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":11227,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":7135,"thread_start_us":116955,"threads_started":1,"wal-append.queue_time_us":447009}
W20260709 13:14:39.839901  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 184497us
I20260709 13:14:39.815472 12693 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.834748  8130 pending_rounds.cc:85] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 6
I20260709 13:14:39.846555  8130 pending_rounds.cc:107] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:14:39.823194 12775 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:39.846966  8130 raft_consensus.cc:2889] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:39.850064  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:39.849960  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:39.860832  8104 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:39.820104 12553 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:39.863492  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:39.844197 12573 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:39.820007 12546 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:39.866518  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:39.801116 12772 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.866513  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.872998 12775 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:39.845368 12752 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.873660 12616 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:39.867098  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:39.874742  8026 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:39.855211 12690 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:39.862438 12820 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.846274 12610 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.858444 12721 consensus_queue.cc:1243] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 9.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:39.905984 12713 raft_consensus.cc:3055] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:39.906384 12713 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:39.907262 12713 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.908324 12616 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:14:39.912014 12864 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.433s	user 0.000s	sys 0.003s
I20260709 13:14:39.885480 12253 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:39.913311 12253 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.915433  8727 raft_consensus.cc:2410] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:14:39.919061 12772 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:39.919586 12772 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:39.920091  8026 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:39.833858 12805 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.920825 12881 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:39.921693 12881 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:39.922550 12805 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:39.923408  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.923524 12866 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:39.924077  8056 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:39.924459  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3032) took 1067 ms. Trace:
I20260709 13:14:39.924863  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1871) took 1163 ms. Trace:
I20260709 13:14:39.925415 12866 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:39.890931  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:39.925344  8206 rpcz_store.cc:276] 0709 13:14:38.762004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.762149 (+   145us) service_pool.cc:224] Handling call
0709 13:14:38.768347 (+  6198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:38.768551 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:38.768803 (+   252us) log.cc:844] Serialized 14 byte log entry
0709 13:14:39.924554 (+1155751us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:39.924599 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:39.924644 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.924720 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.924811 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":854799,"prepare.queue_time_us":6253,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"thread_start_us":299456,"threads_started":1,"wal-append.queue_time_us":570797}
I20260709 13:14:39.926867  8721 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:39.927204  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:39.874645 12728 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:39.927456 12847 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.927829  8206 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:39.928051 12847 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.928791  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:39.891435 12757 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:39.930104  8027 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:39.895284  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:39.864970 12871 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.930959  8726 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 8.
I20260709 13:14:39.865001 12693 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:39.930981  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:39.893033  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:39.932194  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 637298us
I20260709 13:14:39.933363 12253 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.895316  8057 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:39.934620 12882 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:39.935036 12882 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.935371 12882 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:39.935848  8057 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:39.936646  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.937812  8057 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:39.898633 12572 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:39.878804 12523 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:39.934198  8027 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:39.883018 12867 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.939138 12553 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.939007 12867 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.939685 12553 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:14:39.902431  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.924801  8211 rpcz_store.cc:276] 0709 13:14:38.856596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.857732 (+  1136us) service_pool.cc:224] Handling call
0709 13:14:38.870655 (+ 12923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:38.870851 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:38.871122 (+   271us) log.cc:844] Serialized 14 byte log entry
0709 13:14:38.931340 (+ 60218us) log.cc:844] Serialized 14 byte log entry
0709 13:14:39.924033 (+992693us) spinlock_profiling.cc:243] Waited 7.64 ms on lock 0x7b180035e840. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ef72a6 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ea3c0a 00007f6dc15e3219
0709 13:14:39.924171 (+   138us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:39.924213 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:39.924258 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.924323 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.924415 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1153790,"prepare.queue_time_us":454443,"prepare.run_cpu_time_us":198,"prepare.run_wall_time_us":4068,"raft.queue_time_us":1112890,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":9449,"spinlock_wait_cycles":18305664}
I20260709 13:14:39.888244 12573 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:39.941084 12867 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:39.902772 12820 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.908545 12713 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:39.942811 12423 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:39.943106 12820 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:39.926246  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:39.943758  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.944403  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:39.944963  8208 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:39.924901  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:39.945765  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:39.946065  8056 raft_consensus.cc:2410] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:39.922164 12881 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:39.940099 12553 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.947065  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:39.941519  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:39.941864 12573 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.900252 12546 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:39.992358  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3033) took 1101 ms. Trace:
I20260709 13:14:39.992717  8207 rpcz_store.cc:276] 0709 13:14:38.891017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.508068 (+617051us) service_pool.cc:224] Handling call
0709 13:14:39.536180 (+ 28112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.536362 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:39.613902 (+ 77540us) log.cc:844] Serialized 14 byte log entry
0709 13:14:39.992078 (+378176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:39.992122 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:39.992167 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.992223 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.992314 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":147093,"prepare.queue_time_us":79142,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":5759,"thread_start_us":229390,"threads_started":1,"wal-append.queue_time_us":302281}
I20260709 13:14:39.893193 12506 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:39.994426 12881 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:39.995810 12881 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:39.996038  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 462523us
I20260709 13:14:39.997277  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:39.998090 12690 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:39.999378 12690 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:39.999691 12690 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.999998 12690 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:40.003024  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1719) took 1261 ms. Trace:
I20260709 13:14:40.003374  8061 rpcz_store.cc:276] 0709 13:14:38.741616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.742701 (+  1085us) service_pool.cc:224] Handling call
0709 13:14:39.956737 (+1214036us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b540027ae54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:40.002949 (+ 46212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1516712832}
I20260709 13:14:40.004680  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:40.005230  8729 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:40.007396  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.008145  8204 raft_consensus.cc:2410] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:14:40.010313  8136 raft_consensus.cc:606] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:40.010818  8136 raft_consensus.cc:614] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:39.995810  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:40.012108  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 85158.7us
I20260709 13:14:39.946955 12553 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:39.960384 12865 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.478s	user 0.000s	sys 0.002s
I20260709 13:14:39.918716  8059 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:39.921736  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 3034) took 1029 ms. Trace:
I20260709 13:14:40.024055  8212 rpcz_store.cc:276] 0709 13:14:38.892576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.986284 (+ 93708us) service_pool.cc:224] Handling call
0709 13:14:39.921398 (+935114us) spinlock_profiling.cc:243] Waited 7.44 ms on lock 0x7b5400208928. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0df3924 00007f6dbffdc227 00007f6dbffdbab8 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6
0709 13:14:39.921682 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17118080}
I20260709 13:14:39.901072 12610 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:39.975335  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:39.982460 12523 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:39.938174  8062 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:39.984956 12775 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:40.031345  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:39.947651 12773 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:39.987155  8209 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:14:39.987507 12847 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:39.986256 12800 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:39.921237 12772 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:39.943334 12423 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:39.989382  8208 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:40.019161  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:40.019776  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:40.031386  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:40.031476  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.031888  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1865) took 1344 ms. Trace:
I20260709 13:14:40.034066  8101 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:40.047168  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:40.032400  8206 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
W20260709 13:14:40.061795  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.050864  8138 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:40.026798 12616 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:40.053725  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:40.032406 12778 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:14:40.055580 12610 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:40.056025  8207 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:14:40.023672 12716 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:40.049955 12523 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:40.062155  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:40.115084  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 48255.3us
W20260709 13:14:40.060844  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.060376  8130 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:40.065481  8131 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:40.060353  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:40.033120 12713 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.114988  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:40.124958  8129 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:40.118158  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:40.027441 12853 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.060967  8205 rpcz_store.cc:276] 0709 13:14:38.687713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.688262 (+   549us) service_pool.cc:224] Handling call
0709 13:14:38.734569 (+ 46307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:38.759828 (+ 25259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:38.759873 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:38.760430 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:14:39.144037 (+383607us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:39.660845 (+516808us) log.cc:844] Serialized 14 byte log entry
0709 13:14:39.959603 (+298758us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:39.987611 (+ 28008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.987686 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:40.031629 (+ 43943us) raft_consensus.cc:1685] finished
0709 13:14:40.031714 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:40.031837 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":288550,"prepare.queue_time_us":192639,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":360921,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":11738,"spinlock_wait_cycles":270713600,"thread_start_us":640563,"threads_started":2,"wal-append.queue_time_us":671020}
I20260709 13:14:40.069824  8696 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:40.133544  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:40.115667  8027 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:40.119570  8062 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:14:40.118012  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:40.122788  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:40.128168  8212 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:40.136760  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.064934 12520 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:40.139791  8133 raft_consensus.cc:2410] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 4.
I20260709 13:14:40.058029 12690 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:40.129274  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:40.058977 12772 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:40.059562 12693 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:40.141499  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 71747.8us
W20260709 13:14:40.141851  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 108441us
I20260709 13:14:40.068786 12159 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:40.144667  8209 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:40.134644  8062 pending_rounds.cc:85] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 6
I20260709 13:14:40.145547  8212 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 13: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:40.068781 12253 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.146361  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.147589  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.115753 12716 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.123087 12610 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.168876  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:40.170886  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.068967 12828 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 13:14:40.122262 12616 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:40.164162  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.189833  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.165683  8062 pending_rounds.cc:107] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:14:40.192319  8062 raft_consensus.cc:2889] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:40.162983  8026 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:40.161278 12159 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.219051  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:40.224489  8728 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:40.142565 12872 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:40.153942 12506 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:40.171458 12875 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:40.178647 12860 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:14:40.179404 12690 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:40.180022  8729 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:50008
I20260709 13:14:40.180840 12423 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:40.139209 12710 consensus_queue.cc:1059] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:14:40.182756  8207 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 13:14:40.148443 12866 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:40.195415 12616 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:40.183727 12882 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:40.185680  8721 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:14:40.186198  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.146914 12853 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:40.154472 12871 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:40.187530  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.159853 12847 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:14:40.158608  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.256517  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.257369  8209 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:40.221194  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:40.219450  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:40.178268 12773 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:40.259238 12773 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:40.260442 12773 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:40.262759  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:40.263504  8724 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:40.264986 12523 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:40.268923 12853 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:40.138811 12713 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:40.271080 12713 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:40.225898  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 62990.4us
I20260709 13:14:40.220032  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:40.273908  8026 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:40.236093  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.154166 12772 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:40.245721  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.274917  8027 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:40.224902 12828 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.238000  8729 raft_consensus.cc:606] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:40.277717  8207 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:40.247812  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.232622 12546 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:40.179344 12693 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:40.159746 12800 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:14:40.167790 12757 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:14:40.172153 12253 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:40.253122  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.188978 12553 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:40.242482 12866 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.259011 12690 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:14:40.212680  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.351631  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:40.352377  8061 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
W20260709 13:14:40.352595  8060 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.003s	sys 0.029s
W20260709 13:14:40.355537  8060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.003s	sys 0.029s
I20260709 13:14:40.370074  8101 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:40.167584 12716 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:40.230201 12159 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:40.278800  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.278859  8026 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:40.283105  8728 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:14:40.191118 12610 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:40.413807  8730 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:40.281478 12867 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:40.300679 12693 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:40.306464  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:40.242077 12506 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:40.255784 12871 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.338809  8206 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:14:40.341722  8208 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:40.239297 12875 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:40.342687  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 474762us
I20260709 13:14:40.411967 12710 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:40.344672  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:40.285358 12546 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.313539 12616 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:40.294580 12828 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:40.414383  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.411108  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:40.414529  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 413878us
I20260709 13:14:40.467164  8057 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:40.426330  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:40.356024 12718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.023s	sys 0.014s
I20260709 13:14:40.347973 12866 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:40.440521  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 361987us
W20260709 13:14:40.442497  8104 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.000s	sys 0.017s
W20260709 13:14:40.459604  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.468595  8101 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:40.476918  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.437736 12867 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.411764 12713 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 13:14:40.477661  8104 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.001s	sys 0.018s
I20260709 13:14:40.449146 12693 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:40.494371  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.485148  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:40.486614  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:40.497262  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 632635us
I20260709 13:14:40.500873  8133 raft_consensus.cc:2410] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:14:40.501541  8206 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:14:40.502017  8206 pending_rounds.cc:85] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 7
I20260709 13:14:40.465813 12546 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:40.502298  8211 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.019s	sys 0.052s
W20260709 13:14:40.473640 12718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.907s	user 0.023s	sys 0.014s
I20260709 13:14:40.466053 12616 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:40.502277  8206 pending_rounds.cc:107] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:14:40.508013  8206 raft_consensus.cc:2889] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:40.503762  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.504354  8056 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:40.451164 12871 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:40.457141 12710 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:40.506407  8211 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.019s	sys 0.052s
I20260709 13:14:40.514786  8205 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:40.515259  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:40.484509 12867 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:40.500166 12523 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.519945  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:40.469112 12858 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:14:40.515246  8204 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:40.505509 12800 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:14:40.528731  8203 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.002s	sys 0.034s
I20260709 13:14:40.513804 12693 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:40.535954  8728 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:40.537248 12693 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:40.516786 12616 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:40.538569  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:40.543704  8730 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:14:40.544756  8724 raft_consensus.cc:2368] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15.
I20260709 13:14:40.545672  8725 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
W20260709 13:14:40.531101  8203 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.002s	sys 0.034s
I20260709 13:14:40.546950  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.517747 12710 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:40.550504  8029 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:40.551421  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 813) took 1049 ms. Trace:
I20260709 13:14:40.551725  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:40.552675  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.551884  8203 rpcz_store.cc:276] 0709 13:14:39.502481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.988307 (+485826us) service_pool.cc:224] Handling call
0709 13:14:39.988562 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.988731 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:39.988975 (+   244us) log.cc:844] Serialized 14 byte log entry
0709 13:14:40.551027 (+562052us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:40.551089 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:40.551150 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.551226 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:40.551348 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3693,"prepare.queue_time_us":101,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":3805,"thread_start_us":557943,"threads_started":1}
I20260709 13:14:40.557547  8133 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:40.557755  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.562016  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:40.532646 12800 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.582892  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:40.530347 12798 consensus_queue.cc:1059] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:14:40.625267  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:40.563336 12573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.034s	sys 0.062s
I20260709 13:14:40.615530  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:40.548673 12693 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.554677 12881 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:40.651526  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69215.6us
I20260709 13:14:40.651650  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:40.532486 12523 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:40.655948  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:40.584350 12800 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:40.683200 12573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.037s	sys 0.062s
I20260709 13:14:40.684163  8725 raft_consensus.cc:2410] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:14:40.684530  8724 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 8.
I20260709 13:14:40.581503 12572 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:40.630015 12757 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:40.689640  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2899) took 1824 ms. Trace:
W20260709 13:14:40.693300  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 414514us
I20260709 13:14:40.577447 12713 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:40.662173  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:40.657537 12882 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:40.683140 12553 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:40.798652  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:40.799441  8203 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
W20260709 13:14:40.800407  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 96179.1us
I20260709 13:14:40.765756  8054 rpcz_store.cc:276] 0709 13:14:38.865751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.589389 (+723638us) service_pool.cc:224] Handling call
0709 13:14:39.596880 (+  7491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.613914 (+ 17034us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:39.841155 (+227241us) log.cc:844] Serialized 14 byte log entry
0709 13:14:40.688633 (+847478us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:40.689282 (+   649us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:40.689352 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.689436 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:40.689575 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":259498,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":141133,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":6627,"thread_start_us":1035229,"threads_started":3,"wal-append.queue_time_us":835563}
I20260709 13:14:40.823313  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:40.713253 12253 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:40.801352 12423 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:40.809720 12887 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.338s	user 0.000s	sys 0.001s
I20260709 13:14:40.807618 12757 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:40.708001 12693 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:40.828727 12506 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:40.818805 12610 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:40.797931 12572 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:40.807014 12713 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:40.882867 12895 log.cc:927] Time spent T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
W20260709 13:14:40.883994 12893 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:14:40.904786  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:40.918071 12891 log.cc:927] Time spent T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:14:40.918933 12903 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.219s	user 0.000s	sys 0.000s
W20260709 13:14:40.921905  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:40.923139 12520 consensus_meta.cc:287] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.764s	user 0.004s	sys 0.010s
I20260709 13:14:40.836071 12553 consensus_queue.cc:1059] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:14:40.915952 12757 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:40.924490 12820 raft_consensus.cc:3055] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:40.952811 12820 raft_consensus.cc:740] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:40.953663 12820 consensus_queue.cc:260] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:40.861114 12775 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 13:14:40.585379  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 341102us
I20260709 13:14:40.968838  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:40.885365 12506 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:40.925427  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2914) took 1023 ms. Trace:
I20260709 13:14:40.950765 12882 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:40.953625 12520 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:40.989943  8063 rpcz_store.cc:276] 0709 13:14:39.902167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.939215 (+ 37048us) service_pool.cc:224] Handling call
0709 13:14:39.939500 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:39.939825 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:14:39.939852 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:39.940275 (+   423us) log.cc:844] Serialized 26 byte log entry
0709 13:14:40.409062 (+468787us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:40.409150 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.409244 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:40.925016 (+515772us) raft_consensus.cc:1685] finished
0709 13:14:40.925040 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:40.925339 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1726,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"thread_start_us":468371,"threads_started":1,"wal-append.queue_time_us":869064}
W20260709 13:14:41.002040  8175 leader_election.cc:336] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.532s (SENT)
I20260709 13:14:41.012400  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:40.934175 12828 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:40.614854 12875 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:41.021572  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 459632us
I20260709 13:14:41.000770 12506 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:41.028326  8029 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:41.171346  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:41.179713 12915 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:14:41.179450  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 151241us
W20260709 13:14:41.185632 12716 consensus_meta.cc:287] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.756s	user 0.000s	sys 0.010s
W20260709 13:14:41.189896 12616 consensus_meta.cc:287] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.641s	user 0.005s	sys 0.002s
W20260709 13:14:41.193652 12884 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.615s	user 0.001s	sys 0.000s
I20260709 13:14:41.198930 12572 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:41.202481 12772 consensus_meta.cc:287] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.907s	user 0.006s	sys 0.006s
I20260709 13:14:41.202883 12772 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:41.204929 12772 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:41.205406 12798 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.205895 12798 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.207921 12721 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.208513 12721 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:41.210502 12922 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.211021 12922 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:41.213038 12922 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:41.213523 12922 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.210541 12932 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.214458 12932 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.215384 12925 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.215868 12925 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.216748 12932 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:41.217965 12925 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:41.207952 12798 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:41.189978 12710 consensus_meta.cc:287] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.611s	user 0.007s	sys 0.000s
W20260709 13:14:41.198153 12546 consensus_meta.cc:287] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.682s	user 0.004s	sys 0.004s
W20260709 13:14:41.142565  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 449670us
W20260709 13:14:41.084612  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 106050us
I20260709 13:14:41.155973  8132 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:41.222528  8206 raft_consensus.cc:2393] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 16: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:41.055801  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 814) took 1542 ms. Trace:
I20260709 13:14:41.029294  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 788) took 1095 ms. Trace:
I20260709 13:14:41.070500  8133 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:14:41.167979  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2898) took 2305 ms. Trace:
I20260709 13:14:41.177484  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2900) took 2298 ms. Trace:
I20260709 13:14:41.159576 12610 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:41.029724 12506 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:41.058444 12523 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:41.016925 12820 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:41.016166 12897 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:41.225236 12866 consensus_meta.cc:287] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.729s	user 0.009s	sys 0.000s
I20260709 13:14:41.194316 12800 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:41.021534 12690 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:41.230330  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:41.230672  8060 rpcz_store.cc:276] 0709 13:14:38.879303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.590618 (+711315us) service_pool.cc:224] Handling call
0709 13:14:39.597115 (+  6497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.606431 (+  9316us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:39.818545 (+212114us) log.cc:844] Serialized 14 byte log entry
0709 13:14:41.142863 (+1324318us) log.cc:844] Serialized 14 byte log entry
0709 13:14:41.143216 (+   353us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:41.143287 (+    71us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:41.177192 (+ 33905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:41.177297 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:41.177420 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":732445,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":868,"raft.queue_time_us":159427,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":10144,"thread_start_us":1508364,"threads_started":4,"wal-append.queue_time_us":873764}
I20260709 13:14:41.261104  8054 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:41.234251 12932 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.230466  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:41.227386  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 404516us
I20260709 13:14:41.262812  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:41.231364  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:41.236441  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7794913759998963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2219s for a response
I20260709 13:14:41.237075  8727 rpcz_store.cc:276] 0709 13:14:39.931264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.939298 (+  8034us) service_pool.cc:224] Handling call
0709 13:14:39.939557 (+   259us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:39.939838 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:39.939863 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:40.022487 (+ 82624us) log.cc:844] Serialized 26 byte log entry
0709 13:14:40.342503 (+320016us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:40.343560 (+  1057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.343631 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:41.025987 (+682356us) raft_consensus.cc:1685] finished
0709 13:14:41.026010 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:41.026135 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2678,"raft.run_cpu_time_us":67,"raft.run_wall_time_us":68,"thread_start_us":319635,"threads_started":1,"wal-append.queue_time_us":555983}
I20260709 13:14:41.072659 12916 raft_consensus.cc:3055] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:14:41.113715 12909 log.cc:927] Time spent T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:14:41.078519 12913 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:41.236626 12546 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:41.231127  8211 rpcz_store.cc:276] 0709 13:14:39.513931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.992265 (+478334us) service_pool.cc:224] Handling call
0709 13:14:39.992564 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:39.992915 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:39.992941 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:39.993360 (+   419us) log.cc:844] Serialized 26 byte log entry
0709 13:14:40.514855 (+521495us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:40.514894 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.514956 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:41.055580 (+540624us) raft_consensus.cc:1685] finished
0709 13:14:41.055601 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:41.055727 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2093,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"thread_start_us":521135,"threads_started":1,"wal-append.queue_time_us":1014835}
I20260709 13:14:41.234601  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:41.235709  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:41.277053  8059 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:41.277382  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:41.235495  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:41.277676  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:41.227138  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 461616us
W20260709 13:14:41.279155  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 43519.8us
W20260709 13:14:41.111932 12899 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:14:41.237866  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:41.244150  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1549) took 1047 ms. Trace:
I20260709 13:14:41.186031 12716 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.250252  8104 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:41.116506 12693 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:41.256536  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:41.287217  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:41.291668  8063 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:41.221318 12721 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:41.229043 12871 consensus_meta.cc:287] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.709s	user 0.010s	sys 0.000s
W20260709 13:14:41.154646 12159 consensus_meta.cc:287] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.715s	user 0.005s	sys 0.005s
I20260709 13:14:41.228921  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1552) took 1001 ms. Trace:
I20260709 13:14:41.213960 12922 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.126353 12713 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:41.322855  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:41.324023  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:41.290566 12773 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:41.327018  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2022) took 1296 ms. Trace:
I20260709 13:14:41.327389  8130 rpcz_store.cc:276] 0709 13:14:40.030198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.133178 (+102980us) service_pool.cc:224] Handling call
0709 13:14:40.155657 (+ 22479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:40.156053 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:40.156091 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:40.156567 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:14:40.452876 (+296309us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:40.453079 (+   203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.453721 (+   642us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:41.326832 (+873111us) raft_consensus.cc:1685] finished
0709 13:14:41.326854 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:41.326968 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":6995,"thread_start_us":257704,"threads_started":1,"wal-append.queue_time_us":766820}
I20260709 13:14:41.329440  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:41.278295 12872 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.330449  8138 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:41.267711 12772 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:14:41.332191  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:41.262948  8057 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
W20260709 13:14:41.159080 12910 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.183s	user 0.003s	sys 0.000s
I20260709 13:14:41.227629 12616 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.336280 12716 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:41.336602 12616 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:41.244608  8058 rpcz_store.cc:276] 0709 13:14:38.862195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.584048 (+721853us) service_pool.cc:224] Handling call
0709 13:14:39.597266 (+ 13218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.597433 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:39.845294 (+247861us) log.cc:844] Serialized 14 byte log entry
0709 13:14:41.158124 (+1312830us) log.cc:844] Serialized 14 byte log entry
0709 13:14:41.159130 (+  1006us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:41.159205 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:41.159275 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:41.167795 (+  8520us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:41.167916 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":733130,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":1150,"raft.queue_time_us":110608,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":10137,"spinlock_wait_cycles":11678464,"thread_start_us":1531392,"threads_started":4,"wal-append.queue_time_us":779640}
I20260709 13:14:41.274710  8101 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:41.228905 12710 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.280037  8176 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:41.338390  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 63777.6us
W20260709 13:14:41.338708  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 58737us
I20260709 13:14:41.339568 12710 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:41.340101 12710 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:41.339923 12713 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:41.340504 12710 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.340852 12713 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:41.341423 12713 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:41.281632  8175 connection.cc:256] RPC call timeout handler was delayed by -1.7473855629998525s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2886s for a response
I20260709 13:14:41.291692  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:41.342468  8175 connection.cc:256] RPC call timeout handler was delayed by -1.6588913759999286s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23530000000000004s for a response
W20260709 13:14:41.342933  8175 connection.cc:256] RPC call timeout handler was delayed by -1.2782202780001626s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
W20260709 13:14:41.343312  8175 connection.cc:256] RPC call timeout handler was delayed by -1.2566202780001277s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
W20260709 13:14:41.343618  8175 connection.cc:256] RPC call timeout handler was delayed by -1.2323202780000884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
I20260709 13:14:41.299024 12159 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:41.293956  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44298.3us
I20260709 13:14:41.345188  8135 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:41.345820 12159 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:41.345773  8104 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:41.298748 12871 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.346452  8101 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:41.294983  8054 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:41.290141  8133 rpcz_store.cc:276] 0709 13:14:40.197086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.529434 (+332348us) service_pool.cc:224] Handling call
0709 13:14:40.557447 (+ 28013us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:41.243835 (+686388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:41.244082 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:41.295828  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:41.349274  8728 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:41.293901  8061 raft_consensus.cc:2410] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 12 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:41.296075  8132 rpcz_store.cc:276] 0709 13:14:40.227548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.155165 (+927617us) service_pool.cc:224] Handling call
0709 13:14:41.155544 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:41.228638 (+ 73094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:41.228870 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:41.275718 12916 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:41.290156 12913 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.311704  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3044) took 1906 ms. Trace:
W20260709 13:14:41.361380  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 98672.2us
I20260709 13:14:41.313016  8060 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:14:41.362124  8029 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:41.314217 12922 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:41.315400 12867 consensus_meta.cc:287] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.785s	user 0.009s	sys 0.001s
I20260709 13:14:41.320912 12800 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:41.269330 12926 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:14:41.386934  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:41.264021 12897 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:41.336850 12716 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.337463 12872 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.340885 12710 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:41.283277  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:41.344138  8175 connection.cc:256] RPC call timeout handler was delayed by -1.0838202780000756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
I20260709 13:14:41.267875 12932 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:41.260439 12866 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.350888  8102 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:41.352481  8722 raft_consensus.cc:2393] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:41.352749  8056 raft_consensus.cc:2393] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:14:41.352953  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 52934.5us
I20260709 13:14:41.364928  8136 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:41.295158 12523 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:41.365773  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:41.374282  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:41.378808  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:41.379607  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:41.379844  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:41.361039  8210 rpcz_store.cc:276] 0709 13:14:39.405703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.651036 (+245333us) service_pool.cc:224] Handling call
0709 13:14:39.660331 (+  9295us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:39.990160 (+329829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:39.990188 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:39.990839 (+   651us) log.cc:844] Serialized 52 byte log entry
0709 13:14:40.431089 (+440250us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:40.431125 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.431198 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:41.311439 (+880241us) raft_consensus.cc:1685] finished
0709 13:14:41.311467 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:41.311633 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":295050,"raft.queue_time_us":296535,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":439832,"threads_started":1,"wal-append.queue_time_us":1023179}
I20260709 13:14:41.385210  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:41.481711  8175 connection.cc:256] RPC call timeout handler was delayed by -1.0406202780000058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20910000000000004s for a response
I20260709 13:14:41.327919 12546 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:41.389919  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:41.480413  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 129599us
I20260709 13:14:41.517617  8133 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:41.401365  8132 raft_consensus.cc:2393] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:41.359138 12690 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:14:41.486912  8175 connection.cc:256] RPC call timeout handler was delayed by -1.0289202779999869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21040000000000003s for a response
I20260709 13:14:41.359570 12828 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:41.499552  8726 raft_consensus.cc:2410] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:14:41.499527  8058 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:41.500814  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:41.522687  8175 connection.cc:256] RPC call timeout handler was delayed by -1.0066855630000191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3709s for a response
I20260709 13:14:41.502025 12871 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:41.522670  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:41.498849  8130 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:41.505259  8728 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:14:41.505776  8059 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:41.441838  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:41.524922  8725 raft_consensus.cc:2393] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 16: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:41.520282  8026 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:41.524178  8210 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:41.441955  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:41.521193  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:41.527426  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:41.527946  8132 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
I20260709 13:14:41.381189 12888 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.529394  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:41.442696 12553 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:41.537523  8055 raft_consensus.cc:2393] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:41.530133 12932 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:41.530242  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:41.342257 12713 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.505815 12800 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:41.471231 12935 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:14:41.520222 12773 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:41.373530 12919 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:41.536116  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:41.539811  8130 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:41.385906 12853 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:41.533679  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 2018) took 1504 ms. Trace:
I20260709 13:14:41.546706  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:41.384656 12867 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:41.572655  8726 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 9: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:41.381745 12721 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.161s
I20260709 13:14:41.390298 12925 consensus_queue.cc:1059] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.161s
I20260709 13:14:41.550734 12938 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:41.528748  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:41.570210 12892 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.777s	user 0.002s	sys 0.001s
I20260709 13:14:41.520153 12912 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:14:41.571501  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:41.570983  8138 rpcz_store.cc:276] 0709 13:14:40.029127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.077655 (+ 48528us) service_pool.cc:224] Handling call
0709 13:14:41.533452 (+1455797us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540036ef54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:41.533613 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2750749568}
I20260709 13:14:41.525949 12828 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:41.576402  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 228920us
I20260709 13:14:41.589211  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:41.570870 12853 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:41.539618 12616 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:41.499862 12716 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:41.601459  8104 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:41.552233  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:14:41.580392  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 194851us
I20260709 13:14:41.553154 12423 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:14:41.569366  8027 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:41.582286  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2023) took 1551 ms. Trace:
I20260709 13:14:41.580969 12925 consensus_queue.cc:1243] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:14:41.583141 12938 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:41.585088 12908 log.cc:927] Time spent T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.531s	user 0.001s	sys 0.001s
I20260709 13:14:41.674691 12553 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:41.683984 12867 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:41.684542 12553 raft_consensus.cc:697] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:41.685407 12553 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.686658  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:41.687248  8138 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:41.687912 12897 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:41.683502 12913 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:41.689649 12889 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.677s	user 0.001s	sys 0.001s
I20260709 13:14:41.573691  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:41.690284  8133 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:41.577562 12932 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:41.579959  8132 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:41.602960  8029 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:41.557928 12690 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:41.540483 12888 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.606999  8131 rpcz_store.cc:276] 0709 13:14:40.030467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.133274 (+102807us) service_pool.cc:224] Handling call
0709 13:14:40.155157 (+ 21883us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:40.272864 (+117707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:40.272907 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:40.273404 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 13:14:41.186950 (+913546us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b18004b92e0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:14:41.276495 (+ 89545us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:41.297617 (+ 21122us) log.cc:844] Serialized 14 byte log entry
0709 13:14:41.329083 (+ 31466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:41.543969 (+214886us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540035d9d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:41.544034 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:41.582110 (+ 38076us) raft_consensus.cc:1685] finished
0709 13:14:41.582129 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:41.582238 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74400,"prepare.queue_time_us":59620,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":34253,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":7594,"spinlock_wait_cycles":315095040,"thread_start_us":89379,"threads_started":1,"wal-append.queue_time_us":1045684}
I20260709 13:14:41.588635 12828 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:41.505185 12916 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:41.591714 12573 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:41.691336  8210 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
W20260709 13:14:41.675211  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 145966us
I20260709 13:14:41.692842  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:41.693565  8725 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:41.602990 12853 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:41.677985 12937 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:14:41.726843  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:41.739028  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2020) took 1709 ms. Trace:
I20260709 13:14:41.727245 12866 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:41.728741  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 118142us
W20260709 13:14:41.729235  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 126349us
I20260709 13:14:41.754294  8102 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:41.729292  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:41.609532 12523 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:41.743538  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:41.744323  8133 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:41.621320 12423 consensus_queue.cc:1243] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:14:41.600186 12800 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.137s
W20260709 13:14:41.746819  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 145478us
I20260709 13:14:41.781255  8102 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:41.682857 12914 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:41.744184  8137 rpcz_store.cc:276] 0709 13:14:40.029653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.118536 (+ 88883us) service_pool.cc:224] Handling call
0709 13:14:40.118908 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:40.123802 (+  4894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:40.123843 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:40.145258 (+ 21415us) log.cc:844] Serialized 26 byte log entry
0709 13:14:40.460923 (+315665us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:40.461045 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.461193 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:41.584560 (+1123367us) spinlock_profiling.cc:243] Waited 5.23 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:41.690763 (+106203us) raft_consensus.cc:1685] finished
0709 13:14:41.690790 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:41.738967 (+ 48177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":331,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":12465,"spinlock_wait_cycles":12022656,"thread_start_us":273762,"threads_started":1,"wal-append.queue_time_us":811387}
I20260709 13:14:41.746644 12945 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:41.754505  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:41.755069  8723 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:41.682842 12938 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:41.757720 12944 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:41.682514 12773 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:41.745430 12892 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:14:41.771389 12716 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:41.773687 12690 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:41.774490  8138 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:41.774922 12872 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:41.782656  8727 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:41.755097  8027 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:41.746688  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:41.784564  8104 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:41.785014  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 1997) took 2545 ms. Trace:
I20260709 13:14:41.854825  8131 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
W20260709 13:14:41.855756  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 104075us
I20260709 13:14:41.785744  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:41.749518 12922 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:14:41.870483  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 85995.8us
I20260709 13:14:41.854296  8026 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:41.874817  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:41.875383  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 66723.6us
I20260709 13:14:41.876137  8138 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:41.877584 12926 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:41.878067 12926 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:41.879103 12926 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:41.882926  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:41.795650  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:41.797209 12952 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:14:41.800362  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:41.892230  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 96707us
I20260709 13:14:41.838397  8723 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:41.839690  8175 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:41.841065 12940 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:14:41.789114 12721 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:41.907379  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:41.792176 12914 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.781682 12523 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:41.916317 12523 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:41.798491 12772 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:41.856926  8132 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:41.858624 12866 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:14:41.860577 12934 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
W20260709 13:14:41.861069  8135 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 2019) took 1831 ms (client timeout 1935 ms). Trace:
I20260709 13:14:41.854895  8134 rpcz_store.cc:276] 0709 13:14:39.235530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.575007 (+339477us) service_pool.cc:224] Handling call
0709 13:14:39.575311 (+   304us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:39.575752 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:39.826230 (+250478us) log.cc:844] Serialized 14 byte log entry
0709 13:14:40.342209 (+515979us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:40.410614 (+ 68405us) log.cc:844] Serialized 52 byte log entry
0709 13:14:40.410788 (+   174us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:40.412729 (+  1941us) log.cc:844] Serialized 14 byte log entry
0709 13:14:40.427555 (+ 14826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.427649 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:41.220238 (+792589us) spinlock_profiling.cc:243] Waited 8.43 ms on lock 0x7b18004149a0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f29522 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:14:41.689103 (+468865us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:41.780341 (+ 91238us) raft_consensus.cc:1685] finished
0709 13:14:41.780360 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:41.780479 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16165,"prepare.queue_time_us":283866,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":3097,"raft.queue_time_us":18628,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":5845,"spinlock_wait_cycles":25603712,"thread_start_us":741592,"threads_started":2,"wal-append.queue_time_us":1061887}
I20260709 13:14:41.866137  8129 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:41.886751  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:41.893237  8029 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:41.894538  8027 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:41.895843  8137 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 9: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:41.929426  8133 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:41.903265  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:41.932827  8057 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:41.904840  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:41.906061  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:41.891093 12919 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:41.896142  8101 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:41.975128  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 79058.2us
I20260709 13:14:41.805658 12916 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:41.910916  8061 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:41.912868  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:41.979890  8211 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:41.890821 12925 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:41.910573 12159 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:41.915885 12914 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:41.918435  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:14:41.919199  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.987900  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1546) took 1811 ms. Trace:
I20260709 13:14:41.903041  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:41.925900  8135 rpcz_store.cc:269] 0709 13:14:40.029390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.059427 (+ 30037us) service_pool.cc:224] Handling call
0709 13:14:41.860824 (+1801397us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400408dd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:41.860998 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2803265280}
I20260709 13:14:41.939862  8693 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:41.889585  8026 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.935s (SENT) [suppressed 35 similar messages]
I20260709 13:14:41.938764 12953 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:41.941425 12886 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:14:41.941749  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 102173us
I20260709 13:14:41.942832 12882 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:41.943087 12866 raft_consensus.cc:697] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:42.002174 12629 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:41.944239  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:41.942826  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 49726us
I20260709 13:14:41.926149 12775 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.984699  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:41.984699  8134 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:14:42.002303  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:42.006628  8175 leader_election.cc:400] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
I20260709 13:14:42.006892  8175 leader_election.cc:403] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:14:42.002521 12882 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.007694 12882 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:42.017233  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:42.017848  8058 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:41.986770  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:41.929734  8137 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:14:41.996506  8056 raft_consensus.cc:2410] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 12 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:14:41.988256  8136 rpcz_store.cc:276] 0709 13:14:40.176701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.177744 (+  1043us) service_pool.cc:224] Handling call
0709 13:14:40.177993 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:41.486535 (+1308542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:41.987699 (+501164us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540015b654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:41.987846 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2729288448}
I20260709 13:14:41.999964  8135 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:14:42.000474 12858 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:14:42.000626  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 60846.5us
I20260709 13:14:41.939680 12888 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:41.979584 12772 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:42.005115  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:42.003903  8138 raft_consensus.cc:2410] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:14:42.068506  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1555) took 1839 ms. Trace:
I20260709 13:14:42.004346  8055 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:42.069592  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1547) took 1892 ms. Trace:
I20260709 13:14:42.004647  8134 raft_consensus.cc:606] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:42.070254  8134 raft_consensus.cc:614] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:42.069949  8129 rpcz_store.cc:276] 0709 13:14:40.177080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.177419 (+   339us) service_pool.cc:224] Handling call
0709 13:14:41.865905 (+1688486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:41.927084 (+ 61179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:42.069426 (+142342us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b5400408dd0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:42.069543 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3882525568}
I20260709 13:14:41.983168 12800 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:42.068882  8137 rpcz_store.cc:276] 0709 13:14:40.229004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.929079 (+1700075us) service_pool.cc:224] Handling call
0709 13:14:41.929454 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:42.068266 (+138812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:42.068457 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:42.072883  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:42.079051  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 74014.2us
I20260709 13:14:41.985206 12159 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:42.084120  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:42.084467  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:42.085167  8059 raft_consensus.cc:2368] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11.
I20260709 13:14:41.995808 12872 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.101418  8056 raft_consensus.cc:2368] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10.
I20260709 13:14:42.089811  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:42.101635  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:42.102033  8055 raft_consensus.cc:2376] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12. Re-sending same reply.
I20260709 13:14:42.064005 12629 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
I20260709 13:14:42.064457 12520 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.104672  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:42.062907 12858 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:14:42.106787  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 100335us
I20260709 13:14:42.106925  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:42.071067 12690 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:41.986052 12853 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:42.081588 12772 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:42.075824 12938 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:42.075357 12572 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:42.109469  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:42.110173  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:42.111909  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:42.112881  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:42.113684  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:42.082867 12919 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:42.117349  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:42.119203  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:42.108666  8061 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:42.101192 12912 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:14:42.122419  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:42.124150 12858 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:42.004228 12775 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:42.080237 12800 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.003417 12953 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 13:14:42.139873 12914 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:42.105983 12872 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:42.123956 12520 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:42.142287  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:42.154915  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:42.190275  8203 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:14:42.171264  8131 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:42.173699  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:42.180979  8726 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:42.174904  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:42.176031  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:42.196719  8211 raft_consensus.cc:2393] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:42.182307  8055 raft_consensus.cc:2368] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12.
I20260709 13:14:42.172921  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:42.174247  8058 raft_consensus.cc:2368] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:14:42.197844  8204 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:42.174189  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:42.175664  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:42.181365  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:42.181021  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:42.172158  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:42.202787  8206 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:42.203780  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:42.200871  8059 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 8.
I20260709 13:14:42.171027  8056 raft_consensus.cc:2376] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11. Re-sending same reply.
I20260709 13:14:42.003175 12866 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:42.209834  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:42.265139  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:42.364238  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 295745us
I20260709 13:14:42.412367  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:42.226819  8057 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:42.350723 12629 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:42.370096  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:42.366118 12520 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:42.428421  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:42.428999  8203 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:42.429859  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:42.366003 12506 consensus_meta.cc:287] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 1.102s	user 0.011s	sys 0.000s
W20260709 13:14:42.431245  8028 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.006s	sys 0.036s
W20260709 13:14:42.350757 12710 consensus_meta.cc:287] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.814s	user 0.011s	sys 0.000s
I20260709 13:14:42.310117 12693 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:42.432816 12710 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:42.433004 12693 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:42.435714 12710 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:42.435778 12693 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:42.296571  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:42.493607  8063 raft_consensus.cc:2376] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12. Re-sending same reply.
I20260709 13:14:42.193822  8208 raft_consensus.cc:2376] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15. Re-sending same reply.
I20260709 13:14:42.183859 12914 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:42.463816  8057 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:42.438190  8102 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:42.501264  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 63160.7us
I20260709 13:14:42.463080  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:42.310077  8212 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
W20260709 13:14:42.431737  8028 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.006s	sys 0.036s
W20260709 13:14:42.465685  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 55045.8us
I20260709 13:14:42.504668  8137 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:42.474844  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:42.184322 12872 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:42.173861 12919 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:42.179764 12775 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:42.366689 12757 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.480494  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:42.411477 12913 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:42.170861 12938 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.310s
W20260709 13:14:42.481251  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 339053us
I20260709 13:14:42.126485 12690 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:42.481283  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:42.334388 12616 consensus_meta.cc:287] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.724s	user 0.006s	sys 0.003s
I20260709 13:14:42.430781 12506 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:42.489676  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.179946 12800 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:42.334281 12908 log.cc:927] Time spent T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
W20260709 13:14:42.496023  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 606438us
W20260709 13:14:42.438606 12820 consensus_meta.cc:287] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 1.180s	user 0.010s	sys 0.001s
I20260709 13:14:42.495072  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:42.446355  8058 raft_consensus.cc:2368] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11.
I20260709 13:14:42.173483 12853 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:42.480727  8059 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:14:42.511842  8727 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:42.518442  8129 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:14:42.519352  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:42.170754 12572 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:42.573751  8175 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:42.575766  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:42.507575 12922 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.513190 12897 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:42.603917  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:42.581182 12912 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:42.519299 12773 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.005s	sys 0.013s
I20260709 13:14:42.515956 12919 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:42.174427 12159 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:42.584599  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:42.590047  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 95145.6us
I20260709 13:14:42.591739  8026 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:42.516731 12942 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.601591  8062 raft_consensus.cc:1240] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
W20260709 13:14:42.579111  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 345894us
W20260709 13:14:42.584977 12944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.008s	sys 0.034s
I20260709 13:14:42.615067  8058 raft_consensus.cc:2393] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:42.584945 12610 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.523039 12913 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:42.703152  8059 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:14:42.728241  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.728580  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:42.729188  8059 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:14:42.651552  8204 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:42.731073  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:42.731276  8135 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:14:42.732116  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:42.732401  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 140725us
I20260709 13:14:42.731079  8693 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:42.618863  8132 consensus_meta.cc:287] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.702s	user 0.007s	sys 0.002s
I20260709 13:14:42.734735  8132 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
W20260709 13:14:42.735793  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 140672us
I20260709 13:14:42.732686  8204 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:42.737540  8204 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:42.737941  8204 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:42.603153 12828 consensus_meta.cc:287] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.850s	user 0.003s	sys 0.006s
I20260709 13:14:42.586424 12616 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:42.602222 12897 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.602563 12553 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:42.522845 12757 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:42.618613 12932 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:14:42.741773 12944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.008s	sys 0.037s
W20260709 13:14:42.795128  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 221471us
W20260709 13:14:42.644162 12957 log.cc:927] Time spent T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.411s	user 0.001s	sys 0.000s
W20260709 13:14:42.735658 12773 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.005s	sys 0.028s
I20260709 13:14:42.645727 12912 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:42.617698 12253 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:42.700666 12820 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.800503 12897 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:42.807765  8205 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.000s	sys 0.013s
W20260709 13:14:42.858116  8205 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.000s	sys 0.014s
I20260709 13:14:42.515960 12772 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:42.640678 12690 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:42.775508 12966 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.746578 12914 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:42.810165  8204 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:42.641589 12956 log.cc:927] Time spent T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.431s	user 0.000s	sys 0.001s
I20260709 13:14:42.832028  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:42.607610 12922 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:42.834254  8056 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:14:42.817673 12253 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:42.816321 12775 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:42.809342 12932 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:42.775359 12828 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:42.743847 12610 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:42.890995  8204 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:42.902454  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 103367us
I20260709 13:14:42.904181  8104 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:42.746397 12942 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:42.839376 12820 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:42.888618 12690 raft_consensus.cc:2764] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:14:42.892693 12253 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:42.937354  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 121705us
I20260709 13:14:42.891577 12775 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:42.894204 12932 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:42.894604 12553 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:42.894937 12914 raft_consensus.cc:697] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:42.901808 12866 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:42.968065 12520 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:42.972754 12757 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:42.972543  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:42.944893 12506 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:42.968567 12775 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:42.889976 12966 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:42.988511  8204 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
W20260709 13:14:42.984930  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 80815.2us
I20260709 13:14:42.996610  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:42.997278  8204 pending_rounds.cc:85] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 7
I20260709 13:14:42.997594  8204 pending_rounds.cc:107] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:14:42.997867  8204 raft_consensus.cc:2889] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:42.984378  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 247913us
I20260709 13:14:43.000476  8104 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:43.028865  8175 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:43.030966  8104 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:43.002219 12853 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:43.038916 12853 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.039359 12853 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:43.039158 12914 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:43.042867 12970 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
W20260709 13:14:43.043814 12968 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
W20260709 13:14:43.045075 12858 consensus_meta.cc:287] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.869s	user 0.004s	sys 0.003s
I20260709 13:14:43.048014 12942 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.035125 12913 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.038044  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:42.973577 12553 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.028246 12713 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:43.038184  8696 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:42.978615 12866 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:43.081383  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 43693us
I20260709 13:14:43.037856 12520 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:14:43.081923  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 45252.1us
I20260709 13:14:43.035370 12710 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:43.048934 12523 consensus_meta.cc:287] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 1.132s	user 0.008s	sys 0.003s
I20260709 13:14:43.071178 12913 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:43.048834 12853 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:43.094906  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 93404.9us
I20260709 13:14:43.071290 12982 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:43.085533 12520 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.045466 12858 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.109810  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:43.089850 12912 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:43.097079 12916 consensus_meta.cc:287] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 1.112s	user 0.011s	sys 0.005s
I20260709 13:14:43.117991 12553 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:43.120472 12976 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:14:43.122601 12980 log.cc:927] Time spent T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
W20260709 13:14:43.122890 12977 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:14:43.123821  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1903) took 1009 ms. Trace:
W20260709 13:14:43.124145  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.124202  8208 rpcz_store.cc:276] 0709 13:14:42.113911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:42.494921 (+381010us) service_pool.cc:224] Handling call
0709 13:14:42.495251 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:42.516471 (+ 21220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:42.516513 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:42.580390 (+ 63877us) log.cc:844] Serialized 26 byte log entry
0709 13:14:42.817377 (+236987us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:42.817432 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:42.832299 (+ 14867us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:43.123594 (+291295us) raft_consensus.cc:1685] finished
0709 13:14:43.123615 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:43.123766 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8747,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":10506,"thread_start_us":173566,"threads_started":1,"wal-append.queue_time_us":347912}
I20260709 13:14:43.126058 12573 raft_consensus.cc:993] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:14:43.127938 12853 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:43.129892  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2928) took 1894 ms. Trace:
I20260709 13:14:43.130214 12984 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:43.130815 12984 raft_consensus.cc:697] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:43.131727 12984 consensus_queue.cc:237] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.133240 12975 raft_consensus.cc:1081] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:14:43.130244  8062 rpcz_store.cc:276] 0709 13:14:41.235215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.235389 (+   174us) service_pool.cc:224] Handling call
0709 13:14:41.258679 (+ 23290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:42.730579 (+1471900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:43.129686 (+399107us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400219154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:43.129839 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3088049920}
I20260709 13:14:43.135396  8061 raft_consensus.cc:2410] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:14:43.112980 12520 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:43.157191  8060 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:43.094110 12523 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.137181 12773 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:14:43.162017  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 430985us
W20260709 13:14:43.163084  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 80654.8us
I20260709 13:14:43.163760  8104 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:43.162727  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:43.147990 12828 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.163165  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:43.171182  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:43.158793 12820 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.111409 12913 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.175741  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:43.180939  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 96781.6us
I20260709 13:14:43.183797  8696 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:43.184533 12523 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.174362 12978 raft_consensus.cc:2764] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:14:43.178400 12782 raft_consensus.cc:2764] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:14:43.170805 12987 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:43.193210 12978 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:43.193645 12978 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.194013 12978 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:43.194726 12922 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:43.186848 12882 consensus_meta.cc:287] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 1.179s	user 0.011s	sys 0.000s
I20260709 13:14:43.189604 12982 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:43.209082  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.209278  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.212641  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.213004  8207 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:43.213202  8203 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:43.210031  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.213872  8206 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:43.215885  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.216449  8203 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:43.218612 12573 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:43.220669 12858 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:43.220916  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 70247.9us
I20260709 13:14:43.221393  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:43.221639  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:43.222677 12913 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:43.222919  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:43.223628  8102 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:43.225220 13000 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:43.197052 12996 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:43.227315  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:43.227411  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.229565  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 719) took 1880 ms. Trace:
I20260709 13:14:43.230463  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:43.231302  8208 raft_consensus.cc:2410] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:43.229967  8054 rpcz_store.cc:276] 0709 13:14:41.349156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.349358 (+   202us) service_pool.cc:224] Handling call
0709 13:14:41.349632 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:41.381552 (+ 31920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:41.381595 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:41.391174 (+  9579us) log.cc:844] Serialized 26 byte log entry
0709 13:14:42.891708 (+1500534us) spinlock_profiling.cc:243] Waited 8.21 ms on lock 0x7b180013f120. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:14:43.154587 (+262879us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:43.154638 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:43.154721 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:43.226729 (+ 72008us) raft_consensus.cc:1685] finished
0709 13:14:43.226757 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:43.229496 (+  2739us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21917,"raft.queue_time_us":13799,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":60575,"spinlock_wait_cycles":18887552,"thread_start_us":262654,"threads_started":1,"wal-append.queue_time_us":1658268}
I20260709 13:14:43.182592 12820 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:14:43.197930 12987 raft_consensus.cc:697] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:43.203495 12966 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:43.241775  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.244415  8206 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:43.241994  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:43.245694  8054 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:43.248265  8062 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:37156
I20260709 13:14:43.248859  8062 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:43.249188  8062 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:43.252287 12253 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:43.234911 12800 consensus_meta.cc:287] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.647s	user 0.009s	sys 0.001s
I20260709 13:14:43.254617 12800 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.255451 12775 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.256560 12800 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.243884 12523 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:43.257493 12523 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.257584 12775 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.257907 12523 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:43.232571 12773 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:43.258776 12773 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:43.235317 12998 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:43.261056 12998 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.261397 12998 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:43.235431 12938 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:43.240932 12573 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:43.263211 12573 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.264024 12520 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:43.265249  8060 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2935) took 1359 ms (client timeout 1689 ms). Trace:
I20260709 13:14:43.265381 12573 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:43.265657  8060 rpcz_store.cc:269] 0709 13:14:41.905698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.905821 (+   123us) service_pool.cc:224] Handling call
0709 13:14:43.265082 (+1359261us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b540028a5d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:43.265196 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1969227008}
I20260709 13:14:43.266022 12520 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.202484 12882 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.266958 12710 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:43.266860 12919 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.268000  8058 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:43.268330 12882 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.269838 12919 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.270118 12866 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:43.254066 12978 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.272384 12978 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.226547 12996 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:14:43.278415 12773 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:43.278826 12610 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.279042  8057 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:43.279234 12984 raft_consensus.cc:3055] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:43.279650 12984 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:43.280483 12984 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.281800 12984 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:43.282006  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 728) took 2002 ms. Trace:
I20260709 13:14:43.282377  8130 rpcz_store.cc:276] 0709 13:14:41.279477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.970221 (+690744us) service_pool.cc:224] Handling call
0709 13:14:41.970540 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:42.016332 (+ 45792us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:42.016373 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:42.016429 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:42.016471 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:43.281796 (+1265325us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b54001ab154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:43.281848 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:43.281956 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":104779136}
I20260709 13:14:43.235595 12982 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:43.292845 12969 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.398s	user 0.001s	sys 0.003s
I20260709 13:14:43.293871 12828 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.295806 12922 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:43.296278 12922 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.298483 12828 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.299521 12932 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:43.300244 12981 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.321s	user 0.001s	sys 0.000s
I20260709 13:14:43.300552 12828 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.298130 12922 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.301944  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:43.302881 12972 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.408s	user 0.000s	sys 0.008s
I20260709 13:14:43.303653 12520 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.278537 12987 consensus_queue.cc:237] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.306425  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:43.309711  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:43.345253  8056 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:43.311976  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:43.346514  8060 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:43.317143  8054 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
I20260709 13:14:43.268052  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1900) took 1155 ms. Trace:
I20260709 13:14:43.347652  8209 rpcz_store.cc:276] 0709 13:14:42.113127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:42.113248 (+   121us) service_pool.cc:224] Handling call
0709 13:14:42.113564 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:42.171208 (+ 57644us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:42.171239 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:42.171715 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:14:42.520324 (+348609us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:42.520360 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:43.267694 (+747334us) spinlock_profiling.cc:243] Waited 57.3 ms on lock 0x7b540010c054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:43.267739 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:43.267783 (+    44us) raft_consensus.cc:1685] finished
0709 13:14:43.267795 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:43.268001 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7182,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":131731200,"thread_start_us":348229,"threads_started":1,"wal-append.queue_time_us":446313}
I20260709 13:14:43.349730  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:43.350558 12992 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:43.350871  8104 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:43.351641 12853 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:43.352006 12999 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:43.352074 12853 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.352587 12999 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:43.352661 12853 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:43.353072 12999 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:43.354592 12716 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:43.354983 12716 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.355309 12716 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:43.343693  8062 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.365377  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1760) took 1278 ms. Trace:
W20260709 13:14:43.363394  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 769107us
I20260709 13:14:43.371781  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.372582  8209 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:43.365867  8061 rpcz_store.cc:276] 0709 13:14:42.086984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:42.174708 (+ 87724us) service_pool.cc:224] Handling call
0709 13:14:43.365146 (+1190438us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b54001e82d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:43.365308 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2160525568}
I20260709 13:14:43.379036  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1556) took 3149 ms. Trace:
I20260709 13:14:43.379511  8026 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:43.382378  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2031) took 2041 ms. Trace:
I20260709 13:14:43.385699  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.388069  8206 raft_consensus.cc:1217] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Deduplicated request from leader. Original: 8.5->[10.6-13.8]   Dedup: 10.6->[11.7-13.8]
I20260709 13:14:43.460107  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.460184  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:43.460821  8057 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:43.389266  8062 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:43.462082  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:43.390515 12866 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:43.363219 12998 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.471798  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:43.403129 12932 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:43.345296 12520 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.411332  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.481381  8203 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:43.416605  8056 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:43.379410  8133 rpcz_store.cc:276] 0709 13:14:40.229287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.932832 (+1703545us) service_pool.cc:224] Handling call
0709 13:14:41.933184 (+   352us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:41.981144 (+ 47960us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:41.981175 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:42.063833 (+ 82658us) log.cc:844] Serialized 52 byte log entry
0709 13:14:42.576129 (+512296us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:43.276258 (+700129us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b1800480e80. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf
0709 13:14:43.276389 (+   131us) log.cc:844] Serialized 14 byte log entry
0709 13:14:43.342181 (+ 65792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:43.342287 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:43.342350 (+    63us) raft_consensus.cc:1685] finished
0709 13:14:43.378847 (+ 36497us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:43.378983 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":209182,"prepare.queue_time_us":254869,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":65959,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":38351,"spinlock_wait_cycles":17133184,"thread_start_us":626491,"threads_started":2,"wal-append.queue_time_us":639211}
W20260709 13:14:43.421983  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 50595.5us
I20260709 13:14:43.554190  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:43.424817 12965 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:43.425408  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:43.427434 12828 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.427838 12820 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:43.428438 12800 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:43.419656  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.435881 12572 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:43.386329 12881 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.382830  8136 rpcz_store.cc:276] 0709 13:14:41.341273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:42.064620 (+723347us) service_pool.cc:224] Handling call
0709 13:14:42.064968 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:42.101262 (+ 36294us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:14:42.612735 (+511473us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b180038f500. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ea3c0a 00007f6dc15e3219 00007f6db7f2ba40 00007f6db7f264e4 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf 00007f6dbb030038
0709 13:14:43.165383 (+552648us) log.cc:844] Serialized 14 byte log entry
0709 13:14:43.341312 (+175929us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:43.349089 (+  7777us) log.cc:844] Serialized 26 byte log entry
0709 13:14:43.349243 (+   154us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:43.349274 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:43.349338 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:43.363958 (+ 14620us) raft_consensus.cc:1685] finished
0709 13:14:43.382128 (+ 18170us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:43.382293 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96341,"prepare.queue_time_us":689468,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":58,"raft.queue_time_us":83796,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":16288,"spinlock_wait_cycles":8681728,"thread_start_us":571414,"threads_started":2,"wal-append.queue_time_us":136834}
I20260709 13:14:43.439441 12159 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:43.443729  8727 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:14:43.444569  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:43.584993  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 354577us
I20260709 13:14:43.445184  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:43.446074  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:43.446966  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:43.447710  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:43.448697 12966 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:43.344074 12610 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:43.380179 12897 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.453559  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.458745 12253 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.362311 12982 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:43.462152  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:43.460191  8206 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: [10.6-13.8]
I20260709 13:14:43.471264  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.507656 12999 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.595036  8208 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:43.474253 12728 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:43.470744 12882 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:43.474682 12546 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:43.475050  8060 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:14:43.476624  8721 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:14:43.479367  8057 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:43.484151 12520 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:43.485711  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.480561 12914 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.483263  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:43.487299  8730 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:14:43.487680  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 887) took 1594 ms. Trace:
I20260709 13:14:43.604290  8137 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 12: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:43.604424  8726 rpcz_store.cc:276] 0709 13:14:41.886304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.886477 (+   173us) service_pool.cc:224] Handling call
0709 13:14:43.480727 (+1594250us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400422a54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:43.480853 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582819712}
I20260709 13:14:43.482656  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:43.486240  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:43.600625  8693 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:43.606914  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.607472  8206 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:43.608307  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.608731  8206 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:43.498804  8055 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1750) took 1414 ms (client timeout 1881 ms). Trace:
I20260709 13:14:43.502780  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:43.470746 12866 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:43.502929  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:43.503518 12913 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:43.504029  7958 catalog_manager.cc:5697] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 9 to 14, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 14 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:43.505191 12945 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:43.502146  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:43.511826  8209 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
W20260709 13:14:43.534492  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 336980us
I20260709 13:14:43.495244 12853 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.582593 12965 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:43.630915  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 128072us
I20260709 13:14:43.573767  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:43.629453  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.631711  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:43.631860  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58197us
I20260709 13:14:43.580938 12523 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.592759  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:43.579223 12710 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:43.579849 12423 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:43.643903  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:43.579455 12772 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:43.644436  8062 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:43.644712 12772 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.636981 12919 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:43.599157  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.646823 12919 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.647547  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.648592  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.619058 12913 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:43.649255 12913 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:43.649515  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.650056  8210 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:14:43.601219  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:43.651149  8727 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:43.601898  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.654610  8211 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:43.654968  8730 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:14:43.655480  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.656570  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.657539  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.602073  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:43.601898  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:43.658627  8060 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:43.603116 12728 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.659103  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:43.660113  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:43.660604  8057 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:43.612919 12914 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.606540  8061 raft_consensus.cc:2393] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:43.485404 12998 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:43.663738  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:43.588627 12881 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.664283  8132 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:43.610333  8058 raft_consensus.cc:2410] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:14:43.609926  8055 rpcz_store.cc:269] 0709 13:14:42.084557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:42.102784 (+ 18227us) service_pool.cc:224] Handling call
0709 13:14:43.498621 (+1395837us) spinlock_profiling.cc:243] Waited 66.7 ms on lock 0x7b5400335c54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:43.498748 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153381248}
I20260709 13:14:43.595770  8134 raft_consensus.cc:2955] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 5 to 8, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:43.617614 12986 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:43.669238 12986 raft_consensus.cc:697] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:43.670295 12986 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.617653 12882 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.603276 12867 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:14:43.613402 12984 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:14:43.604039 12920 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.685225  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.685986  8211 raft_consensus.cc:2368] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14.
I20260709 13:14:43.685475 12920 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.686682 12932 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:43.687747  8132 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:43.618992  8056 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:43.586092 12800 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:43.572713  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 147394us
I20260709 13:14:43.626919  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:43.619948  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.626531  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:43.619339  8063 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:43.625494  8102 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:43.581683 12828 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.591867  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:43.631083  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:43.635936  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:43.635984 12945 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.632035  8203 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:43.642768  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:43.617697 12546 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:43.667835  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.687928 12897 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:43.685986  8060 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:43.721243  8135 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:43.816334  8696 leader_election.cc:400] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14.
I20260709 13:14:43.819454  8696 leader_election.cc:403] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:14:43.638440 12916 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:43.689013  8062 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
I20260709 13:14:43.583351 12820 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:43.689764 12553 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:43.692870  8209 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
W20260709 13:14:43.864289  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 230886us
W20260709 13:14:43.700517  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 75091.7us
W20260709 13:14:43.694018  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58811.4us
I20260709 13:14:43.696153  8693 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:43.868211  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 172146us
I20260709 13:14:43.697016  8204 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:43.697322  8726 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:43.867372  8104 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:43.687934  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 95243us
I20260709 13:14:43.704972  8056 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:43.708112  8727 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:43.703429  8057 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:43.710997  8210 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:43.711393  8208 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:43.875175  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:43.875980  8210 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 12: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:43.877415  8208 raft_consensus.cc:1240] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:43.699692  8101 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:43.816109  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:43.878854  8132 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:14:43.755734  8206 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:43.712169  8138 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:43.598649 12966 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:43.882033  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:43.883755  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 184138us
I20260709 13:14:43.884420  8211 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:14:43.884912  8101 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:43.617738 12866 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:43.690871 12713 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:43.660629 12728 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:43.690001 12520 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:43.697713 12987 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:43.831775  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:43.905881  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:43.923947  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:43.924645  8727 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 12: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:43.924832  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:43.906030  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 194813us
I20260709 13:14:43.926160  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:43.833343 12992 raft_consensus.cc:2764] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:14:43.696630 12710 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:43.816373 12932 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:43.994442  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 67213.5us
W20260709 13:14:43.995169  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69487.3us
I20260709 13:14:43.690560 12982 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.224s
I20260709 13:14:43.995849  8696 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:43.904686 13003 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:43.915290 12871 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14.
I20260709 13:14:43.895714 12897 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:14:43.972070  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 90136.5us
I20260709 13:14:43.894375 12423 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:43.690913 12690 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.223s
I20260709 13:14:43.908715 12927 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:43.904486 13009 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:43.870931 12716 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:44.033000 13012 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:14:44.041813 12881 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:44.043226 12546 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:44.044474 12523 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:44.045063 12423 raft_consensus.cc:697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:44.012614 12962 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:44.046136 12962 raft_consensus.cc:697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:44.046032 12423 consensus_queue.cc:237] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:44.047299 12962 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.050182 12945 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:44.051064 12800 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:44.002480 12853 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:44.053195 12828 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.930263 12520 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.058516 12520 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:44.060213 12966 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:44.061648  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:44.064818  8135 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
I20260709 13:14:44.067332  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:44.068753 12984 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:44.069370 12984 raft_consensus.cc:697] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:44.070216 12919 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:43.997658 12997 consensus_queue.cc:1059] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:14:44.071442 12984 consensus_queue.cc:237] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.075667 13003 raft_consensus.cc:3055] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:44.075991 13003 raft_consensus.cc:740] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:44.076723 13003 consensus_queue.cc:260] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:44.078018 12999 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:44.086576  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 221568us
I20260709 13:14:44.088691  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:44.098269  8029 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:44.099460  8029 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:44.021701 12942 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:44.102782 12610 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:44.102882 12942 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.103262 12610 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:44.103370 12942 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:44.103530 12716 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:44.104092 12867 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:44.104488 12867 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:44.104772 12820 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:44.106125 12920 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:44.107856  8029 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:43.914597 12713 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:44.018260  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 132058us
I20260709 13:14:44.111549  8101 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:44.113297  8101 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:44.020993  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:43.992755 12253 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:43.930203 12987 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:44.003046 12998 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:44.103711 12610 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:44.104827 12867 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:44.015503  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 116998us
I20260709 13:14:44.108887 13020 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:43.920897  8203 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:44.164726  8203 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:44.108610 12945 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:14:44.116463  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 188637us
I20260709 13:14:44.130191  8101 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:14:44.173730  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 43632.9us
I20260709 13:14:44.176136  8101 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:44.019721 12959 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 13:14:44.109081  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:44.115147 12713 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:44.006989 12932 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:44.185710  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 76689.5us
I20260709 13:14:44.142200 12992 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:44.129195 12820 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:44.186451  8029 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:44.179349  8203 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:44.114552 12728 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:14:44.187870  8726 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:14:44.141259 12998 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:44.188830  8101 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:44.162243 12253 raft_consensus.cc:697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:44.115356 12913 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.164637 13020 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:44.129376 12716 consensus_queue.cc:1059] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:14:44.196781 12919 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:44.170361 12945 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.189721 12828 consensus_queue.cc:1059] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.057s
W20260709 13:14:44.238081  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.189556 12872 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:44.193995 12800 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:14:44.183128 12773 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:44.194172 12998 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:44.199139 12881 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
W20260709 13:14:44.190590  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 185654us
I20260709 13:14:44.199275 13023 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:44.158898 12710 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.198284  8203 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:44.192767 12159 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:44.267423 12945 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:44.272348  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 251420us
W20260709 13:14:44.247041  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 58284.8us
I20260709 13:14:44.293521  8101 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:44.278669 12872 raft_consensus.cc:697] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:44.299549 12872 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:44.280864 12159 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:44.199793 12820 raft_consensus.cc:697] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:44.302517 12159 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:44.303110 12820 consensus_queue.cc:237] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:44.285255 12965 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:44.287070 13019 log.cc:927] Time spent T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:14:44.288398 13023 raft_consensus.cc:697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:44.306666 12690 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:44.307386 13023 consensus_queue.cc:237] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.313153 12987 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:44.245239 12915 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:14:44.256875 13028 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:44.245494 12992 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:44.297508 12932 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:44.288940 12773 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:44.316567  8101 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:44.324673  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:44.325678  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:44.258744 13020 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:44.269658 12919 raft_consensus.cc:697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:44.285866 12942 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:44.320422  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 419882us
I20260709 13:14:44.321628  8208 raft_consensus.cc:3055] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:44.327299  8101 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:44.340386  8205 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:14:44.343448 12913 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:44.358765  8208 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:44.321357 12800 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:44.362912  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:44.367043  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:44.332562 12716 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:44.363799  8208 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.337783 12987 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:14:44.384044  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 112673us
I20260709 13:14:44.337674 13028 raft_consensus.cc:697] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:44.386780  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 59576.8us
I20260709 13:14:44.321101 12690 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:14:44.390720  8208 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:44.320580 13009 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:14:44.366936 13026 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:44.399271  8132 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:14:44.466792  8132 pending_rounds.cc:85] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 6
I20260709 13:14:44.467097  8132 pending_rounds.cc:107] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:14:44.467370  8132 raft_consensus.cc:2889] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:44.400575  8133 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:44.344872 12932 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:14:44.463001  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:44.356496 12610 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:44.482575  8135 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:44.490470  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:44.490716  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:44.491627  8138 raft_consensus.cc:2368] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10.
I20260709 13:14:44.494294  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 718) took 1039 ms. Trace:
I20260709 13:14:44.494792  8728 rpcz_store.cc:276] 0709 13:14:43.454885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.455029 (+   144us) service_pool.cc:224] Handling call
0709 13:14:43.455309 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:43.826906 (+371597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:43.826941 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:43.826980 (+    39us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:43.827021 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:44.494032 (+667011us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b540041e6d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:44.494091 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:44.494230 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":852913024}
I20260709 13:14:44.338186 12253 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.507335  8134 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:44.509332  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:44.512378  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:44.515884  8135 raft_consensus.cc:2410] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:44.519295  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:44.392112 13034 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:14:44.520522  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.566772  8138 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:44.576645  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:44.579023  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.595186  8203 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:14:44.595834  8203 pending_rounds.cc:85] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 7
I20260709 13:14:44.596158  8203 pending_rounds.cc:107] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:14:44.596508  8203 raft_consensus.cc:2889] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:44.520669  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:44.523746 13035 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:14:44.496181 13001 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.527796 12942 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:44.482375 12987 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.627629  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:44.633296  8138 raft_consensus.cc:2393] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:44.633098  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:44.627791  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1910) took 1173 ms. Trace:
I20260709 13:14:44.634017  8203 rpcz_store.cc:276] 0709 13:14:43.454485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.824344 (+369859us) service_pool.cc:224] Handling call
0709 13:14:43.824679 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:44.627403 (+802724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:44.627679 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:44.635130  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 437635us
I20260709 13:14:44.637001  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:44.637648  8132 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:44.638602  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:44.639338  8209 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:44.640204  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:44.640516  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:44.641108  8206 raft_consensus.cc:2393] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:44.642182  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:44.642942  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:44.656721  8135 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:44.657249  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:44.585866 13026 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:44.660802  8137 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 740) took 2150 ms (client timeout 2239 ms). Trace:
W20260709 13:14:44.661427  8137 rpcz_store.cc:269] 0709 13:14:42.505563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:42.519133 (+ 13570us) service_pool.cc:224] Handling call
0709 13:14:44.655354 (+2136221us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540008cad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:44.655616 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2334246272}
I20260709 13:14:44.581946 13034 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:44.515928 12867 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.663909  8208 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:14:44.663684  8203 raft_consensus.cc:3055] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:44.673182  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:44.727372  8205 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:44.727669  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:44.673987  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1905) took 2547 ms. Trace:
I20260709 13:14:44.730168  8203 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:44.734828  8062 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:44.735206  8203 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:44.738808  8203 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:44.507252 12992 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.496549 12919 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:44.728291  8212 rpcz_store.cc:276] 0709 13:14:42.114485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:42.502999 (+388514us) service_pool.cc:224] Handling call
0709 13:14:42.503348 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:43.460917 (+957569us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:14:43.461198 (+   281us) log.cc:844] Serialized 14 byte log entry
0709 13:14:43.493353 (+ 32155us) spinlock_profiling.cc:243] Waited 976 us on lock 0x7b48002301d8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:14:43.594369 (+101016us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:43.594832 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 13:14:43.594968 (+   136us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:43.873695 (+278727us) spinlock_profiling.cc:243] Waited 936 us on lock 0x7b180045cb80. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf 00007f6dbb030038
0709 13:14:43.984979 (+111284us) log.cc:844] Serialized 14 byte log entry
0709 13:14:43.985326 (+   347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:44.661539 (+676213us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b54000d5054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:44.661580 (+    41us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:44.661624 (+    44us) raft_consensus.cc:1685] finished
0709 13:14:44.661636 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:44.661752 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":485207,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":16711,"raft.run_cpu_time_us":2520,"raft.run_wall_time_us":33894,"spinlock_wait_cycles":2205758336,"thread_start_us":211282,"threads_started":2,"wal-append.queue_time_us":555709}
I20260709 13:14:44.748219  8203 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:14:44.759739 12610 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:44.659436 12987 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.660845 13028 consensus_queue.cc:237] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:44.776134  8137 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:14:44.635962 13001 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.788178 13001 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:44.793326  8724 raft_consensus.cc:3055] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:44.793721  8724 raft_consensus.cc:740] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:44.794458  8724 consensus_queue.cc:260] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.795238  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 743) took 1169 ms. Trace:
I20260709 13:14:44.795313  8724 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:44.795679  8136 rpcz_store.cc:276] 0709 13:14:43.625913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.626641 (+   728us) service_pool.cc:224] Handling call
0709 13:14:44.795042 (+1168401us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b54004097d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:44.795183 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1772957696}
W20260709 13:14:44.799005 12998 consensus_meta.cc:287] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.509s	user 0.006s	sys 0.002s
I20260709 13:14:44.664511 12159 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.804127  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:44.676122 13026 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.806306 13034 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:44.807374 13026 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:44.818177  8061 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:44.822852 12920 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:44.823117  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 745) took 1196 ms. Trace:
I20260709 13:14:44.823364 12920 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:44.823485  8130 rpcz_store.cc:276] 0709 13:14:43.626475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.626582 (+   107us) service_pool.cc:224] Handling call
0709 13:14:43.627067 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:43.688482 (+ 61415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:43.688527 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:43.689055 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:14:44.104739 (+415684us) spinlock_profiling.cc:243] Waited 9.79 ms on lock 0x7b18002172a0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:14:44.190231 (+ 85492us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:44.190276 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:44.190374 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:44.822837 (+632463us) raft_consensus.cc:1685] finished
0709 13:14:44.822856 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:44.823064 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14194,"raft.queue_time_us":110,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":11297,"spinlock_wait_cycles":22627968,"thread_start_us":85317,"threads_started":1,"wal-append.queue_time_us":649324}
I20260709 13:14:44.826570  8029 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:44.828572  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:44.766125 12987 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:44.832672  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:44.834169 12610 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:44.834666 12610 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.835086 12610 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:44.849071  8135 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:44.801242 12914 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:44.857407  8724 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
W20260709 13:14:44.847389 12713 consensus_meta.cc:287] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.001s
W20260709 13:14:44.858057  8724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 813) took 1413 ms (client timeout 1519 ms). Trace:
I20260709 13:14:44.857834 12914 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:14:44.872364  8724 rpcz_store.cc:269] 0709 13:14:43.444528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.444974 (+   446us) service_pool.cc:224] Handling call
0709 13:14:44.793216 (+1348242us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b540017b4a0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf8ddc0 00007f6dbb019e18 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6
0709 13:14:44.857988 (+ 64772us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4317568}
I20260709 13:14:44.861148 12962 raft_consensus.cc:3055] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:44.825500 12920 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:44.863502 12159 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:44.864768 12867 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:44.846673 12913 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:14:44.848685 12772 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:44.799403 12998 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:44.872535  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:14:44.877310  8026 leader_election.cc:336] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.935s (SENT)
I20260709 13:14:44.878379  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:44.885665  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:44.892269  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:44.899125  8054 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:45.223042  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:45.223766  8136 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:45.224617  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:45.225231  8136 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
W20260709 13:14:45.224838  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 392238us
I20260709 13:14:45.227056  8029 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:45.206130  7959 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34120 (request call id 248) took 19265 ms (client timeout 15000 ms). Trace:
I20260709 13:14:45.228739 13023 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:45.229079  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 752) took 1565 ms. Trace:
I20260709 13:14:45.226133  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:45.229197 13023 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:45.229688 13023 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:45.230034 12920 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:14:45.235117  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.883610 12962 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:45.236013  8063 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 2951) took 1774 ms (client timeout 2174 ms). Trace:
W20260709 13:14:45.235493 13020 consensus_meta.cc:287] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.878s	user 0.010s	sys 0.000s
I20260709 13:14:45.236356 12962 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:45.237195 13020 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:45.237735 12962 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:45.238364  8062 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:44.881444 12713 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:45.236500  8063 rpcz_store.cc:269] 0709 13:14:43.461773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.461938 (+   165us) service_pool.cc:224] Handling call
0709 13:14:45.235832 (+1773894us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b540016a654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:45.235960 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57663744}
I20260709 13:14:45.240078  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:45.240140  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:45.241277  8131 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:45.241466  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.235000 13003 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:45.240427 12713 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:45.170302  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:45.257328  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:45.258031  8206 raft_consensus.cc:2393] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:45.243666  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:44.881070  7958 catalog_manager.cc:5697] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 9 to 10, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 10 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.139964  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:44.881619 12945 consensus_meta.cc:287] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.561s	user 0.000s	sys 0.009s
I20260709 13:14:45.204874  8729 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:14:44.998935  8175 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.026s	sys 0.035s
W20260709 13:14:45.269766  8175 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.027s	sys 0.045s
W20260709 13:14:45.270321  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 952106us
I20260709 13:14:44.894279  8055 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:14:45.159201  8138 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:14:44.985526  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:45.272593  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:45.248013  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.249964  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:45.195621  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:45.273897  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:45.252024  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:45.205780 13043 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.403s	user 0.000s	sys 0.001s
W20260709 13:14:45.151942 12572 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.033s	sys 0.041s
W20260709 13:14:45.168418  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.255110  8207 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:45.281771  8727 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:45.282009  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:45.255528  8057 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
W20260709 13:14:44.881331 12866 consensus_meta.cc:287] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.964s	user 0.009s	sys 0.002s
I20260709 13:14:45.285718 12866 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:45.256475  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:45.262326  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:45.287180  8060 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:45.288017 12866 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:45.245256  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:45.289327  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 412041us
I20260709 13:14:45.289755  8062 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:45.290886 12866 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:45.291016  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.291810  8057 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 11: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:45.292505  7958 catalog_manager.cc:5697] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.266690 12820 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:45.228303  7959 rpcz_store.cc:269] 0709 13:14:25.608168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.608459 (+   291us) service_pool.cc:224] Handling call
0709 13:14:44.873251 (+19264792us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:37.591008 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:37.591992 (+   984us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:37.592057 (+    65us) write_op.cc:435] Acquired schema lock
0709 13:14:37.592107 (+    50us) tablet.cc:662] Decoding operations
0709 13:14:37.703955 (+111848us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:37.704030 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:37.704059 (+    29us) tablet.cc:685] Acquiring locks for 24 operations
0709 13:14:37.733092 (+ 29033us) tablet.cc:701] Row locks acquired
0709 13:14:37.733144 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:14:37.733319 (+   175us) write_op.cc:270] Start()
0709 13:14:37.733490 (+   171us) write_op.cc:276] Timestamp: P: 1783602877733260 usec, L: 0
0709 13:14:37.733514 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:14:37.734306 (+   792us) log.cc:844] Serialized 9401 byte log entry
0709 13:14:38.347031 (+612725us) op_driver.cc:464] REPLICATION: finished
0709 13:14:38.519582 (+172551us) write_op.cc:301] APPLY: starting
0709 13:14:38.519723 (+   141us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:38.832215 (+312492us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:38.832328 (+   113us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:39.890730 (+1058402us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:39.891218 (+   488us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=0,mrs_lookups=0
0709 13:14:39.891267 (+    49us) write_op.cc:312] APPLY: finished
0709 13:14:39.891791 (+   524us) log.cc:844] Serialized 212 byte log entry
0709 13:14:40.353303 (+461512us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:40.353534 (+   231us) write_op.cc:454] Released schema lock
0709 13:14:40.353633 (+    99us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:40.450940 (+ 97307us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:40.458466 (+  7526us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5973,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":155159,"apply.run_cpu_time_us":69774,"apply.run_wall_time_us":1899475,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104447,"num_ops":24,"prepare.queue_time_us":187084,"prepare.run_cpu_time_us":23663,"prepare.run_wall_time_us":324540,"raft.queue_time_us":204584,"raft.run_cpu_time_us":14484,"raft.run_wall_time_us":121037,"replication_time_us":608367,"thread_start_us":871668,"threads_started":5,"wal-append.queue_time_us":905704,"wal-append.run_cpu_time_us":39999,"wal-append.run_wall_time_us":2564335}]]}
I20260709 13:14:45.294881 12820 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:45.153453 12610 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:45.297327 12866 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.297824 12866 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:45.298738  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:45.273695 12942 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.299945 12942 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:45.299964 12866 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:45.262768  8730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 816) took 1541 ms. Trace:
I20260709 13:14:45.302129  8730 rpcz_store.cc:276] 0709 13:14:43.721331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.827403 (+106072us) service_pool.cc:224] Handling call
0709 13:14:43.827697 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:43.866619 (+ 38922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:43.866660 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:44.799239 (+932579us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b2c000c0e58. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db990fa11 00007f6db991f372 00007f6dbb051a01 00007f6dbb01643a 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2
0709 13:14:44.799631 (+   392us) log.cc:844] Serialized 26 byte log entry
0709 13:14:45.244370 (+444739us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:45.244406 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.244495 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:45.244534 (+    39us) raft_consensus.cc:1685] finished
0709 13:14:45.244545 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:45.261440 (+ 16895us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38469,"raft.queue_time_us":54081,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":15725,"spinlock_wait_cycles":9423616,"thread_start_us":444395,"threads_started":1,"wal-append.queue_time_us":411980}
I20260709 13:14:45.278075 12772 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:14:45.264590  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.274052  8132 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:45.305917  7958 catalog_manager.cc:5697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.253886 12966 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:45.307492  8132 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:14:45.253304  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:45.283260 13051 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.065s	user 0.006s	sys 0.005s
I20260709 13:14:45.274910  8212 raft_consensus.cc:2393] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 13: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:45.274561  8204 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:45.235473  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:45.308976  8721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 821) took 1034 ms. Trace:
I20260709 13:14:45.266983 13006 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:14:45.307576  7958 catalog_manager.cc:5697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.311328  8054 raft_consensus.cc:2368] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14.
I20260709 13:14:45.280689  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:45.256273  8130 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:14:45.283888  8175 leader_election.cc:400] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11.
I20260709 13:14:45.254133 12629 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:45.261322 13008 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:14:45.291738  8203 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:45.296295 12920 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.341754 12920 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:45.344434  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:45.315398  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.309408  8721 rpcz_store.cc:276] 0709 13:14:44.274131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.274551 (+   420us) service_pool.cc:224] Handling call
0709 13:14:44.274939 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:44.310556 (+ 35617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:44.310588 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:44.310998 (+   410us) log.cc:844] Serialized 26 byte log entry
0709 13:14:44.496226 (+185228us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:44.496270 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:44.496331 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:45.308787 (+812456us) raft_consensus.cc:1685] finished
0709 13:14:45.308807 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:45.308925 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35174,"raft.queue_time_us":37175,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"thread_start_us":184934,"threads_started":1,"wal-append.queue_time_us":326069}
I20260709 13:14:45.346768  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:45.348112  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:45.348719  8208 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:45.307243 12966 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:45.309154  8136 raft_consensus.cc:2410] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:45.352327  8026 leader_election.cc:400] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10.
I20260709 13:14:45.279606  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:44.988644 12914 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:45.279927  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:45.229498  8129 rpcz_store.cc:276] 0709 13:14:43.663952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.665898 (+  1946us) service_pool.cc:224] Handling call
0709 13:14:43.666135 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:44.060027 (+393892us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:44.060060 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:44.060104 (+    44us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:14:44.060142 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.228867 (+1168725us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54001ab154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:45.228910 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:45.229026 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":904274304}
I20260709 13:14:45.314234  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:45.279917 13049 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.069s	user 0.003s	sys 0.008s
I20260709 13:14:45.302028 12942 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:45.297633 12820 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:45.264892  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:45.315004  8212 raft_consensus.cc:3055] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:14:45.340360  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 56701.9us
I20260709 13:14:45.371985  8212 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:45.350804  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:45.266009  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.373227  8063 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:45.352674  8026 leader_election.cc:403] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:14:45.280753 12572 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.040s	sys 0.046s
I20260709 13:14:45.346763  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.330503  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:45.345927  8058 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:45.363392  8208 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:45.375773  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.362985  7958 catalog_manager.cc:5697] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 10 to 13, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 13 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.368268  8204 raft_consensus.cc:3055] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:45.377147  8204 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:45.369828  8104 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:45.344022 12920 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:45.379272  8057 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 13:14:45.358794 13034 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.010s	sys 0.004s
I20260709 13:14:45.376533  8055 raft_consensus.cc:1813] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Handling vote request from an unknown peer 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:45.310473 12610 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:45.278210 12945 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:45.394411  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.377210  8212 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:45.385303  8208 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:14:45.395545  8212 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:45.379978  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:45.364018 12713 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.399727 12713 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:45.400180  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:45.401057  8063 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:45.401816 12713 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:45.402484 12713 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:45.402815 12713 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.403124 12713 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:45.403946  8212 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:14:45.404335  8212 pending_rounds.cc:85] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 7
I20260709 13:14:45.404632  8212 pending_rounds.cc:107] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:14:45.404888  8212 raft_consensus.cc:2889] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:45.390218  8055 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:45.392169  8175 leader_election.cc:400] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10.
I20260709 13:14:45.407374  8175 leader_election.cc:403] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:14:45.409310  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:45.410264 12982 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 9. Current term is 10.
I20260709 13:14:45.411084 12982 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:45.411966  8175 leader_election.cc:400] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:14:45.413914  8203 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:45.418501  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:45.421725  8204 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:45.422964  8204 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:45.358037 12772 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:14:45.427454  8063 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:45.427491 12920 consensus_queue.cc:1059] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:14:45.385309 12919 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10.
I20260709 13:14:45.428598 13048 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:45.430553 13020 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:45.339538 12629 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:45.373688 12159 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:14:45.363955 12867 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:14:45.388470 12820 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:45.437039  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:45.436513  8175 leader_election.cc:400] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11.
I20260709 13:14:45.428416  7958 catalog_manager.cc:5697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 6 to 15. New cstate: current_term: 15 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.367498 12914 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:45.469542  8204 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 13:14:45.389231 13034 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.011s	sys 0.004s
I20260709 13:14:45.442507 12772 consensus_queue.cc:1243] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 12.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:45.465693  8175 leader_election.cc:403] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:45.432191 12966 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:45.472707  7958 catalog_manager.cc:5697] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 8 to 10, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 10 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.477098  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 818) took 1315 ms. Trace:
I20260709 13:14:45.478675  8727 rpcz_store.cc:276] 0709 13:14:44.161540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.161870 (+   330us) service_pool.cc:224] Handling call
0709 13:14:44.186588 (+ 24718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.282394 (+1095806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.476820 (+194426us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400505fd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:45.477030 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2518109952}
I20260709 13:14:45.458119 12610 raft_consensus.cc:3055] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:45.482941  7958 catalog_manager.cc:5697] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 6 to 8, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 8 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:14:45.447770 12820 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:45.493239  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 118095us
W20260709 13:14:45.498164  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 118276us
I20260709 13:14:45.483558 12966 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:45.489127  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 207886us
I20260709 13:14:45.477000 12992 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:45.502602  8026 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:45.489024 12888 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.966s	user 0.018s	sys 0.034s
W20260709 13:14:45.511364  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 85381.7us
I20260709 13:14:45.497798 12820 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:45.508496  8026 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:45.487241 12610 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:45.483028 12713 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:45.529824  8175 leader_election.cc:400] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12.
W20260709 13:14:45.518616 12888 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.995s	user 0.019s	sys 0.034s
I20260709 13:14:45.544225  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:45.565562  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:45.571431  8135 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:45.574719  8129 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:14:45.576053  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:45.577306  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:45.577979  8063 raft_consensus.cc:2368] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12.
I20260709 13:14:45.578718  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:45.582784  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:45.583700  8058 raft_consensus.cc:2368] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:14:45.584519  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.585589  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.586650  8138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:14:45.587387  8138 raft_consensus.cc:606] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:45.518440 12546 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:45.606138  8133 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
W20260709 13:14:45.612938  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:45.614585  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.619364  8057 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 10.
W20260709 13:14:45.644825  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 48870.1us
I20260709 13:14:45.645710  8101 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:45.646243  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.647936  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.648164  8138 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 13:14:45.648375  8063 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: []
W20260709 13:14:45.651336  8175 connection.cc:256] RPC call timeout handler was delayed by -3.7360075809999671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18330000000000002s for a response
W20260709 13:14:45.651825  8175 connection.cc:256] RPC call timeout handler was delayed by -3.7126075809999293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1859s for a response
W20260709 13:14:45.652227  8175 connection.cc:256] RPC call timeout handler was delayed by -3.5938075809999646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19910000000000003s for a response
W20260709 13:14:45.652631  8175 connection.cc:256] RPC call timeout handler was delayed by -3.4057075809998878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22000000000000003s for a response
I20260709 13:14:45.653256  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:45.623766  8129 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 14. Ops: []
I20260709 13:14:45.506436 12772 consensus_queue.cc:1059] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:14:45.530436 12897 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.638504  8137 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:14:45.546317 12728 consensus_queue.cc:1059] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.096s
I20260709 13:14:45.659245  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:45.620054 12886 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:45.664919  8101 leader_election.cc:400] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14.
I20260709 13:14:45.665673  8101 leader_election.cc:403] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:45.666290 12997 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:14:45.658427  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:45.526366 13048 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.119s
I20260709 13:14:45.540074 12998 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:45.660305  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:45.514128 12992 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:45.669777  8101 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:45.642359 12629 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:45.673585  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:45.674183  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 90032.2us
I20260709 13:14:45.785076  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:45.704645  8057 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:45.546833 13007 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.129s
I20260709 13:14:45.799038  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:45.556805 12253 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.148s
I20260709 13:14:45.800388  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.805469  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.808879  8057 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: [11.5-11.5]
W20260709 13:14:45.812072  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 142753us
I20260709 13:14:45.811200  8057 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:14:45.813607  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:45.705085  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:45.815295  8063 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:45.708225  8135 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:45.663913 12886 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.816143  8057 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: [10.10-12.11]
I20260709 13:14:45.846853  8057 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15. Ops: [13.7-14.8]
I20260709 13:14:45.708596 12945 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:45.849911  8057 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 13:14:45.871567  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:45.878690  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:45.790961 12629 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:45.673303 12922 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.192s
I20260709 13:14:45.888556  8057 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:14:45.664856 12159 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:45.849164 12886 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:45.894198 12882 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:45.894939 12945 consensus_queue.cc:1059] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:45.895653 13020 raft_consensus.cc:3055] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:45.896152 13020 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:45.897090 13020 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:45.900396 13041 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 1.379s	user 0.006s	sys 0.006s
I20260709 13:14:45.900663 13075 raft_consensus.cc:3055] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:45.901122 13075 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:45.901859 13075 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:45.902424 12914 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:45.902873 13070 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.303s	user 0.001s	sys 0.000s
I20260709 13:14:45.903139 12546 raft_consensus.cc:3055] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:45.903563 12546 raft_consensus.cc:740] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:14:45.903924 13062 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.397s	user 0.001s	sys 0.000s
I20260709 13:14:45.904335 12546 consensus_queue.cc:260] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:45.907152  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.907305 13001 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:45.817777 13003 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:45.908486 13003 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:45.870587 12713 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:45.909864 12629 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:45.912246 12629 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:45.912961 13059 log.cc:927] Time spent T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.371s	user 0.001s	sys 0.000s
I20260709 13:14:45.914026  8056 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:14:45.914616  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 752) took 1026 ms. Trace:
I20260709 13:14:45.914923  8056 rpcz_store.cc:276] 0709 13:14:44.887939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.888320 (+   381us) service_pool.cc:224] Handling call
0709 13:14:45.913521 (+1025201us) spinlock_profiling.cc:243] Waited 4.67 ms on lock 0x7b5800000c98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:14:45.913718 (+   197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.914389 (+   671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.914562 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10750336}
I20260709 13:14:45.919891 13083 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:45.817265 13081 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.120s
I20260709 13:14:45.922992 12253 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:45.923544 12253 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:45.925181  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2042) took 1530 ms. Trace:
I20260709 13:14:45.925572  8131 rpcz_store.cc:276] 0709 13:14:44.395932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.396109 (+   177us) service_pool.cc:224] Handling call
0709 13:14:44.396399 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.241739 (+845340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.924965 (+683226us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b54001ebc54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:45.925125 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1942651136}
I20260709 13:14:45.926905  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:45.931233  8208 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:14:45.911972  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 746) took 1182 ms. Trace:
W20260709 13:14:45.907617  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.934068  8061 rpcz_store.cc:276] 0709 13:14:44.730088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.730218 (+   130us) service_pool.cc:224] Handling call
0709 13:14:44.730533 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:44.818697 (+ 88164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.911741 (+1093044us) spinlock_profiling.cc:243] Waited 87.4 ms on lock 0x7b540009bad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:45.911903 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201043712}
I20260709 13:14:45.934649  8137 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
W20260709 13:14:45.936997  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 283813us
I20260709 13:14:45.938086  8203 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:45.819834  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:45.671293 12610 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:45.940847  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:45.947710  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:45.912052 13067 log.cc:927] Time spent T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
I20260709 13:14:45.667606 12897 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:45.959719  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 146193us
W20260709 13:14:45.944959  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 153710us
I20260709 13:14:45.961714  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:45.945618  8138 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:45.947927  8203 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:45.948854  8208 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:45.942147  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:45.804293 12772 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.132s
I20260709 13:14:45.980224  8101 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:45.981596  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:45.982411  8208 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:45.983400  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:45.983785  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:45.984694  8101 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:45.939728 13074 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:45.950321 13020 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:45.936363 12159 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:45.993801  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:45.789690 13023 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:45.933951 12253 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:45.997740  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:45.944377 13083 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:45.979521 12962 consensus_meta.cc:287] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.728s	user 0.009s	sys 0.005s
I20260709 13:14:46.002250  8203 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:45.994594  8101 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:46.027565  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:46.050694  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:46.058696  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 64181.8us
I20260709 13:14:45.980206 12987 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:46.051842  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:46.060904  8721 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:46.062510 13074 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:46.063163 13074 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:46.064145 13074 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.052996  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:46.052325  8205 raft_consensus.cc:2410] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:14:46.067430  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:45.998066 12610 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:46.068771  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:46.065379  8054 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.002s	sys 0.015s
I20260709 13:14:45.997949 13028 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:46.073523  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:46.101189  8054 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.003s	sys 0.015s
I20260709 13:14:46.111622  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:46.027573 12159 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:45.992563 13084 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:46.123803  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:46.126827  8058 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:46.129258  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:46.129630  8208 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
W20260709 13:14:46.130929  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.055053 12828 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:46.062244 12982 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:46.138497  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:45.993662 12713 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.120s
I20260709 13:14:46.064167 13081 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.165614  8056 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:46.071493 12853 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:46.174326  8026 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.024s	sys 0.025s
W20260709 13:14:46.174868  8026 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.024s	sys 0.025s
W20260709 13:14:46.175179  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 584125us
I20260709 13:14:46.176327  8029 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:46.123965 13078 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.242s	user 0.003s	sys 0.000s
I20260709 13:14:46.178211  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:46.176528  8026 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:46.113937 13028 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.185225  8729 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:46.188756  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:46.189808  8211 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
W20260709 13:14:46.222240  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 280113us
W20260709 13:14:46.223508  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 47485.4us
I20260709 13:14:46.224653  8029 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:46.225631  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59856 (request call id 1926) took 1551 ms. Trace:
I20260709 13:14:46.113476 13080 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:46.234804  8211 rpcz_store.cc:276] 0709 13:14:44.639205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.641668 (+  2463us) service_pool.cc:224] Handling call
0709 13:14:46.188580 (+1546912us) spinlock_profiling.cc:243] Waited 4.26 ms on lock 0x7b5800168398. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffdc0a0 00007f6dbffdb984 00007f6db95566d4 00007f6db9556662
0709 13:14:46.190583 (+  2003us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9804544}
I20260709 13:14:46.245350  8730 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:14:46.247852  8056 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:46.248345  8721 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:14:46.249773  8029 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:46.173648 13082 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:14:46.250645  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 74213.7us
I20260709 13:14:46.068523 13083 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:46.132480 13084 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:46.259073  8026 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:46.198864 13028 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:46.265220  8026 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:46.266466  8026 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:46.227653 12913 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:14:46.256801  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 203734us
I20260709 13:14:46.173815 12920 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:14:46.191063 12982 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.268680  8026 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:46.271351  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:46.300683  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:46.271606  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:46.272540  8138 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:46.273963  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:46.276937  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:46.245213 12912 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:46.181466 12772 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:14:46.286262  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:46.179862 12853 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.258112 13094 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:46.241146 13048 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:14:46.261735 12915 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:46.330528  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:46.304615 13078 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:14:46.266856 12914 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:46.183432 13081 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:46.329772 12966 consensus_meta.cc:287] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.825s	user 0.012s	sys 0.000s
I20260709 13:14:46.278157 12867 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:46.259610 13095 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:46.280507 13101 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.281952 13082 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:46.279129 12913 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:46.347115  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 107310us
I20260709 13:14:46.268097 13084 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:46.351389  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 52098.7us
I20260709 13:14:46.351388  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:46.355636  8137 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:46.350585  8721 raft_consensus.cc:2410] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:14:46.333683 12912 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:46.350930  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:46.374709  8136 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 12: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:14:46.376444  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.374315  8026 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:46.358902 12966 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11.
I20260709 13:14:46.360486 13095 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.395457  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:46.402664  8026 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:46.335594 13007 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:14:46.347150 12882 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:14:46.398614  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2971) took 1041 ms. Trace:
I20260709 13:14:46.415760  8062 rpcz_store.cc:276] 0709 13:14:45.357184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.357355 (+   171us) service_pool.cc:224] Handling call
0709 13:14:45.357650 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.485364 (+127714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:45.485403 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:45.485455 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:45.485508 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.398301 (+912793us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54001e87d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:46.398370 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:46.398541 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":293196032}
I20260709 13:14:46.341663 12996 consensus_queue.cc:1059] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:14:46.373986 12913 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:46.330003 12982 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.364508 12915 raft_consensus.cc:697] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:46.412051  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:46.396317  8137 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:14:46.418715  8212 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:46.407382 13095 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:46.345453 13094 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.406045 13082 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:46.399919 13023 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:46.451651  8212 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
W20260709 13:14:46.453401  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50853.9us
I20260709 13:14:46.455868  8026 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:46.420622 12897 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:46.426870 12966 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:46.482110 13023 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:46.482540 13099 consensus_queue.cc:1059] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:14:46.485001  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.487474  8026 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:46.375631 13101 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:46.491968 13114 consensus_queue.cc:1059] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:14:46.493047 13115 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:46.494491 13115 raft_consensus.cc:3055] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:46.494933 13115 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:46.500202 12982 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:46.497756 13115 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:46.392792 12912 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:46.501677 13115 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:46.501960 13094 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:46.509861  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:46.465512 13082 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:46.419623 13109 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
W20260709 13:14:46.540367  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.528702 12546 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:46.503343 13023 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:46.502722 13055 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:14:46.530872 12728 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.015s	sys 0.025s
W20260709 13:14:46.559612  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 141496us
I20260709 13:14:46.488605 12966 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:46.570927  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 291404us
I20260709 13:14:46.579142  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:46.554316 13055 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.585222  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:46.586297  8721 raft_consensus.cc:2376] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10. Re-sending same reply.
W20260709 13:14:46.569746 12728 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.016s	sys 0.027s
I20260709 13:14:46.587821  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:46.590694  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:46.592835  8721 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:46.593469  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:46.599803  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:46.602277  8101 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:46.572309 12853 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:46.595141 12912 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.610970 12912 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.613029 13122 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:46.586226 13114 consensus_queue.cc:1243] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 15.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:14:46.577688 12915 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:46.615734 13106 log.cc:927] Time spent T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Append to log took a long time: real 0.177s	user 0.000s	sys 0.004s
I20260709 13:14:46.601495 13094 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:14:46.595235 12982 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:46.618119 12982 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:46.618970 12982 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:46.619443 12982 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:46.620067 12982 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:46.626602  8129 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12. Ops: []
W20260709 13:14:46.632355 12997 consensus_meta.cc:287] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.966s	user 0.010s	sys 0.003s
I20260709 13:14:46.631994 13095 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:46.634786  8102 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:46.635512 13081 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:46.633912  8101 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:46.637138 12820 consensus_meta.cc:287] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 1.099s	user 0.015s	sys 0.000s
I20260709 13:14:46.591328 13083 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:46.638731 12999 raft_consensus.cc:3055] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:46.639788 12966 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:46.640560 13093 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.258s	user 0.001s	sys 0.001s
I20260709 13:14:46.635856 12853 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:46.644276  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 44521.9us
I20260709 13:14:46.597568 13055 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:46.649585  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:46.650323 13026 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:46.603518 13112 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.649734 13121 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:14:46.651443 13110 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:14:46.651687 13121 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.650643 12853 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.651934 13122 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:46.652099 13121 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:46.652478 13122 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:46.649533 13034 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.653067 13034 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.653450 13122 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:46.656335  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.656931 12713 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:46.610653 13111 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:46.632797 12997 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14.
I20260709 13:14:46.660097  8102 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:46.647301 12999 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:46.674719  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:46.677430  8129 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:46.677582  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:46.682037  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:46.683382  8131 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:46.687110  8132 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:14:46.688143  8135 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:14:46.695974  8102 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:46.700840  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:46.701476  8727 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:46.675203 12716 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:46.700105 13094 consensus_queue.cc:1243] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:14:46.705320  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:46.648629 13028 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:46.662004 12820 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:46.688715 13101 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:46.691207 12997 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:46.719602  8129 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:46.694061 13095 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:46.700371 12897 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:46.651479 13112 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:46.737637 12716 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:46.769440 12716 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.749495 12820 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:46.774076 12853 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:46.776825 13034 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:46.737803 13111 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:46.741433 12828 consensus_meta.cc:287] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.573s	user 0.013s	sys 0.000s
I20260709 13:14:46.742442 13107 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.747830 13028 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:46.752564 12610 consensus_meta.cc:287] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.624s	user 0.009s	sys 0.002s
W20260709 13:14:46.755905 12886 consensus_meta.cc:287] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.863s	user 0.012s	sys 0.000s
I20260709 13:14:46.759239  8210 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15. Ops: [10.5-14.6]
I20260709 13:14:46.796960  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1934) took 2134 ms. Trace:
I20260709 13:14:46.797433  8210 rpcz_store.cc:276] 0709 13:14:44.662562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.662666 (+   104us) service_pool.cc:224] Handling call
0709 13:14:46.758646 (+2095980us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b5800168398. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:14:46.758975 (+   329us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:46.796482 (+ 37507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.796885 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6614912}
I20260709 13:14:46.760241 13000 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:46.762800 12159 consensus_meta.cc:287] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.584s	user 0.006s	sys 0.006s
I20260709 13:14:46.762444 12913 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:46.722330 12999 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:46.701010 13084 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.732589 12997 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.764617  8123 tablet_service.cc:1558] Processing DeleteTablet for tablet a478a4313e6149378cae4b6f833e3c2a with delete_type TABLET_DATA_TOMBSTONED (TS 8a968cada1e4446ebb2a5e167176e649 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:36736
I20260709 13:14:46.765703 13112 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:46.767606  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2969) took 1451 ms. Trace:
W20260709 13:14:46.769956 13020 consensus_meta.cc:287] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.742s	user 0.013s	sys 0.000s
I20260709 13:14:46.809159 12886 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.835806 13003 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:46.836283  8730 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:46.738130  8727 raft_consensus.cc:2410] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:14:46.770882 12912 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:46.801162  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:46.801924  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:46.781133 12982 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:46.837412  8054 rpcz_store.cc:276] 0709 13:14:45.315867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.316045 (+   178us) service_pool.cc:224] Handling call
0709 13:14:45.316325 (+   280us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:45.345170 (+ 28845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:45.345198 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:45.355564 (+ 10366us) log.cc:844] Serialized 52 byte log entry
0709 13:14:46.101786 (+746222us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:46.101850 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.102667 (+   817us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:46.767295 (+664628us) raft_consensus.cc:1685] finished
0709 13:14:46.767327 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:46.767530 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28401,"raft.queue_time_us":28498,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":11771,"thread_start_us":726482,"threads_started":1,"wal-append.queue_time_us":865338}
I20260709 13:14:46.843133  8102 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:46.780781 13121 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:46.802577 12828 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:46.840026 12965 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:46.847513 13020 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.848184 13020 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.839053 13125 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:46.794803 13111 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.859860 13084 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:46.823738 12159 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.861927 12159 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:46.805519 13107 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.846732 12965 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.860318 13111 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:46.869410 13107 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:46.875761 12886 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:46.874984  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:46.864528 13125 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:46.879195 13125 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:46.868566 12965 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:46.821169 13000 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:46.844799 13112 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:46.867136  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:46.897341 13121 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:46.903808 13092 raft_consensus.cc:3055] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:46.904862 13092 raft_consensus.cc:740] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:46.889297 12913 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:46.907997  8133 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 2048) took 2447 ms (client timeout 1854 ms). Trace:
I20260709 13:14:46.850188 13055 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:46.912086 13055 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:46.896739 13000 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:46.912719  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 743) took 3217 ms. Trace:
I20260709 13:14:46.915285 12997 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:46.913329  8060 rpcz_store.cc:276] 0709 13:14:43.695149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.817771 (+122622us) service_pool.cc:224] Handling call
0709 13:14:43.818076 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.287529 (+1469453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.912118 (+1624589us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540001bdd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:46.912600 (+   482us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3378305920}
I20260709 13:14:46.869338 13020 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:46.909008  8133 rpcz_store.cc:269] 0709 13:14:44.460907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.490229 (+ 29322us) service_pool.cc:224] Handling call
0709 13:14:46.907057 (+2416828us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540035cad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:46.907230 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2429635328}
I20260709 13:14:46.948763  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:46.957649  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.926188 13000 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:14:47.001573  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.008337  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:47.014761  8728 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:47.016158 13138 tablet_replica.cc:333] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:14:46.928828 12913 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 13:14:47.019996  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:47.016777  8027 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:47.025125  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:47.026880  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:47.018462 13136 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.177s	user 0.001s	sys 0.001s
I20260709 13:14:47.000207 13092 consensus_queue.cc:260] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:47.045336 13098 log.cc:927] Time spent T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.676s	user 0.001s	sys 0.003s
I20260709 13:14:47.047016  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1943) took 1116 ms. Trace:
I20260709 13:14:47.049101  8207 rpcz_store.cc:276] 0709 13:14:45.930299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.932534 (+  2235us) service_pool.cc:224] Handling call
0709 13:14:45.932814 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:45.933118 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:45.933147 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:45.933538 (+   391us) log.cc:844] Serialized 26 byte log entry
0709 13:14:46.167707 (+234169us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:46.167969 (+   262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.168332 (+   363us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:47.044662 (+876330us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:47.046624 (+  1962us) raft_consensus.cc:1685] finished
0709 13:14:47.046654 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:47.046946 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128898,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"spinlock_wait_cycles":4804864,"thread_start_us":233080,"threads_started":1,"wal-append.queue_time_us":370669}
I20260709 13:14:47.052208 12886 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:47.011351 13000 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.027856 13020 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:47.056272 13104 raft_consensus.cc:3055] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:14:47.068764 13135 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:14:47.069751 13104 raft_consensus.cc:740] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:47.074879 12982 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:47.101854  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 744) took 3406 ms. Trace:
I20260709 13:14:47.109670  8026 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:47.111997  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:47.104710  8059 rpcz_store.cc:276] 0709 13:14:43.695489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:43.818751 (+123262us) service_pool.cc:224] Handling call
0709 13:14:45.235968 (+1417217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.236253 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:45.381398 (+145145us) log.cc:844] Serialized 14 byte log entry
0709 13:14:45.484769 (+103371us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:45.484839 (+    70us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:45.484918 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.485002 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:47.101613 (+1616611us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b540016a650. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:47.101776 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":62021,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":2811,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":6937,"spinlock_wait_cycles":3258688896,"thread_start_us":247221,"threads_started":2,"wal-append.queue_time_us":423060,"wal-append.run_cpu_time_us":1372,"wal-append.run_wall_time_us":1076046}
I20260709 13:14:47.106621 13104 consensus_queue.cc:260] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.117411  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:47.118821  8729 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: [13.10-15.11]
I20260709 13:14:47.122069 13080 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:14:47.126755  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 83890.3us
I20260709 13:14:47.116076 12945 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.126504  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:47.123270 13111 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.127120 13104 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:47.130102  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:47.129458  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:47.132333  8132 raft_consensus.cc:2393] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:14:47.128393 13137 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:14:47.133723  8026 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:47.131779 13080 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:47.141151  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:47.131213 12945 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.147616  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:47.137607  8026 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:47.149138  8727 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:47.150381  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:47.140134 13020 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:47.133507 13038 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:14:47.158247  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:47.159024  8138 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:47.169507  8026 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:47.133841 13111 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.174041  8026 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:47.178925  8132 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:47.180173  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:47.181113  8132 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:47.157807 13080 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:47.172693 12546 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:47.185320 12992 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:47.186827  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:47.168537 13000 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:47.166623 13115 consensus_queue.cc:1059] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:14:47.187163 12913 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:47.174317 13020 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
W20260709 13:14:47.201892 13097 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.615s	user 0.000s	sys 0.003s
I20260709 13:14:47.200682 12913 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:47.203063 12913 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.204468 13026 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.205260  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1942) took 1275 ms. Trace:
I20260709 13:14:47.205668  8204 rpcz_store.cc:276] 0709 13:14:45.929994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.930978 (+   984us) service_pool.cc:224] Handling call
0709 13:14:45.931279 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:45.931628 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:45.931657 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:45.932090 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 13:14:46.178984 (+246894us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:46.179038 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:46.179125 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:47.171658 (+992533us) spinlock_profiling.cc:243] Waited 4.09 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:47.204894 (+ 33236us) raft_consensus.cc:1685] finished
0709 13:14:47.204922 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:47.205188 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95737,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":4162,"spinlock_wait_cycles":9411328,"thread_start_us":246505,"threads_started":1,"wal-append.queue_time_us":594737}
I20260709 13:14:47.205013 13026 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.225980  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:47.227106 12992 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:47.228794 13026 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:47.231492 13111 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:47.233932  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:47.236248 13026 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:47.223738 13027 raft_consensus.cc:3055] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:47.235802 12992 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:14:47.241339 13026 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:47.243204 13074 raft_consensus.cc:993] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:14:47.247527 13074 raft_consensus.cc:1081] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:14:47.244277  8721 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:47.249884 13026 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.5, Last appended by leader: 5, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.240728 13027 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:14:47.254966  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.255707 13126 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:47.259604  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:47.259896 13126 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.260491  8129 raft_consensus.cc:2410] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:47.261245  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:47.261759  8129 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:47.261946 13126 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:47.273514  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:47.274567  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:47.275614  8062 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:47.276592  8059 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14. Ops: []
W20260709 13:14:47.276500  8026 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.642s (SENT) [suppressed 11 similar messages]
W20260709 13:14:47.277585  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.278179  8055 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:14:47.281374  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:47.281690  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:47.283143  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.282087  8055 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:47.287315  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:47.288357  8056 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:47.288393  8059 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:47.276798 13001 consensus_queue.cc:1059] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:14:47.291735 13124 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:47.292088 13001 raft_consensus.cc:3055] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:47.280417 12886 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.286006 13088 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.303737  8059 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:14:47.283274 12966 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:47.306912  8056 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:14:47.308593  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:47.295526 13109 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.297580 13001 raft_consensus.cc:740] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:47.296290 13124 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.311982 13001 consensus_queue.cc:260] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.313792 13001 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:47.314709 13109 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.317937 13109 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:47.303246 13048 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:47.303949 12886 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.319347 13048 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.321189 12886 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:47.297844 13125 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:47.322351 13048 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:47.318532 13124 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:47.295717 13094 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:47.259492 13027 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.330042  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:47.287451 13074 consensus_queue.cc:1059] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:14:47.332302  8721 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:14:47.341841  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 65377.4us
I20260709 13:14:47.326843 13119 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:47.346140  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.310506 13088 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:47.351486  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.351424  8056 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:47.324537 12966 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.331271 13083 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:14:47.330997 13094 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.333415 12932 consensus_queue.cc:1059] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:14:47.348929 13119 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:47.359277  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 72049.8us
I20260709 13:14:47.360740  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1767) took 1796 ms. Trace:
I20260709 13:14:47.361384  8175 leader_election.cc:400] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:14:47.361197  8058 rpcz_store.cc:276] 0709 13:14:45.564286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.585345 (+ 21059us) service_pool.cc:224] Handling call
0709 13:14:47.360534 (+1775189us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b54001e8cd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:47.360685 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1120748928}
I20260709 13:14:47.361599  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:47.363967  8207 raft_consensus.cc:3055] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:47.364306  8207 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:47.365228  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:47.369122 13151 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:47.368116 13048 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:47.369973 13151 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.370692 13119 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:47.370983 13048 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.368034  8212 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 13:14:47.373714  8212 pending_rounds.cc:85] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 6
I20260709 13:14:47.374073  8212 pending_rounds.cc:107] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:14:47.374468  8212 raft_consensus.cc:2889] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:47.375110  8212 pending_rounds.cc:107] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:14:47.375706  8212 raft_consensus.cc:2889] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:47.365013  8207 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.377172  8207 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:47.382669  8208 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:14:47.384641  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:47.385699  8208 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
W20260709 13:14:47.390555  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:47.421478  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.425405  8209 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:14:47.556675  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:47.596578  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:47.616878  8061 raft_consensus.cc:2393] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 12: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:47.548766  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:47.425263  8729 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: []
W20260709 13:14:47.464762  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 103474us
I20260709 13:14:47.617515  8058 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:47.619351  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:47.613801  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:47.546389  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:47.626089  8056 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:47.551661  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:47.629251  8054 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:47.632890  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:47.637585  8132 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:47.637739  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:47.638155  8131 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 2078) took 1364 ms (client timeout 1642 ms). Trace:
I20260709 13:14:47.639276  8055 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:47.641850  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:47.664925  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:47.697852  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:47.703576  8131 rpcz_store.cc:269] 0709 13:14:46.273489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.273647 (+   158us) service_pool.cc:224] Handling call
0709 13:14:47.637085 (+1363438us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b5400381b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:47.638085 (+  1000us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":820456320}
I20260709 13:14:47.665387  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:47.643882  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:47.706126  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:47.706173  8061 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:47.706887  8058 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:47.707319  8063 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:47.697422  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:47.459136 13153 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.395233 12932 consensus_queue.cc:1243] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:14:47.715958  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:47.717571  8211 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:47.427126 13146 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:14:47.589766 12998 raft_consensus.cc:3055] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:14:47.691193 13113 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.194s	user 0.007s	sys 0.017s
W20260709 13:14:47.715898  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.716536  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 2981) took 1647 ms. Trace:
I20260709 13:14:47.649509 13126 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:47.649188  8175 leader_election.cc:400] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12.
I20260709 13:14:47.720930  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:47.429126 13088 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:47.432416 13094 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:47.639146 13072 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.638633 13161 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.060s
W20260709 13:14:47.727885 13113 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.229s	user 0.007s	sys 0.021s
I20260709 13:14:47.722066 12966 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:47.596246 13151 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:47.723480  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:47.741802  8212 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 14: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:47.430502 13048 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:47.615851 13000 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:47.743942 13000 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.720944 13089 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.015s
W20260709 13:14:47.745793  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.730674  8206 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:47.732822  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:47.726405  8057 rpcz_store.cc:276] 0709 13:14:46.069713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.071816 (+  2103us) service_pool.cc:224] Handling call
0709 13:14:47.231924 (+1160108us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b5800000c98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:14:47.232130 (+   206us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:47.232517 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:47.232547 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:47.234800 (+  2253us) log.cc:844] Serialized 52 byte log entry
0709 13:14:47.304364 (+ 69564us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:47.601113 (+296749us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b1800448540. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:14:47.636703 (+ 35590us) log.cc:844] Serialized 14 byte log entry
0709 13:14:47.716173 (+ 79470us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:47.716283 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:47.716325 (+    42us) raft_consensus.cc:1685] finished
0709 13:14:47.716339 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:47.716476 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79180,"prepare.queue_time_us":22444,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":91910,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":901,"spinlock_wait_cycles":91586816,"thread_start_us":149853,"threads_started":2,"wal-append.queue_time_us":104471}
I20260709 13:14:47.643236 12913 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:47.736230  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 95867.5us
I20260709 13:14:47.739441  8056 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:47.740286  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:47.725032 12998 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:47.748334  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:47.758657  8203 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:47.766417  8063 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:47.719786 13165 consensus_queue.cc:1059] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:14:47.773480  8058 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:47.723078 13153 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:47.777789  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.746703 13000 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:47.780079  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:47.784245  8175 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:47.786482  8175 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:47.737614 13072 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.792083 13153 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:47.792959 13072 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:47.803293  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.803592  8208 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59868
I20260709 13:14:47.804239  8208 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:47.804724  8208 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:47.808041  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:47.802919 13151 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:47.802671 12982 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:47.764663 12913 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.820672 13048 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:14:47.811172 12966 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:47.811102 13153 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:47.833124 13048 raft_consensus.cc:3055] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:14:47.826647 13152 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 13:14:47.821272 12998 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.840540 13079 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:47.841579 13079 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:47.843140 13001 consensus_meta.cc:287] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.000s
I20260709 13:14:47.843592 13079 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.844504 13079 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:47.845180 12913 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:47.836354 13048 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:47.847404 12629 raft_consensus.cc:3055] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:47.847811 12629 raft_consensus.cc:740] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:47.848284 13048 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:47.849149 13048 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:47.848860 12629 consensus_queue.cc:260] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.846598 13055 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:47.850533 13055 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:47.851907 13055 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.853134 13055 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:47.865949 13142 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.683s	user 0.000s	sys 0.017s
I20260709 13:14:47.871845 13092 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:47.877748 13075 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:47.883121 12998 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:47.886266 13080 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.886807 13080 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.890595 13080 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:47.892570 13027 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:47.896483 13114 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:47.897125 13114 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:47.898010 13114 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.898988 13164 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:47.899369 13114 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:47.899402 13164 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:47.900513 13164 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:47.901383 13164 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:47.945681  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:47.950865  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.960243  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:47.961989  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:47.977957  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:47.983202  8137 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 761) took 1698 ms (client timeout 1835 ms). Trace:
W20260709 13:14:47.983680  8137 rpcz_store.cc:269] 0709 13:14:46.284933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.300361 (+ 15428us) service_pool.cc:224] Handling call
0709 13:14:47.982756 (+1682395us) spinlock_profiling.cc:243] Waited 54.2 ms on lock 0x7b540035d9d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:47.982970 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124628224}
I20260709 13:14:47.987429  8130 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:47.989413  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2088) took 1307 ms. Trace:
I20260709 13:14:47.961849  8204 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:14:47.964641  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:47.965485  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 147186us
I20260709 13:14:47.989809  8730 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:47.963821  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:47.987270  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:47.968982  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:48.000881  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.031711  8137 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14. Ops: []
W20260709 13:14:48.029841  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 51650.9us
I20260709 13:14:47.994096  8203 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
W20260709 13:14:47.997771  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.989948  8130 rpcz_store.cc:276] 0709 13:14:46.681877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.682007 (+   130us) service_pool.cc:224] Handling call
0709 13:14:47.986865 (+1304858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:47.987952 (+  1087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:47.989068 (+  1116us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540035d9d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:47.989283 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2999913216}
I20260709 13:14:47.994375  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 762) took 1709 ms. Trace:
I20260709 13:14:47.962015  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:48.046576  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.030558  8721 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:47.996670  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:48.016227  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:47.992980  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:48.044303  8138 rpcz_store.cc:276] 0709 13:14:46.285201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.309517 (+ 24316us) service_pool.cc:224] Handling call
0709 13:14:46.349944 (+ 40427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:47.159657 (+809713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:47.993467 (+833810us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b54003c91d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:47.994114 (+   647us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1860102912}
I20260709 13:14:48.032831 12945 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:48.024012 12629 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:48.029004 12915 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.070962  8209 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
W20260709 13:14:48.073305  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.074932  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:48.075950  8730 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:48.078462 13085 consensus_queue.cc:1059] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:14:48.079694 13114 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:48.079538 12915 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.082693  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:48.082530 13124 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:14:48.035297 12999 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:48.096057  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 470593us
I20260709 13:14:48.096182  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:48.098744  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 102427us
I20260709 13:14:48.098843  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:48.101591 13055 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.101783  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:48.102594  8727 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:48.103202 13055 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.104975 12159 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:48.106732  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:48.110846 13048 raft_consensus.cc:3055] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:48.111171 13099 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.111341 13048 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:48.112100 13099 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.117045 13074 raft_consensus.cc:3055] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:48.117666 13074 raft_consensus.cc:740] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:48.118754  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2091) took 1010 ms. Trace:
I20260709 13:14:48.092150 13085 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.091600 12915 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:48.120162 13085 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.124614  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.125550  8130 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:48.126062 13109 raft_consensus.cc:3055] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:48.126626 13109 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:48.127514  8137 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 13. Ops: []
I20260709 13:14:48.119088  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:48.127831 13109 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.129573 13109 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:48.130007  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.131117  8135 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:48.130719  8208 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.105981 13055 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:48.150463  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:48.138307  8027 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:48.165416  8060 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:48.150898  8724 consensus_queue.cc:237] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.193809 13099 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:48.194463  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.169764 12915 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:48.114106 13048 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.200080 13048 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:48.202589 13006 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.203491 13006 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.205909 13083 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:48.206491  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 64665.5us
W20260709 13:14:48.209172  7958 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37774 (request call id 141) took 12796 ms (client timeout 15000 ms). Trace:
I20260709 13:14:48.209475  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:48.214926  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 116366us
I20260709 13:14:48.216501  8026 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:48.219151 13074 consensus_queue.cc:260] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.221374 13074 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:48.222093  8063 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:48.223883  8056 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:48.224022 12945 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:48.224536 12945 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:48.226562  8176 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:48.228076  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:48.237625  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.227320  8027 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:48.172534 13055 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:48.225687 12945 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.246233  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:48.246945  8211 consensus_meta.cc:287] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.011s
I20260709 13:14:48.247108  8136 raft_consensus.cc:1813] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Handling vote request from an unknown peer d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:48.247542  8211 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:48.247927  8136 raft_consensus.cc:2410] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:14:48.249647 13114 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:48.250589 13083 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.244206 13055 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.251972 13055 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:48.252007 13103 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:48.252882 13083 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:48.254881 13085 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:48.252878 13102 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:48.256522 13102 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.260609 13112 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:48.261837 13112 raft_consensus.cc:3055] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:48.264153  8208 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:48.209784  7958 rpcz_store.cc:269] 0709 13:14:35.412082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:35.418495 (+  6413us) service_pool.cc:224] Handling call
0709 13:14:48.208824 (+12790329us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:45.905801 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:45.906477 (+   676us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:45.906511 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:14:45.906527 (+    16us) tablet.cc:662] Decoding operations
0709 13:14:45.907851 (+  1324us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:45.907936 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:45.907976 (+    40us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:14:45.908955 (+   979us) tablet.cc:701] Row locks acquired
0709 13:14:45.908976 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:14:45.909174 (+   198us) write_op.cc:270] Start()
0709 13:14:45.909311 (+   137us) write_op.cc:276] Timestamp: P: 1783602885909105 usec, L: 0
0709 13:14:45.909335 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:14:46.173116 (+263781us) log.cc:844] Serialized 3570 byte log entry
0709 13:14:46.745586 (+572470us) op_driver.cc:464] REPLICATION: finished
0709 13:14:46.863978 (+118392us) write_op.cc:301] APPLY: starting
0709 13:14:46.864061 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:46.889096 (+ 25035us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:46.904168 (+ 15072us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:47.016580 (+112412us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:47.036490 (+ 19910us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400172d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb011e28 00007f6dbea4c581 00007f6dbea4bd58 00007f6dbea509f1 00007f6dbea509a9 00007f6dbea50939 00007f6dbea50901 00007f6dbea4fb2d 000000000038f354 000000000038f199
0709 13:14:47.057375 (+ 20885us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 13:14:47.060877 (+  3502us) write_op.cc:312] APPLY: finished
0709 13:14:47.081719 (+ 20842us) log.cc:844] Serialized 89 byte log entry
0709 13:14:47.094266 (+ 12547us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:47.097995 (+  3729us) write_op.cc:454] Released schema lock
0709 13:14:47.103817 (+  5822us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:47.128979 (+ 25162us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:47.130048 (+  1069us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1309363,"reported_tablets":21,"spinlock_wait_cycles":14720,"child_traces":[["op",{"apply.queue_time_us":118133,"apply.run_cpu_time_us":16927,"apply.run_wall_time_us":261309,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41747,"num_ops":9,"prepare.queue_time_us":204519,"prepare.run_cpu_time_us":34575,"prepare.run_wall_time_us":1168264,"raft.queue_time_us":103214,"raft.run_cpu_time_us":47021,"raft.run_wall_time_us":120465,"replication_time_us":835957,"spinlock_wait_cycles":274414976,"thread_start_us":261602,"threads_started":4,"wal-append.queue_time_us":201220,"wal-append.run_cpu_time_us":16140,"wal-append.run_wall_time_us":1734751}]]}
I20260709 13:14:48.276021  8060 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
W20260709 13:14:48.278326  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69037.7us
I20260709 13:14:48.279285 13006 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:48.281363 12999 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:48.261902 13034 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
W20260709 13:14:48.286706  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 104126us
I20260709 13:14:48.287027 12546 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:48.287683 12546 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.288496 12546 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:48.289702  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:48.290881 12920 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.291209 13084 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:48.291980 12920 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.292301 13084 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.291281  8056 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:48.287166  8137 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:14:48.119146  8133 rpcz_store.cc:276] 0709 13:14:47.107921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.110535 (+  2614us) service_pool.cc:224] Handling call
0709 13:14:47.110994 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:47.205982 (+ 94988us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:47.206014 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:47.222317 (+ 16303us) log.cc:844] Serialized 26 byte log entry
0709 13:14:47.271754 (+ 49437us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:47.295153 (+ 23399us) log.cc:844] Serialized 14 byte log entry
0709 13:14:48.065131 (+769978us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b18004c9b40. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ea3c0a 00007f6dc15e3219 00007f6db7f2ba40 00007f6db7f264e4 00007f6db7f26455 00007f6dbb00e060 00007f6dbb020716 00007f6dbb031287 00007f6dbb031208
0709 13:14:48.083580 (+ 18449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.093119 (+  9539us) spinlock_profiling.cc:243] Waited 94.6 ms on lock 0x7b54003c87d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:48.093182 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:48.118375 (+ 25193us) raft_consensus.cc:1685] finished
0709 13:14:48.118400 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.118588 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1776,"prepare.queue_time_us":22905,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":809201,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":12268,"spinlock_wait_cycles":219950080,"thread_start_us":43103,"threads_started":1,"wal-append.queue_time_us":75366}
W20260709 13:14:48.342916  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 185527us
I20260709 13:14:48.271186 13112 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:48.347820  8063 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:14:48.345845  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:48.354530  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.357853  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:48.363015  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:48.371496  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.372520  8209 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:14:48.368520  8060 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:48.380519  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:48.382572 12920 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:48.385228 12546 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.386807 13080 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.387562 13080 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.389194  8056 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:48.390678 13126 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:48.391156 13126 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:48.391515  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.391896  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.391533 13126 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:48.393000 13085 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:48.393133 12629 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:48.393850 13085 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:48.395345  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.372041  8212 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:48.398689  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:48.411017 12713 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.411859 12713 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.411954  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:48.367549 13090 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 13:14:48.415104 13090 consensus_queue.cc:1243] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:48.417337  8058 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:14:48.418864 13084 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:48.419806  8137 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:48.414650  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:48.423426  8729 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:48.422103  8101 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:48.426733  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.399179 12546 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:48.433957  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.402720 13083 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.393862 12629 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.451561  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:48.454528 13178 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:14:48.456367 13090 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:48.458535  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2086) took 1778 ms. Trace:
I20260709 13:14:48.459817  8130 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:14:48.464157  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:48.459879  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.461651  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.478950  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.480417  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.480578 13028 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:48.480980 13028 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:48.481412 13028 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:48.482707 13102 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:48.483484  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:48.484072  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:48.485739  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 146891us
I20260709 13:14:48.488272 12713 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:48.489898 13112 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.491410 13112 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:48.492835 13028 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.499029 13114 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.442691 13080 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:48.514919  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:48.459087  8134 rpcz_store.cc:276] 0709 13:14:46.679957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.680070 (+   113us) service_pool.cc:224] Handling call
0709 13:14:46.700364 (+ 20294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:46.707010 (+  6646us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:46.707056 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:46.715870 (+  8814us) log.cc:844] Serialized 26 byte log entry
0709 13:14:48.007184 (+1291314us) spinlock_profiling.cc:243] Waited 926 us on lock 0x7b18000806e0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:14:48.174122 (+166938us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:48.184789 (+ 10667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.190674 (+  5885us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:48.457682 (+267008us) raft_consensus.cc:1685] finished
0709 13:14:48.457720 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.458298 (+   578us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13605,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":2650,"spinlock_wait_cycles":2130688,"thread_start_us":64526,"threads_started":1,"wal-append.queue_time_us":1447329}
I20260709 13:14:48.516113 13114 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.517048 12629 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:48.517989  8060 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:48.518096 13114 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:48.520133  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 81620.9us
I20260709 13:14:48.526600  8207 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:14:48.518819  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.534461  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:48.537206  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.519560  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.558821  8204 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:14:48.467730 13083 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.538282  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:48.538445  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.527288  8696 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:48.543884 13184 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.288s	user 0.002s	sys 0.000s
I20260709 13:14:48.544452  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:48.540280 13085 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:48.541855  8136 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:48.514541 13099 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:48.565248  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 51572.6us
I20260709 13:14:48.550086  8729 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:48.530002 13055 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.528689  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:48.565454  8129 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:48.551201 12915 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:48.530427  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 108433us
W20260709 13:14:48.561183  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:48.529883 13181 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 13:14:48.570797  8133 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:48.578423  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:48.570797  8137 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
W20260709 13:14:48.572870  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.575289  8130 raft_consensus.cc:2393] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:48.579833  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:48.559837 13028 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:48.583474  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.572460 13006 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:48.587289  8056 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:48.588227  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.592693  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.593657  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:48.597169  8134 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:48.597641  8134 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:48.520764 13126 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.599191  8131 raft_consensus.cc:2410] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:14:48.592545  8212 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
W20260709 13:14:48.600735  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 72164.4us
I20260709 13:14:48.589006 13006 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:48.574185 12915 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.604837  8129 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:48.606727  8133 raft_consensus.cc:2410] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:14:48.607178  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:48.611881 13028 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:48.612299 13028 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:48.612694 13028 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:48.613696  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 82603.9us
I20260709 13:14:48.614444  8693 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:48.615253 13084 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:48.618116  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.618870  8136 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:14:48.619872  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.620711 13099 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:48.621060  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:48.622474  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.625137  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:48.626188  8209 raft_consensus.cc:2410] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:14:48.626786  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:48.627295  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:48.627767 12915 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:48.628068  8204 raft_consensus.cc:2393] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:48.609934 13006 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:48.627529  8206 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:48.631500 13083 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:48.632117  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:48.632748  8210 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:48.633478  8204 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:48.632745  8208 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:48.635308  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:48.635859  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.636132  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:48.636656  8133 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:48.637502  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.637639  8131 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:14:48.638278  8129 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: [13.10-15.11]
I20260709 13:14:48.625185  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:48.635892  8209 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:14:48.590186 12914 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:14:48.640743  8207 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:48.640581  8137 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:48.641345  8027 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:48.648984  8206 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:14:48.648445  8133 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:48.649004  8134 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.656956  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:48.662703  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.657769 13080 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:14:48.670125  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 45042.9us
I20260709 13:14:48.664239  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:48.656213 13090 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:14:48.671468  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 57119us
I20260709 13:14:48.665103 12546 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.675523 12546 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.676566  8203 raft_consensus.cc:2410] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:48.680783  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:48.682968  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:48.651993 12713 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:14:48.669968 13074 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:48.689553  8134 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:48.692306  8730 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:48.696590  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:48.699812  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:48.691368  8132 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:14:48.705843  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:48.706676  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.677673 12546 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:48.706804  8203 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:48.714545  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:48.715366  8724 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:48.684036 13080 raft_consensus.cc:697] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:48.717473  8027 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:48.708433  8209 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:48.708693  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:48.722164  8136 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:48.723771  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 54816.3us
I20260709 13:14:48.693996 13055 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:48.726176  8129 raft_consensus.cc:2410] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 14 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:14:48.692443 13138 raft_consensus.cc:2243] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:14:48.729244  8104 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:48.711711 12914 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:48.718120 13080 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.732894 13030 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:48.724438 12546 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:48.733084 13126 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:48.733326  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:48.733606 13084 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:48.728853 12915 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.735298  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:48.735893 13112 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:48.736017 13138 raft_consensus.cc:2272] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:14:48.750063  8726 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:48.721899 12999 consensus_queue.cc:1059] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:14:48.747063  8207 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
W20260709 13:14:48.751272  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:48.755146  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.746631 13030 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.759621 13030 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:48.759407 12915 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.756299  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:14:48.762854  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.746881 12546 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:48.768817  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 210642us
W20260709 13:14:48.779790  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44609.4us
W20260709 13:14:48.795166  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.744618 12914 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.769013 12546 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:48.760802 12920 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.839993  8176 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:48.830272  8724 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:14:48.839635  8104 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:48.761909 13006 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.814881 13112 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:48.847193 13026 raft_consensus.cc:3055] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:48.847699  8104 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:48.858639 12999 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:48.859339 12999 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:48.859694 12999 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.860015 12999 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:48.861336 12914 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:48.874398  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:48.880493  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.848100  7956 catalog_manager.cc:5697] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 9 to 10, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 10 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:14:48.906605  8721 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:48.844754 13048 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:48.872802 13112 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.847704 13026 raft_consensus.cc:740] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:48.868002 13055 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:48.923527  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.924484 13055 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.843053 12920 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:48.911778 13048 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.917624 13112 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:48.946298  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.922366 12914 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.947965  8027 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:48.938728 13048 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:48.971014  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.971320 13027 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:48.971964 13027 raft_consensus.cc:697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:48.973057 13027 consensus_queue.cc:237] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:48.971004  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:48.966672  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:48.973917 13192 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.122s	user 0.000s	sys 0.005s
I20260709 13:14:48.968425  7956 catalog_manager.cc:5697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 5 to 7, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 7 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:48.984297  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:48.985304  8060 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 16: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:14:48.992961  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:48.998438  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 366092us
I20260709 13:14:48.999058 12915 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:49.014454  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.018110  8056 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:49.030423  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:48.984272 12914 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:49.047886  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 354439us
I20260709 13:14:49.049283  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:49.055794  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.052798  8056 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
W20260709 13:14:49.061769  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 62616.4us
I20260709 13:14:49.060544  7956 catalog_manager.cc:5697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 7 to 10, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 10 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:49.067333  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:49.052156 12945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.047s	sys 0.054s
I20260709 13:14:49.074872  8062 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:14:49.000092 13026 consensus_queue.cc:260] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.054220 13055 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:49.073644 12945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.047s	sys 0.055s
W20260709 13:14:49.083104  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.095361  8060 raft_consensus.cc:1217] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 10.7->[12.8-12.8]   Dedup: 12.8->[]
I20260709 13:14:49.100862  8060 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:14:49.058131 13006 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:49.108410  8101 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:49.126835  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:49.132393  8696 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:49.111593 13126 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.159551 13083 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:49.159636  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.160211 13055 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.161314 13055 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:49.162599 13201 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:49.163142  7956 catalog_manager.cc:5697] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:49.167196  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:49.168485  8721 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:49.161509 13195 consensus_queue.cc:1059] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.111s
W20260709 13:14:49.162636  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 59196.3us
I20260709 13:14:49.178498  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:49.179283  8060 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:49.193732  8696 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:14:49.196954  8693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.005s	sys 0.030s
W20260709 13:14:49.197500  8693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.005s	sys 0.030s
I20260709 13:14:49.199018  8134 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
W20260709 13:14:49.197947  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 523920us
I20260709 13:14:49.158906 13099 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.194265  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:49.185400  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 337830us
W20260709 13:14:49.251988  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.193478 13006 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:49.185535 13205 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.176142 13055 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:49.213193 13090 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:49.287963  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:49.160288 13083 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.292757  8062 raft_consensus.cc:2410] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:14:49.177079 13126 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:49.192602 12999 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.272293 13197 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:49.295989  8104 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:49.275805 13006 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:49.309799  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:49.312316  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.310691  8062 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:14:49.265867 13099 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.321827  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:49.325527  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:49.326216  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 149839us
I20260709 13:14:49.331086  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:49.306674 13006 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:49.304602 13197 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:49.363039  7956 catalog_manager.cc:5697] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 10 to 12, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 12 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:49.360884  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 81026.1us
I20260709 13:14:49.291332 13205 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.370991  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:49.373322  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:49.375865  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 249166us
I20260709 13:14:49.175849 13138 ts_tablet_manager.cc:1916] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:14:49.388530  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.299141 13090 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:49.374948 13194 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.248s	user 0.000s	sys 0.002s
I20260709 13:14:49.397799  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:49.365708 13199 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:49.370893 12914 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:49.400830  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 115786us
I20260709 13:14:49.369356 13197 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:49.424265  8060 raft_consensus.cc:2410] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:14:49.422478  7956 catalog_manager.cc:5697] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 6 to 11, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 11 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.418625 12713 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:49.432142 12713 raft_consensus.cc:697] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:49.432335 13075 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:49.433271 13099 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:49.433456 13075 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.418792 13102 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.434839 13102 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.435635 13099 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:49.437377 13075 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:49.418881 13193 log.cc:927] Time spent T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.272s	user 0.002s	sys 0.000s
I20260709 13:14:49.403504 13199 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.438601  8063 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:49.433876 12713 consensus_queue.cc:237] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.441550 13213 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:49.456558 13102 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:49.465826 13115 raft_consensus.cc:3055] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:49.446460 13199 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:49.470929 13028 consensus_meta.cc:287] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.858s	user 0.014s	sys 0.001s
I20260709 13:14:49.474936 12999 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:49.476286 13205 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:49.477301 13074 consensus_meta.cc:287] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.781s	user 0.014s	sys 0.005s
I20260709 13:14:49.478684 13090 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:49.479385  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:49.480440  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:49.481138  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:49.547981  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:49.537412  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 233824us
I20260709 13:14:49.482275 13079 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.482435  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:49.483739  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:49.484236 12546 consensus_meta.cc:287] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.642s	user 0.017s	sys 0.003s
I20260709 13:14:49.485242 13026 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:49.485378  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:49.590854  8054 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:49.535974 12945 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:49.486537 13112 consensus_meta.cc:287] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.004s
I20260709 13:14:49.488603  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:49.489496 13030 consensus_meta.cc:287] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.729s	user 0.009s	sys 0.002s
I20260709 13:14:49.489445  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:49.492134 13213 raft_consensus.cc:3055] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:14:49.495039 13207 log.cc:927] Time spent T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.309s	user 0.005s	sys 0.000s
I20260709 13:14:49.496194 12920 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:49.502022 13214 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:49.609525  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:49.613325  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:49.503371  7956 catalog_manager.cc:5697] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:14:49.614898  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:49.504664  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:49.617205  8060 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:49.505134 13198 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:49.509399  8027 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:49.510087 12914 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.620081 13089 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:14:49.620378 12914 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.621124 13200 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:14:49.621822 13200 consensus_queue.cc:1243] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 9.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:49.622742 12914 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:49.623308 12914 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:49.623642 12914 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.623980 12914 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:49.620111  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 110849us
I20260709 13:14:49.628033 13126 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:49.515316  8209 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:14:49.602314 13030 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.629619  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:49.630435  8209 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:49.631484  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:49.632340  8209 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 16: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:49.633214  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:49.633695  8209 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:49.592897 12546 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:49.635473 13090 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.636055 13090 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.636940 13138 ts_tablet_manager.cc:1929] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 10.8
I20260709 13:14:49.597704 13112 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.635494  8058 raft_consensus.cc:1813] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Handling vote request from an unknown peer d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:49.638854 13138 log.cc:1199] T a478a4313e6149378cae4b6f833e3c2a P 8a968cada1e4446ebb2a5e167176e649: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/wals/a478a4313e6149378cae4b6f833e3c2a
I20260709 13:14:49.639051  8058 raft_consensus.cc:2368] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
I20260709 13:14:49.640393 13112 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:49.640916 13030 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:49.641219 13090 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:49.647050  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:49.517309  8207 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 15 index: 6. (index mismatch)
I20260709 13:14:49.518466  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:49.521042 13083 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:49.527846 13048 consensus_meta.cc:287] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.544s	user 0.012s	sys 0.000s
W20260709 13:14:49.532476 13211 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:14:49.534534 13115 raft_consensus.cc:740] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:49.658041  8055 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:14:49.593854  8063 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:49.658631 13138 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36736 (request call id 0) took 2956 ms. Trace:
I20260709 13:14:49.658769 13048 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:49.659094 13138 rpcz_store.cc:276] 0709 13:14:46.720145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.764334 (+ 44189us) service_pool.cc:224] Handling call
0709 13:14:47.015795 (+251461us) ts_tablet_manager.cc:1151] Deleting tablet a478a4313e6149378cae4b6f833e3c2a
0709 13:14:47.015890 (+    95us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:14:49.495964 (+2480074us) tablet_metadata.cc:665] Metadata flushed
0709 13:14:49.636391 (+140427us) tablet_metadata.cc:665] Metadata flushed
0709 13:14:49.658515 (+ 22124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1651847,"tablet-delete.queue_time_us":173617,"thread_start_us":78557,"threads_started":1}
I20260709 13:14:49.661026 13048 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:49.661239 13199 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:49.595870  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:49.546886 13074 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:49.600591  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:49.602035  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.537909 13028 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:49.597851 13213 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:49.657826 13115 consensus_queue.cc:260] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:49.586436  8062 raft_consensus.cc:2393] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:49.704650  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:49.705435  8207 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:49.593794 13079 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:49.729842  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:49.730767  8210 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:49.598853 12945 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:49.662262 13006 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:49.740722  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:49.735697  7956 catalog_manager.cc:5697] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 9 to 12, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 12 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.671716 13198 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 14 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:49.749037  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:49.749763  8135 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:14:49.758695  8729 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:49.762080  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:49.762910  8206 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:49.616495 13214 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.767414  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:49.770282  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 330033us
W20260709 13:14:49.764438  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.766273  8721 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:14:49.781872  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:49.733919 13048 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.778695  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:49.780929  8210 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:49.793733  8203 raft_consensus.cc:2410] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:49.795966  8207 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:49.798555  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:49.799437  8204 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:14:49.802260  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:49.803160  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 396440us
I20260709 13:14:49.802995  8203 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:49.806291  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 776) took 1188 ms. Trace:
I20260709 13:14:49.807422  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:49.777284 13115 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:49.806859  8135 rpcz_store.cc:276] 0709 13:14:48.617559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.691419 (+ 73860us) service_pool.cc:224] Handling call
0709 13:14:48.691850 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:49.765855 (+1074005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.805947 (+ 40092us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400001754. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:49.806211 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2416930176}
I20260709 13:14:49.817111  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:49.802495  7956 catalog_manager.cc:5697] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 5 to 11, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 11 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:49.774439 12920 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:49.830662  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.825070  8212 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:49.833367  8104 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:49.841137  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1977) took 1122 ms. Trace:
I20260709 13:14:49.845504  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:49.846863  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:49.848055  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:49.848654  8212 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:49.859083  8029 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:49.859686  8729 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:49.861505  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:49.862311  8203 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:14:49.861639  8176 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:49.863505  8210 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:49.861961  7956 catalog_manager.cc:5697] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 8 to 9, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 9 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:49.898321  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.843223 12546 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:49.851815 12915 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:49.828413 13048 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.863722 13030 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:49.850328 12920 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:49.928575  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:49.944034  8210 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:49.846954 13090 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:14:49.842371  8208 rpcz_store.cc:276] 0709 13:14:48.696925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.698809 (+  1884us) service_pool.cc:224] Handling call
0709 13:14:48.713862 (+ 15053us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.732930 (+ 19068us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:14:49.053418 (+320488us) log.cc:844] Serialized 14 byte log entry
0709 13:14:49.804389 (+750971us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:49.804459 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:49.804515 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.804620 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.804774 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":272766,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":6886,"raft.queue_time_us":120822,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":77,"thread_start_us":987865,"threads_started":3,"wal-append.queue_time_us":360620}
W20260709 13:14:49.944396  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 338361us
I20260709 13:14:49.913072 13199 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:49.913133 13216 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:14:49.959290  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 125958us
W20260709 13:14:49.941962 13201 consensus_meta.cc:287] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.779s	user 0.013s	sys 0.007s
I20260709 13:14:49.934120 13217 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:49.967950  8724 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:14:49.969906  8721 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:49.968268 13214 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:49.968607  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:49.960904  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:49.974303  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:49.942710 12915 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.955066 13216 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.978917  8729 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:14:50.044775  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 186060us
I20260709 13:14:50.048727  8204 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:49.979257  8104 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:49.964100 13197 consensus_meta.cc:287] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.523s	user 0.005s	sys 0.014s
I20260709 13:14:49.977779  8209 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:14:49.984460  8210 raft_consensus.cc:2410] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:50.001145 13221 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:50.002332  8730 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:14:50.002933 12915 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:50.008113 13075 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:50.008953  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:50.009781 12546 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:50.011132 13199 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:14:50.015020  8208 pending_rounds.cc:85] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 10
I20260709 13:14:50.024147 13224 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:50.023222 13026 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:50.025425  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:50.027647 13048 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:50.030637  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 180718us
W20260709 13:14:50.103785  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 56500.6us
I20260709 13:14:49.934917 13218 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.124s
I20260709 13:14:50.070708  8208 pending_rounds.cc:107] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:14:49.966598 13201 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:49.863579 13213 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:49.975145 13090 consensus_queue.cc:1243] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:14:50.055943  8212 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:49.976445 13028 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:49.904488 13198 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.071712  8209 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:50.075074 13026 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.072962  8206 pending_rounds.cc:85] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 6
W20260709 13:14:50.031579  8175 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.026s	sys 0.050s
I20260709 13:14:50.039136  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:50.074666  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 95724.3us
I20260709 13:14:50.106980  8208 raft_consensus.cc:2889] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:50.047711 13216 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:50.109804  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:50.116286  8204 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
W20260709 13:14:50.122622  8054 consensus_meta.cc:287] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.516s	user 0.012s	sys 0.000s
W20260709 13:14:50.066473 13231 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.102s	user 0.000s	sys 0.004s
I20260709 13:14:50.063874 13197 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.072566 13224 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:50.160307  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.180328  8054 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:14:50.180548 13224 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.176083 13079 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:50.185500  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:50.163069  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:50.072901 12546 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.188493 12546 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:50.071183 13221 raft_consensus.cc:697] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:50.175488  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 166638us
I20260709 13:14:50.189697 13221 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.191462  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:50.177338 13219 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:50.192432 13219 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.192957 13219 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:50.114378 12945 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:50.123294  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.194797 13028 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:50.195194  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:50.133271  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.195487 13028 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.198318 13241 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:50.198951 13241 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.199788 12920 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:50.201573 13241 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:50.186154  8204 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:14:50.182152 13224 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:50.177125 13235 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:50.205289 13235 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:50.205673 13235 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:50.206156  8721 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:14:50.108249 13006 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:50.210647 13126 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:50.211404 13126 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:50.194413 13242 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:50.068048 13075 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:50.228590 13240 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:14:50.229804 13230 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:50.230381 13230 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.179003 13220 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.232415 13220 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:50.234040  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.236670  8206 pending_rounds.cc:107] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:14:50.237397  8206 raft_consensus.cc:2889] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:50.166862  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:50.104043 13214 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:50.241966  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50624.6us
I20260709 13:14:50.241020  8696 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:50.243803  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:50.245285  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:50.245894  8210 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:50.245821 13006 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:50.247083 13006 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.247567 13006 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:50.247996 13075 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:50.245829  8729 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:50.255725 13220 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:50.257010  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:50.259749 13229 log.cc:927] Time spent T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.147s	user 0.002s	sys 0.000s
W20260709 13:14:50.260020  8175 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.027s	sys 0.055s
W20260709 13:14:50.261879 12914 consensus_meta.cc:287] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.637s	user 0.016s	sys 0.001s
I20260709 13:14:50.263011  8726 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:50.267282 12914 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:50.178923 13223 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:14:50.269942 12914 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:50.263414 13126 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:50.264036 13197 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:50.178287 13226 log.cc:927] Time spent T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
W20260709 13:14:50.271165  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 876394us
I20260709 13:14:50.198144 13028 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:50.277931  8203 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:50.277653  8104 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:50.266481 13214 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.276087 13089 raft_consensus.cc:2764] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:14:50.285671  8729 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:50.287604  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:50.240924 13026 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:50.240217 13027 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.287102  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:50.290025  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:50.305214  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:50.314177  8726 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
W20260709 13:14:50.315192  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 276127us
I20260709 13:14:50.310443  8209 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:50.311069 13028 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:50.318280  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:50.319350  8724 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:50.319789  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:50.321179  8726 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:14:50.322903  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:50.324047 13099 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:14:50.324503  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45232.3us
I20260709 13:14:50.326560 13090 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:50.327208 13090 raft_consensus.cc:697] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:50.326884 13099 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:14:50.326972 12914 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:50.328387 13090 consensus_queue.cc:237] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:50.330060  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:50.330778  8724 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:50.332702 13213 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:50.334631  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 229968us
I20260709 13:14:50.335386  8029 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:50.336194 12914 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:50.338611 12914 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:50.339882 13075 raft_consensus.cc:3055] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:50.340221 13075 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:50.341048 13075 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.342084 13075 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:50.342271  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:50.343415  8205 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:50.346261  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:50.346940  8207 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:50.351987 13214 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:50.278683 12920 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:50.352933 13216 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.353699 13230 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:50.355085 13216 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:50.355010 13006 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.356355  8027 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:50.357239 12920 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:50.280277 13201 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:50.357625  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:50.357996 13201 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.358002 12920 raft_consensus.cc:697] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:50.358538  8721 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:50.359118 12920 consensus_queue.cc:237] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.280133 13074 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:50.357318 13006 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:50.358537 13201 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:50.363365  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:50.364104  8726 raft_consensus.cc:2393] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:50.365125  8209 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: [12.10-16.12]
I20260709 13:14:50.337347  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:50.367293  8027 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:50.368889  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.373594  8724 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
W20260709 13:14:50.374840  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 48535.8us
I20260709 13:14:50.377072  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:50.379599  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:50.384511  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 43729.8us
I20260709 13:14:50.387946  8207 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:14:50.391562  8101 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:50.397709  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 60437.4us
W20260709 13:14:50.399482  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 111945us
I20260709 13:14:50.374831 13089 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:50.409340  8696 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:50.370204 13028 raft_consensus.cc:697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:50.407397  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:50.390864 12945 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:50.415855  8721 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:14:50.387779 12914 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:50.417728  8207 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:14:50.416718  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:50.322817 13027 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.420581  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:50.420460  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:50.408169 12713 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:50.425848  8027 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:50.399058 13075 consensus_queue.cc:1059] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:14:50.479223  8101 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:50.421934 12914 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.482796  8207 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:14:50.483502  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:50.424486 13079 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:50.471217 13200 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:14:50.490854 13028 consensus_queue.cc:237] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:50.497068  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:50.444937 13216 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:50.447398 13195 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:50.505267  8054 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:50.486670  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:50.445148 13099 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:50.467345 13080 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:50.508401  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:50.517582  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:50.470314 13218 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.418874 12915 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:14:50.510118  8212 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:50.514605  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:50.517596  8134 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:50.486466 12914 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:50.510666 13195 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.521212  8059 pending_rounds.cc:85] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 10
I20260709 13:14:50.521505 13030 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:50.518159  8129 raft_consensus.cc:2376] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15. Re-sending same reply.
I20260709 13:14:50.522264  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:50.523411  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 52508.3us
I20260709 13:14:50.527294  8059 pending_rounds.cc:107] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 9.11
I20260709 13:14:50.530208  8059 raft_consensus.cc:2889] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:50.494588 13027 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:50.467835 13217 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:14:50.510700 12713 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:14:50.542941  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:50.528659 13115 consensus_meta.cc:287] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.692s	user 0.012s	sys 0.000s
I20260709 13:14:50.482390 12945 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 13:14:50.549929  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 70804.1us
I20260709 13:14:50.528476 13221 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:50.552521  8696 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:50.528746 13230 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.531391 13195 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:50.509769 13197 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:14:50.513800 13079 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:50.533010 13099 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:50.530637 13216 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:50.556329  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3161) took 1035 ms. Trace:
I20260709 13:14:50.528400 13080 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:50.561700  8101 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:50.574290  8101 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:50.575795  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:50.550449  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:50.548847 13225 raft_consensus.cc:3055] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:50.583636  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3169) took 1060 ms. Trace:
I20260709 13:14:50.545575 12920 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:50.584151  8211 rpcz_store.cc:276] 0709 13:14:49.523302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.192859 (+669557us) service_pool.cc:224] Handling call
0709 13:14:50.193177 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:50.193460 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:50.424420 (+230960us) log.cc:844] Serialized 14 byte log entry
0709 13:14:50.583222 (+158802us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:50.583288 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:50.583351 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.583432 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:50.583568 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":259088,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"thread_start_us":299205,"threads_started":2}
I20260709 13:14:50.570858  8208 rpcz_store.cc:276] 0709 13:14:49.520811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.014465 (+493654us) service_pool.cc:224] Handling call
0709 13:14:50.014819 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:50.258658 (+243839us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:50.258688 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:50.259309 (+   621us) log.cc:844] Serialized 52 byte log entry
0709 13:14:50.416916 (+157607us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:50.528081 (+111165us) log.cc:844] Serialized 14 byte log entry
0709 13:14:50.533601 (+  5520us) log.cc:844] Serialized 14 byte log entry
0709 13:14:50.540103 (+  6502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.540199 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:50.556062 (+ 15863us) raft_consensus.cc:1685] finished
0709 13:14:50.556090 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:50.556258 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13199,"prepare.queue_time_us":45836,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":17946,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":2882,"spinlock_wait_cycles":25930496,"thread_start_us":263669,"threads_started":2,"wal-append.queue_time_us":222542}
I20260709 13:14:50.528887 13218 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:50.557886 13221 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:50.557737 12713 raft_consensus.cc:697] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:50.561259 13006 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:50.586642 13228 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:50.604353 13080 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.574535 13250 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:50.583509 13225 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:50.559293 13230 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:50.587033 13241 raft_consensus.cc:3055] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:50.605162 13026 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:50.610786  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 142468us
I20260709 13:14:50.596918 13221 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:50.600229 13126 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:50.626456 13099 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.616886 13216 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:50.632805  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:50.612316 13006 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:50.574491 13235 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.631260 13228 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:50.629115 13250 raft_consensus.cc:697] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:50.660691  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3170) took 1137 ms. Trace:
W20260709 13:14:50.793744  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.622035 12713 consensus_queue.cc:237] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:50.627761 13214 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:50.638413 13079 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.596094 13074 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:14:50.628237 13241 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:50.819512  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 189628us
I20260709 13:14:50.806425  8062 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
W20260709 13:14:50.818944  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 268745us
W20260709 13:14:50.818881  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 242780us
I20260709 13:14:50.809542  8204 rpcz_store.cc:276] 0709 13:14:49.523585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.204095 (+680510us) service_pool.cc:224] Handling call
0709 13:14:50.204451 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:50.204699 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:50.465969 (+261270us) log.cc:844] Serialized 14 byte log entry
0709 13:14:50.591017 (+125048us) log.cc:844] Serialized 14 byte log entry
0709 13:14:50.660295 (+ 69278us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:50.660370 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:50.660420 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.660497 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:50.660639 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38927,"prepare.queue_time_us":250636,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":47616,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":60,"spinlock_wait_cycles":6602112,"thread_start_us":417994,"threads_started":3,"wal-append.queue_time_us":148277}
I20260709 13:14:50.823220  8104 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:50.825852  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:50.828060  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:50.828605  8055 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:14:50.827546  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:50.825870  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:50.829773  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:50.830387  8730 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:50.834818  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:50.806165 13126 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:50.837395  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:50.841066  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:50.841161 13228 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.842077 13250 consensus_queue.cc:237] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:50.845808 13264 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:50.849714 13224 consensus_meta.cc:287] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.645s	user 0.011s	sys 0.004s
I20260709 13:14:50.628886 13201 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:50.858407  8056 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:50.825696 13260 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.880865  8721 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
W20260709 13:14:50.887228  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 59752.9us
I20260709 13:14:50.888219  8029 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:50.834904 13261 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:50.827471 13075 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
W20260709 13:14:50.907308  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.869305 13030 consensus_queue.cc:1059] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:14:50.857399 13089 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:14:50.887701 12945 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:50.907581 13261 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.952448 12915 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:50.947710 13218 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:50.888200 13264 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:50.972920 13213 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.968400 13261 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:50.957734 13214 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:50.969777 13030 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:50.893288 13260 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:50.885169 13225 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.976238 13241 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 10, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:50.959446 13265 consensus_queue.cc:1059] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:14:50.980824 12915 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.915330 13006 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:51.013357 13268 log.cc:927] Time spent T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:14:51.015478 13219 consensus_meta.cc:287] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.822s	user 0.009s	sys 0.012s
I20260709 13:14:51.003859 13213 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.022913 13030 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:51.060650 13030 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:51.061491 13213 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:51.011934 13262 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:14:51.004858 13026 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.015259 13201 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:51.014038 13263 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
W20260709 13:14:51.016644 13252 log.cc:927] Time spent T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:14:51.057433 13218 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:51.057463 12915 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:51.069324 13218 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:51.058871 13241 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:51.015993 13221 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.074304 13267 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:14:51.063313 13219 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:50.995643 13126 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.078701 13219 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.020876 12914 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:51.082700 13221 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:51.082835 12914 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:51.075342 13264 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:51.077469 13273 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:14:51.083844 13273 consensus_queue.cc:1243] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:14:51.075989 13195 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.084441 13074 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:51.084800 13074 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:51.085646 13074 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:51.086774 13214 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.072026 13218 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:51.096160 13218 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:51.099468 13260 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.112648 13261 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.121548 13261 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.114035 13030 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:50.960229  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 134023us
I20260709 13:14:51.127537 13030 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.114010 13195 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.124055 12915 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.135234  7945 catalog_manager.cc:5028] TS 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): tablet a478a4313e6149378cae4b6f833e3c2a (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
W20260709 13:14:51.188455  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.190825 13030 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:51.193316 13030 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:51.210690 13278 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.211763  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.211905 13278 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.212671  8135 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:51.214421 13278 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.214314  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3162) took 1693 ms. Trace:
I20260709 13:14:51.215983  8206 rpcz_store.cc:276] 0709 13:14:49.521099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.064363 (+543264us) service_pool.cc:224] Handling call
0709 13:14:50.072695 (+  8332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:50.367526 (+294831us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:50.367555 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:50.376427 (+  8872us) log.cc:844] Serialized 26 byte log entry
0709 13:14:50.544961 (+168534us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:50.544993 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.545054 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.213874 (+668820us) raft_consensus.cc:1685] finished
0709 13:14:51.213899 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.214247 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":126536,"threads_started":1,"wal-append.queue_time_us":203830}
I20260709 13:14:51.218885  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:51.219508  8135 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:51.226775  8135 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:51.227814  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.228657  8135 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:51.229599  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.230465  8135 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:51.231356  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.232971  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.235417  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.236490  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.237002  8135 raft_consensus.cc:2376] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13. Re-sending same reply.
I20260709 13:14:51.238492  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.239080  8135 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 16: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:51.240151  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.241083  8135 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:14:51.242285  8722 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:14:51.242295  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.245119  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.246784  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.248291  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.249112  8135 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 11: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:51.250281  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:51.251590  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 156518us
I20260709 13:14:51.251694  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:51.252310  8135 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:51.258565  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.263087 13278 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:51.268894  8135 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:51.269999  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.271143  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:51.274094  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.275204  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.277673  8135 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:14:51.314957 13274 raft_consensus.cc:3055] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:51.315526 13274 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:51.316459 13274 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.317835 13274 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:51.331362 13274 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.331948 13274 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.334738 13274 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.373196  8056 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
W20260709 13:14:51.377599  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 489449us
I20260709 13:14:51.378942  8029 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:51.380643  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:51.381618  8029 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:51.383150  8029 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:51.384284  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:51.385398 13281 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:51.386947  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:51.388135  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:51.386169 13281 raft_consensus.cc:697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:51.389225  8029 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:51.390305 13281 consensus_queue.cc:237] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.396518 13080 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:51.397296 13080 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:51.397715  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.398813 13281 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:51.399272 13281 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.399662 13281 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:51.402113  8104 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.000s	sys 0.011s
W20260709 13:14:51.402480  8104 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.001s	sys 0.011s
W20260709 13:14:51.402751  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 566086us
I20260709 13:14:51.403446  8104 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:51.404692 13115 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:51.405102 13115 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.405438 13115 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:51.397828 13080 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:51.411152 13214 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:51.413650 13274 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:51.414170 13274 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:51.416712 13274 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:51.417111 13281 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.419065 12920 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:51.419464 12920 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:51.419804 12920 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:51.420493 13115 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.423362 13214 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:51.423969 13214 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:51.426894  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.427625 13274 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:51.428151 13290 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:51.428233 13274 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:51.428668 13290 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:51.438517 13290 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:51.443418 13225 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:51.447333 13281 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.448015 13281 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:51.448372 13281 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.449623 13281 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:51.430792 13115 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.431206 13274 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:51.426297 13214 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:51.432219 13075 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:51.451176 13075 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:51.451563 13075 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:51.433427 13080 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.454252 13080 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.454382  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:51.455061  8206 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:51.455467 13080 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:51.456103 13080 consensus_queue.cc:1243] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:14:51.464848 12920 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.465438 13290 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.463984 13075 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.468631  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3182) took 1126 ms. Trace:
I20260709 13:14:51.469013  8203 rpcz_store.cc:276] 0709 13:14:50.367937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.369387 (+  1450us) service_pool.cc:224] Handling call
0709 13:14:50.369687 (+   300us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:50.388057 (+ 18370us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:14:50.529618 (+141561us) log.cc:844] Serialized 14 byte log entry
0709 13:14:50.915996 (+386378us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:50.968109 (+ 52113us) log.cc:844] Serialized 52 byte log entry
0709 13:14:50.968350 (+   241us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:50.968385 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.968462 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.468392 (+499930us) raft_consensus.cc:1685] finished
0709 13:14:51.468414 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.468573 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2961,"prepare.queue_time_us":57892,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":50836,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":13461,"thread_start_us":426582,"threads_started":2,"wal-append.queue_time_us":416426}
I20260709 13:14:51.470439 13099 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.470927 13099 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.472817 13099 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:51.476204  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:51.477303  8206 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:51.479504  8206 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:51.481683  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:51.482230  8204 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:51.483268  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:51.484308  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:51.484632  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:51.485400  8210 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:51.485539  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:51.484762  8204 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:14:51.486763  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:51.490084 13281 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.493566  8055 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:51.495644  8057 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:51.502270  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:51.504035  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:51.504627 13259 log.cc:927] Time spent T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.688s	user 0.004s	sys 0.000s
I20260709 13:14:51.505177  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:51.505738  8055 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:51.505348  8204 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:14:51.510834  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.511148 13290 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.511420  8133 raft_consensus.cc:2376] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13. Re-sending same reply.
I20260709 13:14:51.512389 13290 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.512847 13290 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.514858 13290 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.504753  8057 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:51.520071  8055 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:51.524174  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:51.527896  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.532229  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:51.533612  8059 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:51.625119  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:51.625485  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:51.538868 13290 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:51.630121  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:51.631093  8055 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 11: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:51.631582 13290 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.633870 13290 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.635012  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:51.636137  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:51.636665  8059 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:51.637439  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:51.654955 13225 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:51.656167 13219 raft_consensus.cc:3055] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:51.656520 13219 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:51.657298 13219 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.658869 13219 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:51.660537  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.661185  8135 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:51.662065  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:51.663026  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:51.663504  8135 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:51.667511  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:51.668105  8133 raft_consensus.cc:2376] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13. Re-sending same reply.
I20260709 13:14:51.668955  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:51.669097  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:51.669999  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:51.672067  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:51.670194  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:51.672587  8133 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:51.673200  8054 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:51.674003  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:51.675030  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:51.666788  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:51.678705  8059 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:14:51.678918  8101 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:51.670780  8061 consensus_meta.cc:287] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.831s	user 0.010s	sys 0.000s
I20260709 13:14:51.679627  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:51.680106  8061 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:14:51.680388  8059 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:51.681457  8059 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:14:51.682600  8059 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:51.686151  8054 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:51.684001  8059 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:51.688535  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:51.691322  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:51.693377  8059 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:14:51.693542  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:51.690363 13214 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:51.696154 13218 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:51.700193 13075 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.701812  8134 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:51.702953  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:51.703657  8175 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.005s	sys 0.005s
I20260709 13:14:51.704020  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:51.704212  8175 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.005s	sys 0.005s
I20260709 13:14:51.704802  8134 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:51.705557  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:51.705986  8101 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:51.710824  8135 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:14:51.712530  8059 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:51.716640  8026 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:51.717882  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:51.706528 13267 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:51.691694 13219 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:51.721632  8059 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:14:51.720148 13290 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:51.713232 13261 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:14:51.722716 13218 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:51.728255 13218 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.719590  8054 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:51.728605 13218 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:51.723783  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:51.729918  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.733338 13267 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:51.733723 13267 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.733803 13278 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:51.734143 13267 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:51.749279 13099 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:51.750133  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:51.749900 13099 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:51.750936  8209 raft_consensus.cc:2410] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:51.752381 13099 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:51.754833  8133 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:14:51.755702  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.756604  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:51.757208  8133 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:14:51.759495  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.759380 13218 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:51.761736  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 872) took 1392 ms. Trace:
W20260709 13:14:51.757100  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.749997  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:51.763833  8134 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:51.758237  8133 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:14:51.766201  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 61389.4us
I20260709 13:14:51.767218  8027 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:51.768604  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:51.769107  8729 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:51.769748  8054 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 13:14:51.770459  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 51000.1us
I20260709 13:14:51.762094  8726 rpcz_store.cc:276] 0709 13:14:50.369166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.369275 (+   109us) service_pool.cc:224] Handling call
0709 13:14:50.392580 (+ 23305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:50.404068 (+ 11488us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:50.529013 (+124945us) log.cc:844] Serialized 14 byte log entry
0709 13:14:50.629963 (+100950us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:50.635595 (+  5632us) log.cc:844] Serialized 26 byte log entry
0709 13:14:50.635875 (+   280us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:50.635949 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.636112 (+   163us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.761530 (+1125418us) raft_consensus.cc:1685] finished
0709 13:14:51.761550 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.761661 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11009,"prepare.queue_time_us":138827,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":9365,"raft.queue_time_us":12358,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":20320,"thread_start_us":165307,"threads_started":2,"wal-append.queue_time_us":284098}
I20260709 13:14:51.780589  8026 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:51.782186  8135 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:51.783178  8728 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:14:51.785435  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:51.775120  8175 connection.cc:256] RPC call timeout handler was delayed by -1.3241101790001721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29150000000000004s for a response
I20260709 13:14:51.786038  8135 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:51.786270  8175 connection.cc:256] RPC call timeout handler was delayed by -1.123425528000098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
W20260709 13:14:51.786656  8175 connection.cc:256] RPC call timeout handler was delayed by -1.0298255279999466s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
I20260709 13:14:51.786860  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:51.787173  8175 connection.cc:256] RPC call timeout handler was delayed by -1.0136255280001478s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
I20260709 13:14:51.784260  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:51.772804 13075 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:51.797225  8729 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:14:51.798281  8057 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:14:51.776461 13099 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:51.799491  8134 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:51.801774  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.802471  8134 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:51.804618  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:51.804950  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.805209  8134 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:51.809852 13274 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:51.810587 13274 consensus_queue.cc:1243] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:51.812160 13274 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:51.812801 13274 consensus_queue.cc:1243] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 14.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:51.812988 13267 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.813524 13278 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:51.813974 13278 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:51.792699 13219 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:14:51.814841 13278 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:51.815454 13219 raft_consensus.cc:3055] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:51.815845 13219 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:51.815999 13278 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:51.816710 13219 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 15.6, Last appended by leader: 6, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.783581 13214 consensus_queue.cc:1059] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:14:51.818933 13214 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:51.819253 13214 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:51.819990 13214 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.820586  8133 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:14:51.821012 13217 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:51.821445  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.821832 13217 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:51.822542 13273 raft_consensus.cc:3055] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:51.822962 13273 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:51.814589  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.823741 13273 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:51.824934 13273 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:51.824067  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.827764  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.822543  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.830698  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:51.831634  8056 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:51.831808  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.832656  8133 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:51.805701 13099 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:51.793155 13075 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.828374 13218 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:51.821110 13214 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:51.833645  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:51.836827  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.840600  8133 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
I20260709 13:14:51.841701  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.813542 13230 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:51.839775 13075 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:51.842792 13230 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.843230 13230 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:51.847486  8132 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:14:51.848492  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:51.849380 13235 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.849485  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:51.850034 13230 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:51.850116  8132 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:51.842388  8133 raft_consensus.cc:2368] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:14:51.851881 13267 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:51.853653  8132 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:14:51.845212 13099 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:51.846210  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:51.860301  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:51.861563  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:51.847005 13217 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:51.862771  8138 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:51.863370  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 72300.6us
I20260709 13:14:51.863344  8135 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:14:51.871084  8132 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17. Ops: [16.13-16.13]
W20260709 13:14:51.914824  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:51.918244  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 119560us
W20260709 13:14:51.937546  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 76069.1us
I20260709 13:14:51.917052 13099 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:14:51.919997 13199 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.363s	user 0.005s	sys 0.009s
I20260709 13:14:51.888058 13230 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:51.951442 13230 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:51.951987 13230 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:51.962488  8724 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 874) took 1350 ms. Trace:
W20260709 13:14:51.962734  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 170534us
W20260709 13:14:51.954828 13199 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.400s	user 0.006s	sys 0.009s
I20260709 13:14:51.917894 13261 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:14:51.962997  8724 rpcz_store.cc:276] 0709 13:14:50.611859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.612107 (+   248us) service_pool.cc:224] Handling call
0709 13:14:50.626769 (+ 14662us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:50.639463 (+ 12694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:50.639496 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:50.639909 (+   413us) log.cc:844] Serialized 26 byte log entry
0709 13:14:50.858959 (+219050us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:14:50.859001 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.859064 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.962193 (+1103129us) raft_consensus.cc:1685] finished
0709 13:14:51.962215 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.962418 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4704,"raft.queue_time_us":12420,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":128,"thread_start_us":39728,"threads_started":1,"wal-append.queue_time_us":238892}
I20260709 13:14:51.965624  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:51.941804 13099 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:51.957019 13230 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:51.968883  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:51.974123  8724 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:14:51.975927  8026 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:51.986086  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.983281 13306 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:14:51.971186 13099 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:51.995855  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:51.997037  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:51.998351  8060 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:51.999891  8175 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:52.002293  8026 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:51.973354 13074 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:52.006197  8026 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:52.012573  8026 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:52.017758  8138 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 12. (term mismatch)
I20260709 13:14:52.018244  8138 pending_rounds.cc:85] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 11
I20260709 13:14:52.018525  8138 pending_rounds.cc:107] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 11.12
I20260709 13:14:52.018767  8138 raft_consensus.cc:2889] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:52.031077  8209 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:14:52.033164  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:52.033746  8207 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:52.030537  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:52.011818 13303 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:52.038780  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:52.012393 12945 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:52.040459  8205 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:52.040928  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:52.040067  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:52.041615  8209 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:52.038329  8130 raft_consensus.cc:2376] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11. Re-sending same reply.
I20260709 13:14:52.040166  8060 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:52.041777  8203 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:52.044077  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:52.044593  8207 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:52.029660 12546 raft_consensus.cc:3055] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:52.049649  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:52.056113  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:52.056826  8056 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:52.049234  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 49453.7us
I20260709 13:14:52.058205  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:52.004972 13304 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:14:52.062497  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:52.073766 13115 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:52.077551 13115 raft_consensus.cc:697] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:52.078564 13115 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:52.081175  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.087782  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:52.088557  8209 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:52.094463  8207 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:52.049304 13261 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:52.112282  8205 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:52.115833  8059 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:52.060532 13315 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:52.125065  8205 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:14:52.125535  8205 pending_rounds.cc:85] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 5
I20260709 13:14:52.125862  8205 pending_rounds.cc:107] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:14:52.126128  8205 raft_consensus.cc:2889] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:52.050076 13303 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:52.043360 13306 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:14:52.126852  8059 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:52.128573  8209 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 13:14:52.129523  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:52.055935 12546 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:14:52.123538 13294 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:14:52.132089  8204 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:52.135878 12546 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:52.136955 13075 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:52.133931  8056 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:14:52.141623  8205 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:52.077906 13321 consensus_queue.cc:1059] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
W20260709 13:14:52.142738  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 125293us
I20260709 13:14:52.145166  8059 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:52.146704  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:52.175400  8209 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:52.130373 13319 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:52.178500  8134 raft_consensus.cc:2410] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:14:52.177711  8205 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:14:52.178653  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 120555us
I20260709 13:14:52.184300  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:52.196183  8209 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:14:52.198400  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 52603.8us
W20260709 13:14:52.202310  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.204988  8056 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:52.206375 13261 raft_consensus.cc:697] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:52.207966  8133 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 13:14:52.208549  8133 pending_rounds.cc:85] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 6
I20260709 13:14:52.208834  8133 pending_rounds.cc:107] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 13:14:52.209081  8133 raft_consensus.cc:2889] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:52.209751 13261 consensus_queue.cc:237] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.176894 12945 raft_consensus.cc:697] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:52.129266 13315 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.198827 12546 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:52.222510  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:52.233665  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:52.238126  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.233417 13319 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:52.246685 12945 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.248553 13305 raft_consensus.cc:3055] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:52.248929 13319 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:52.249644 13319 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:52.248301 13332 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.243976 13328 consensus_queue.cc:1059] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:52.230746 13329 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:52.255601  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:52.256181  8104 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:52.259033  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:52.290880  8104 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:52.294957  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:52.295601  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:52.296814  8104 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:52.250314 13319 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:52.252255 13315 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:52.255015 13214 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.256161 13328 raft_consensus.cc:3055] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:52.248953 13305 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:52.256328 13074 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:14:52.256910 13329 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:52.290235 13267 raft_consensus.cc:2764] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:14:52.294795 13218 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:52.301173  8104 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:52.255807 13099 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.313335  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:52.316169  8104 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:52.303052 13307 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:52.221237 13303 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:52.308281 13273 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:52.256659 13320 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:14:52.308559 13328 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:52.326999 13074 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:52.344410 13074 raft_consensus.cc:697] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:52.345227 13318 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:14:52.345381 13074 consensus_queue.cc:237] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.346783 13048 raft_consensus.cc:2764] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:14:52.337378 12915 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:52.328195  8207 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:52.314918 13218 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:52.329638 13099 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:52.330080 13075 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:52.309684 13214 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.319439 13315 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:52.332850  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:52.358899  8728 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:52.320434 13329 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:52.250748 13332 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.348266  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:52.350154  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:52.361405  8137 raft_consensus.cc:2368] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 16. Current term is 17.
I20260709 13:14:52.361797  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 2113) took 1142 ms. Trace:
I20260709 13:14:52.362203  8137 rpcz_store.cc:276] 0709 13:14:51.219711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.349887 (+1130176us) service_pool.cc:224] Handling call
0709 13:14:52.361729 (+ 11842us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.327574 13307 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:52.317536 13319 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:52.352018 13305 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.366300 13307 raft_consensus.cc:697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:52.367131 13305 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:52.367523 13307 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.339982 13304 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:52.335084 13230 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:52.373848  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 111914us
I20260709 13:14:52.355065 13218 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:52.391129  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.394372  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:52.396317  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:52.371942 13304 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:52.417948  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 101812us
I20260709 13:14:52.383471 12915 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:52.415691 13304 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:52.427358  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 131847us
I20260709 13:14:52.427981  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:52.383960 13328 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.423338 13341 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:52.435593  8055 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:52.423959 13347 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.426959 12915 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:52.439445  8058 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:14:52.432479 13343 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:52.445866  8728 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:14:52.435823 13278 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:14:52.436539  8693 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:52.435204  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:52.456851 13347 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:52.473459 13347 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:52.426497 13267 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:14:52.475528  8175 leader_election.cc:400] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
I20260709 13:14:52.474665 12546 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.476563 13214 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:52.477370 13230 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:52.477236 13348 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:52.477958 13337 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.492501 13337 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:52.493072 13030 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:52.494788 13337 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:52.495057 12546 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.497155 12546 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:52.498687 13218 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.498881 13329 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.479862 12915 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:52.480887  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:52.481642 13304 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:52.488758 13307 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:52.505522 13307 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:52.505614 13304 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:52.490109  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:52.506616  8136 raft_consensus.cc:2376] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13. Re-sending same reply.
I20260709 13:14:52.507153  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 2110) took 1288 ms. Trace:
I20260709 13:14:52.507434  8136 rpcz_store.cc:276] 0709 13:14:51.218836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.475282 (+1256446us) service_pool.cc:224] Handling call
0709 13:14:52.507089 (+ 31807us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.507988 13230 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.508637 13218 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:52.508495 13230 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:52.487403  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 52775.6us
I20260709 13:14:52.450826 13328 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:52.491508 13348 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:52.516196 13267 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:52.517510 13348 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:52.518473 13304 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:52.519294 13027 raft_consensus.cc:3055] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:52.519773 13027 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:52.520617 13027 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.521819 13027 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:52.505963 13307 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:52.474025 13347 consensus_queue.cc:1059] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:14:52.454893 13332 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:52.514842 13230 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:52.510612 13329 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:52.517982  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 81864.4us
I20260709 13:14:52.528546 13347 raft_consensus.cc:3055] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:52.518093 13348 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:52.529314 13347 raft_consensus.cc:740] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:52.529934  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:52.490406  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 54044us
I20260709 13:14:52.532037  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:14:52.533136  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.535151  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:52.536384  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:52.536973  8205 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:14:52.530440 13347 consensus_queue.cc:260] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:52.538139  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:52.538635 13347 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:52.538739  8206 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:52.539681  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:52.542006  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:52.542587  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:52.542768  8205 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:52.450922 13341 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:52.543540  8104 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:52.555629  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 121274us
I20260709 13:14:52.527060 13304 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:14:52.585446  8205 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:52.590745 13341 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:52.575260 13028 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:52.591728 13028 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:52.592085 13028 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:52.594151 13281 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:52.594504  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.544011 13349 raft_consensus.cc:2764] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:14:52.575114 13332 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:52.598917 13332 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:52.551136  8693 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:52.586987  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 51950.6us
I20260709 13:14:52.544656 13267 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:52.547130 13230 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:52.610460  8104 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:52.649168 13347 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:52.649832 13347 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:52.650861 13347 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:52.647085  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 96040.6us
I20260709 13:14:52.654250  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:52.656044  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:52.656996 13241 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:52.657430 13241 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:52.657874 13241 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:52.663260 13354 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:52.663947 13354 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:52.664976 13354 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:52.675249  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:52.676072  8205 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
W20260709 13:14:52.683434  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 66194.9us
W20260709 13:14:52.686986  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 76643.4us
I20260709 13:14:52.690321  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:52.691431  8205 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:52.690096 13214 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.702391  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:52.702971  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:52.702638  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:52.603606 13332 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:52.714517  8130 raft_consensus.cc:2368] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
I20260709 13:14:52.701833 13214 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:52.714159  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:52.647541 13267 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:52.647786 13230 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:52.755506  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:52.719249 13214 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:52.720794  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:52.713966 13278 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:52.812320  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:52.813184  8137 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:52.755478 13267 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:52.783068 13230 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:52.716557 13074 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 13:14:52.854959  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:52.834633 13278 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:52.930081 13353 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
W20260709 13:14:52.949321  8027 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.68:44573 timed out after 1.822s (SENT) [suppressed 24 similar messages]
W20260709 13:14:53.045046  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 355227us
I20260709 13:14:53.051780  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:53.052595  8727 raft_consensus.cc:2410] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:14:52.753240  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:53.063318  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 310238us
I20260709 13:14:53.065285 13319 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:53.065804 13319 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:53.067296  8212 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.006s	sys 0.018s
W20260709 13:14:53.067767  8212 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.006s	sys 0.018s
I20260709 13:14:53.071779  8137 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:53.022018 13028 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.066223 13319 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:53.064517  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:53.023492 13278 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.086016  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.088744  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.089614  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.089682  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:53.091256  8211 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:53.094834  8175 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:53.096117  8727 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: [11.12-11.12]
W20260709 13:14:53.090092 13356 log.cc:927] Time spent T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 13:14:53.089632 13027 consensus_meta.cc:287] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.567s	user 0.012s	sys 0.000s
W20260709 13:14:53.094722 13307 consensus_meta.cc:287] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.566s	user 0.007s	sys 0.004s
I20260709 13:14:53.100674  8728 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:53.103260  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:53.109138  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1609) took 1307 ms. Trace:
I20260709 13:14:53.121382 13219 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:53.124282 13236 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:53.124830 13236 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:53.125308 13236 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:53.128177 13358 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:14:53.104700  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:53.109647  8132 rpcz_store.cc:276] 0709 13:14:51.801659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.862500 (+ 60841us) service_pool.cc:224] Handling call
0709 13:14:51.870885 (+  8385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:51.871495 (+   610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.108872 (+1237377us) spinlock_profiling.cc:243] Waited 8.13 ms on lock 0x7b540015b150. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:53.109064 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18695424}
I20260709 13:14:53.096781 13216 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:53.130318  8175 leader_election.cc:400] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:14:53.143117  8175 leader_election.cc:403] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:53.150250  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:53.150319  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.152220  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.153543  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.154314  8061 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:53.137687  8102 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:53.158771  8102 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:53.136147  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.138672  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:53.129171 12920 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:53.155797  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.199918  8728 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 16: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:53.142877 13216 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:53.102720 13307 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.124840 13319 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.129294 13241 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.208468 13216 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:53.196256 13364 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:53.196626 13225 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:53.225564 13370 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:53.229694 13216 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:53.231961  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 98496.5us
W20260709 13:14:53.233724  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 103521us
I20260709 13:14:53.199059 13214 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:14:53.247877  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 1998) took 1766 ms. Trace:
I20260709 13:14:53.248232  8211 rpcz_store.cc:276] 0709 13:14:51.481878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.942827 (+460949us) service_pool.cc:224] Handling call
0709 13:14:51.943216 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:53.091695 (+1148479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.247644 (+155949us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540010c054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:53.247825 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2639872256}
I20260709 13:14:53.233966 13347 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:53.240053 13355 log.cc:927] Time spent T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:14:53.230484 13370 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:53.260092 13319 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:53.261237 13307 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:53.262260 13028 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:53.262502  8131 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 2117) took 2040 ms (client timeout 1953 ms). Trace:
I20260709 13:14:53.263679 13241 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:53.264089 13267 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.266505  8730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 743) took 3069 ms. Trace:
W20260709 13:14:53.271605  8751 debug-util.cc:398] Leaking SignalData structure 0x7b08002e1900 after lost signal to thread 6940
I20260709 13:14:53.250272 13230 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.259011 13368 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:53.275099 13354 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:53.276211 13341 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:53.277489  8102 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:53.280210  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:14:53.376480  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 99133.9us
W20260709 13:14:53.286513  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 51643.5us
W20260709 13:14:53.361099  8062 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 776) took 2769 ms (client timeout 1821 ms). Trace:
W20260709 13:14:53.376374  8131 rpcz_store.cc:269] 0709 13:14:51.222183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:53.262285 (+2040102us) service_pool.cc:211] Skipping call since client already timed out
0709 13:14:53.262421 (+   136us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:14:53.374253  8730 rpcz_store.cc:276] 0709 13:14:50.196974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.197205 (+   231us) service_pool.cc:224] Handling call
0709 13:14:50.197636 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:50.834779 (+637143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.266253 (+2431474us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b540060de54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:53.266431 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1445110016}
W20260709 13:14:53.379129  8062 rpcz_store.cc:269] 0709 13:14:50.512695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.517287 (+  4592us) service_pool.cc:224] Handling call
0709 13:14:53.281769 (+2764482us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b540028aad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:53.281866 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75943168}
W20260709 13:14:53.382665  8175 leader_election.cc:336] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.790s (SENT)
I20260709 13:14:53.383685  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:53.384933  8175 leader_election.cc:336] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.807s (SENT)
I20260709 13:14:53.395298 13236 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.397457 13236 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:53.398016 13236 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:53.394141  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:53.403911  8137 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
W20260709 13:14:53.405326 13332 consensus_meta.cc:287] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.623s	user 0.001s	sys 0.009s
I20260709 13:14:53.405489 13216 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:53.405766 13332 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.381088 13214 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:53.378976 13369 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:53.406865  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:53.291698 13370 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:53.384526 13319 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.282323 13347 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.411739 13319 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:53.412814 13338 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.993s	user 0.008s	sys 0.004s
I20260709 13:14:53.413105 13347 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:53.423810  8175 leader_election.cc:336] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.990s (SENT)
I20260709 13:14:53.427793 13319 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:53.409194 13216 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:14:53.433245  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.455542 13214 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:53.456051 13214 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.456696 13364 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.457182 13364 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.458236 13267 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:53.458825 13267 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.459255 13364 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:53.459200 13267 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.460517 13364 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:53.461069 13267 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:53.461125 13364 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.467742 13364 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:53.464215  8101 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:53.465251 13267 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:53.469969  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:53.470111 13307 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:53.470692 13267 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:53.471050  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.471287 13230 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:53.471652 13307 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:53.472200  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.472749  8727 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:53.472641 13307 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.473701  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.470709  8728 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 16: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:53.475183 13236 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.475680 13080 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.475644 13236 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.476356  8102 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:53.476536 13367 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:53.476316 13080 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.477093 13367 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.477882 13028 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:53.478276 13028 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.478639 13028 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:53.479099 13367 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:53.480470 13080 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:53.481454  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.482263  8728 raft_consensus.cc:2410] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:14:53.408879 13369 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:53.483650  8134 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1605) took 1704 ms (client timeout 1902 ms). Trace:
W20260709 13:14:53.484082  8134 rpcz_store.cc:269] 0709 13:14:51.779505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.827541 (+ 48036us) service_pool.cc:224] Handling call
0709 13:14:53.483471 (+1655930us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b540023cdd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:53.483595 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":799263616}
I20260709 13:14:53.485260  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:53.485922  8134 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 15: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:53.486271  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3200) took 1383 ms. Trace:
I20260709 13:14:53.486613  8207 rpcz_store.cc:276] 0709 13:14:52.102813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.103193 (+   380us) service_pool.cc:224] Handling call
0709 13:14:52.103542 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.354179 (+250637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.486090 (+1131911us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54002bf554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:53.486225 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":516217728}
I20260709 13:14:53.488840  8132 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:14:53.379621 13368 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.491353 13218 raft_consensus.cc:3055] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:53.491714 13218 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:53.491787 13368 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:53.492070 13364 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.484009 13369 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.498051  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.498171  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:53.498669  8205 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:53.502132  8209 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:14:53.492573 13218 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.501431  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.500429 13354 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.494571  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:53.518525  8134 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 16: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:53.471613  8730 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:14:53.521206  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:53.525233  8131 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:53.511379 13354 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.492573 13364 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.527447  8206 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:53.496518 13278 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:53.536471  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.536782  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:53.528117  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:53.538580  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.533020  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.528887 13369 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:53.542215  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.542459 13214 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:53.542827  8058 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:53.543094 13214 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:53.544202  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.545297  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.512110 13230 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:14:53.546463 13218 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:53.547492 13028 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.548153 13219 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:53.548420 13236 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:53.548542 13219 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.549018 13219 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:53.549461 13028 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:53.550642  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.551681  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.551900  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.552894  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.553921  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.554636  8724 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:53.555330  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:53.555696 13225 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:53.555933  8211 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:53.556075 13225 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:53.556787  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:53.557466 13354 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:53.557694  8751 debug-util.cc:398] Leaking SignalData structure 0x7b080032ec20 after lost signal to thread 7995
I20260709 13:14:53.558027 13278 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:53.558959  8751 debug-util.cc:398] Leaking SignalData structure 0x7b08000cfb80 after lost signal to thread 8087
I20260709 13:14:53.540635  8728 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
W20260709 13:14:53.559268  8751 debug-util.cc:398] Leaking SignalData structure 0x7b08002b93a0 after lost signal to thread 8088
I20260709 13:14:53.540053  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:53.520335 13216 consensus_queue.cc:1059] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:14:53.561772  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.563098  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.564275  8062 raft_consensus.cc:2376] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12. Re-sending same reply.
I20260709 13:14:53.563148  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:53.565219  8724 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:53.565640  8176 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:53.567317 13216 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:14:53.567747 13363 log.cc:927] Time spent T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.478s	user 0.002s	sys 0.000s
I20260709 13:14:53.568933 13364 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:53.569469  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2154) took 1582 ms. Trace:
I20260709 13:14:53.569818  8138 rpcz_store.cc:276] 0709 13:14:51.986888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.987021 (+   133us) service_pool.cc:224] Handling call
0709 13:14:52.011304 (+ 24283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.019254 (+  7950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.569267 (+1550013us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b54003c91d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:53.569418 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3572608}
I20260709 13:14:53.555943 13332 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:53.564209  8728 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 9.
I20260709 13:14:53.577023  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.578189  8693 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:53.579937  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.582559  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.582661  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.583063 13027 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.583357  8056 raft_consensus.cc:2410] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:14:53.583778  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.584137  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.584297 13323 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:53.584312  8060 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:53.583487 13027 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.585616  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:53.599272  8728 raft_consensus.cc:2376] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16. Re-sending same reply.
I20260709 13:14:53.585999  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.575265 13341 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:53.584852 13323 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.567736 13216 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:53.590773  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.591207  8058 raft_consensus.cc:2368] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:14:53.591255  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:53.585134 13347 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:53.556829 13225 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.541534  8205 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:14:53.592491 13219 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.593497  8730 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:14:53.594884  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.574241 13354 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:53.595850  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:53.597931  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.607466  8056 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:53.607501  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:53.621630  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.601467  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.602010  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.622397  8060 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 11.
I20260709 13:14:53.602602  8209 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:14:53.609120  8724 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:53.625021  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.600674 13341 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.629331  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.631227  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.598806 13369 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:14:53.635124  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.637483  8055 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 16: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:53.638793  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.640203  8056 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:53.640702  8059 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:53.641417  8057 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:14:53.641744  8057 pending_rounds.cc:85] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 11
I20260709 13:14:53.641949  8057 pending_rounds.cc:107] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:14:53.642146  8057 raft_consensus.cc:2889] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:53.648159  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.649461  8060 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:53.650949  8058 raft_consensus.cc:2393] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:53.623627 13347 raft_consensus.cc:697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:53.628456 13236 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:53.624594 12914 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:53.655802  8751 debug-util.cc:398] Leaking SignalData structure 0x7b08002491e0 after lost signal to thread 8162
W20260709 13:14:53.670778  8751 debug-util.cc:398] Leaking SignalData structure 0x7b08002491c0 after lost signal to thread 8163
I20260709 13:14:53.670825 12914 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:53.672003  8751 debug-util.cc:398] Leaking SignalData structure 0x7b08003fbd60 after lost signal to thread 8236
W20260709 13:14:53.672276  8751 debug-util.cc:398] Leaking SignalData structure 0x7b08003ece00 after lost signal to thread 8237
I20260709 13:14:53.672686  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3206) took 1525 ms. Trace:
I20260709 13:14:53.673030  8204 rpcz_store.cc:276] 0709 13:14:52.146990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.147145 (+   155us) service_pool.cc:224] Handling call
0709 13:14:52.183480 (+ 36335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.230801 (+ 47321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:52.230827 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:52.230867 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:52.230894 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.672463 (+1441569us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b54001536d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:53.672507 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.672637 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108373248}
W20260709 13:14:53.673604  8751 debug-util.cc:398] Leaking SignalData structure 0x7b0800383540 after lost signal to thread 8754
I20260709 13:14:53.675815  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:53.676319  8206 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:53.676450  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.678865  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.679407  8061 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:53.667887 13216 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.680768 13216 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:53.626951 13354 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.685202 13354 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:53.690375 13347 consensus_queue.cc:237] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.694646  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:53.695247  8209 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:53.700296  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.701346  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.655862  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.702821  8056 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:53.657073 13370 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.703588  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.704599  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.705283 13370 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:53.705502  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.706074  8056 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:53.637210 13341 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:53.663060  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.721999  8059 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:53.666657  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:53.722995  8212 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 12: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:53.668457  8208 raft_consensus.cc:1217] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Deduplicated request from leader. Original: 8.5->[10.6-14.9]   Dedup: 10.6->[11.7-14.9]
I20260709 13:14:53.668785  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.723942  8057 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:53.669309  8211 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:53.669982  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:53.725467  8061 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:53.726001  8205 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:53.707389 13219 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:53.708570 13027 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:53.643847 13225 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:53.674268  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:53.702303  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.713631  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:53.729431  8208 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15. Ops: [10.6-14.9]
I20260709 13:14:53.730275  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:53.730867  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.731894  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.732569  8060 raft_consensus.cc:2393] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:53.733256  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.734673  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:53.736421  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:53.737725  8205 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:53.738956  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:53.739650  8730 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:53.741871  8211 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 13: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:53.743016  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:53.751585  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:53.752370  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 259105us
I20260709 13:14:53.755751  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.757617  8208 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:53.759167  8209 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:14:53.763833  8206 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:53.761096  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.761610  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:53.765787  8205 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:53.767141  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.768428  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.769088  8060 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:14:53.767479  8061 raft_consensus.cc:2410] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:14:53.774016  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:53.775662  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:53.776839  8205 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 12: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:53.776841  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:53.777841  8059 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:53.779752  8056 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:53.753895  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:53.720885 13218 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.785089  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:53.785972  8204 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:53.805238  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 51710.5us
I20260709 13:14:53.740619 13332 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:53.817278  8163 heartbeater.cc:646] Failed to heartbeat to 127.6.198.126:38543 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.6.198.126:38543 timed out after 15.000s (SENT)
W20260709 13:14:53.842973  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 69049.9us
I20260709 13:14:53.809373  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:53.846159  8205 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:14:53.754819 13376 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:53.754654 13369 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
W20260709 13:14:53.849229  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 221808us
I20260709 13:14:53.805004 13230 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:53.844707 13340 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:14:53.855155  8205 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:14:53.858922  8204 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:53.847807 13376 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.805526 13218 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.851608 13265 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:53.855139 13368 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:53.883466  8204 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:14:53.888404  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 52326.4us
I20260709 13:14:53.841334 13332 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.891192 13332 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:53.904960 13230 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:53.863773 13090 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:14:53.915336 13278 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:53.915822 13278 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:53.916338 13278 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:53.915880  8101 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:53.918489 13219 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:53.918840  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:53.919608 13230 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:53.920006  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:53.920782 13230 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:53.921175  8175 leader_election.cc:400] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
I20260709 13:14:53.921485  8175 leader_election.cc:403] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:14:53.921962 13230 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
I20260709 13:14:53.919039 13219 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:53.926945  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2159) took 1712 ms. Trace:
W20260709 13:14:53.927737  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.881901 13376 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:53.929044  8029 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:53.930302  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:53.933171  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.934026  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:53.940883  8730 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:14:53.927361  8133 rpcz_store.cc:276] 0709 13:14:52.214732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.220059 (+  5327us) service_pool.cc:224] Handling call
0709 13:14:52.220478 (+   419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:52.234999 (+ 14521us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:14:52.290100 (+ 55101us) log.cc:844] Serialized 14 byte log entry
0709 13:14:52.372098 (+ 81998us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:52.385390 (+ 13292us) log.cc:844] Serialized 52 byte log entry
0709 13:14:52.385779 (+   389us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:52.400062 (+ 14283us) log.cc:844] Serialized 14 byte log entry
0709 13:14:52.400540 (+   478us) log.cc:844] Serialized 14 byte log entry
0709 13:14:52.411508 (+ 10968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.411603 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:53.552163 (+1140560us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b18004a58a0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:14:53.706820 (+154657us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b18004a58a0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ef72a6 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1
0709 13:14:53.707426 (+   606us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b18004a58a0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:14:53.926736 (+219310us) raft_consensus.cc:1685] finished
0709 13:14:53.926757 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.926882 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21265,"prepare.queue_time_us":57863,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":1401,"raft.queue_time_us":33662,"raft.run_cpu_time_us":398,"raft.run_wall_time_us":6303,"spinlock_wait_cycles":15260160,"thread_start_us":59639,"threads_started":1,"wal-append.queue_time_us":105821}
I20260709 13:14:53.945405  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:53.945606  8724 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: [13.10-17.12]
I20260709 13:14:53.947433  8057 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:14:53.948091  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:53.950443  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:53.952322  8203 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:53.952694  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:53.955495  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:53.956468  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:53.895819 13370 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:53.961445  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:53.918223 13278 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:53.902884 13332 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:53.929147 13219 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:53.981297  8058 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:53.983376  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.983971  8730 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:53.983924  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:53.989666  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:53.990796  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:53.958202 13213 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:53.957621 13195 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:53.896162 13368 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:54.004603  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 88763.8us
I20260709 13:14:53.973006 13218 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:53.958110 13319 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:53.957464 13369 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:53.996583  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:53.959915 13367 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:54.051713  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.071143  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:53.960593 12920 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:53.960453 13384 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.071842  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.062505  8102 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:54.065820  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:53.972224 13080 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:54.085753  8129 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:14:54.090109  8130 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 9.
I20260709 13:14:54.090797  8132 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:14:54.092535  8133 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:54.093700  8135 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
I20260709 13:14:53.967489 13216 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:54.095265  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:54.098062  8138 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:54.070945 13030 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:54.042282 13219 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.999221 13265 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:54.099370  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.098033  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.125167  8062 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:54.127193  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:54.127776  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:54.129220  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:54.129441  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:54.129688  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:54.130033  8205 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:54.130899  8208 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:14:54.130927  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:54.131183  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:54.131872  8206 raft_consensus.cc:2393] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 13: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:14:54.104516  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 113791us
I20260709 13:14:54.105420  8131 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:14:54.105495  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.043922 13386 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:14:54.094409 12920 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:54.106573 13219 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:54.107049  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:54.151985  8130 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:54.107276 13258 log.cc:927] Time spent T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 3.282s	user 0.021s	sys 0.023s
I20260709 13:14:54.107712  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:54.111351 13374 log.cc:927] Time spent T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.540s	user 0.002s	sys 0.000s
I20260709 13:14:54.113933  8129 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:54.115314 13236 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:54.099440  8102 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:54.116313 13388 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:14:54.116799 13364 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:54.118594 13234 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:54.120585 13214 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:54.120906  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:54.122674 13395 consensus_queue.cc:1059] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:14:54.130650  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:54.043483 13278 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:53.967234 13341 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:54.125808  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:54.134501  8057 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:54.135587  8207 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:14:54.128505  8210 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:54.137040  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:54.003679 13332 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:54.153694  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:54.155485  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:54.200632  8134 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:54.154743  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:54.157033  8055 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:14:54.103751 13030 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:54.159997  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:54.080514 13195 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:54.217113  8132 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:54.163313  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.165319  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:54.167282  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 67926.9us
I20260709 13:14:54.261214  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.268869  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:54.269112  8131 raft_consensus.cc:2368] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18.
I20260709 13:14:54.166826  8061 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:14:54.152138 12920 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:54.262646  8135 raft_consensus.cc:2410] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:14:54.272114  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 139195us
I20260709 13:14:54.268080  8133 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:54.277662  8059 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:14:54.288946  8029 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:54.154706 13290 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:54.325086  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 204248us
I20260709 13:14:54.155004 13080 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:54.154804 13236 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:54.152901 13402 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:54.160938 13216 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:54.271529 13392 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:14:54.337528  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 68734.7us
I20260709 13:14:54.337970  8057 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:54.359019  8057 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:54.335611  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:54.338517  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 342648us
I20260709 13:14:54.338953  8104 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:54.102514 13384 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:54.161140 13364 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:54.261198 13030 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:54.337100  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:54.376444  8131 raft_consensus.cc:2368] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:14:54.318331 13400 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:54.351305 13388 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:14:54.360144  8133 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:54.405292  8207 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:54.392819  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:54.164415 13265 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:54.382700  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 229126us
W20260709 13:14:54.411046  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.375511 13230 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.202149 13337 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.149s
I20260709 13:14:54.262111 13347 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:54.342283 13258 log.cc:927] Time spent T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
W20260709 13:14:54.383486  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44598.2us
I20260709 13:14:54.438889  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:54.359208  8102 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:54.352195 13225 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:54.266551 13234 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:54.269439 13278 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:54.441111  8104 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:14:54.448916  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 89823.3us
I20260709 13:14:54.409176  8211 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:14:54.409165  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:54.345860 13216 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:14:54.413054  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.409461 13392 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:54.448534 13395 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:54.408342 13400 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:54.384563 13403 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 13:14:54.382387 13364 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:54.415115  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:54.343505 13401 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:54.383807 13080 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:54.447964  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 159143us
I20260709 13:14:54.413476 13290 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:14:54.360980  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:54.455953  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 373975us
I20260709 13:14:54.427596 13230 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:54.455814  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.373713  8134 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:54.461208  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:54.535974  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:54.475930  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:54.476890  8104 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:54.536219  8721 raft_consensus.cc:1813] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Handling vote request from an unknown peer d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:54.538472  8721 raft_consensus.cc:2368] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
I20260709 13:14:54.538647  8029 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:54.544191  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.545038  8131 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:54.546168  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.556448  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.556540  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:54.475924 13376 consensus_meta.cc:287] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.000s
I20260709 13:14:54.558643  8136 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:54.563990  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.558302  8133 raft_consensus.cc:2376] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11. Re-sending same reply.
I20260709 13:14:54.456696 13216 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:54.555965  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.567085  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.468040 13398 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:54.467971 13332 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:54.459641 13370 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:54.467687 13234 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:54.574386  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:54.456164 13394 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:54.458534 13406 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:54.577952  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.572624 13214 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:54.571209 13199 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.460034 13400 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:54.581722 13290 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:54.449023 13347 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:54.621626  8131 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:54.570465  8133 raft_consensus.cc:2376] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13. Re-sending same reply.
I20260709 13:14:54.574409 13398 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:54.624789  8136 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:14:54.573077  8134 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:54.585821 13370 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:54.550997 13080 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:54.683727 13390 log.cc:927] Time spent T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.463s	user 0.000s	sys 0.001s
W20260709 13:14:54.685592 13399 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.243s	user 0.000s	sys 0.000s
I20260709 13:14:54.686514 13403 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:54.624488 13394 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:54.686740 13391 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:54.687067 13400 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:54.571832 13376 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:54.686187  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 111929us
I20260709 13:14:54.722508  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:54.693380  8132 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
W20260709 13:14:54.616241  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 48418.8us
I20260709 13:14:54.697710  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.702543  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:54.704401  8134 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:14:54.727103  8134 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:14:54.706609  8176 leader_election.cc:400] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
W20260709 13:14:54.709754  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 232912us
W20260709 13:14:54.703461 13319 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.001s	sys 0.031s
W20260709 13:14:54.720798  8130 consensus_meta.cc:287] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.568s	user 0.008s	sys 0.000s
I20260709 13:14:54.729605  8130 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:54.689735 13341 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:54.728323  8176 leader_election.cc:403] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:54.701543 13398 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:54.701052 13195 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:54.745743  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:54.681610 13199 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:54.748839  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:54.711659 13225 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:54.756398  8063 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:14:54.760668 13393 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.219s
I20260709 13:14:54.761287 13393 consensus_queue.cc:1243] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:14:54.741331 13319 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.001s	sys 0.031s
I20260709 13:14:54.762529  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:54.710925 13394 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:54.766801 13278 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:54.689946 13402 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:14:54.739768 13368 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:54.768438 13368 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.768949 13368 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:54.762532 13319 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:54.772720 13319 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:54.776428  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 52464.6us
I20260709 13:14:54.729310 13391 raft_consensus.cc:697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:54.786410  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 63979.2us
I20260709 13:14:54.792407  8729 raft_consensus.cc:1813] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Handling vote request from an unknown peer d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:54.777549  8027 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:54.793196  8729 raft_consensus.cc:2410] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:14:54.778764  8138 consensus_meta.cc:287] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.678s	user 0.008s	sys 0.000s
I20260709 13:14:54.671403 13406 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:54.802843  8138 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:14:54.816502  8137 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:54.732259 13400 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:54.826273 13278 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.827399 13214 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:54.827853 13214 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:54.742465 13401 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:14:54.826916 13278 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:54.673379 13392 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:54.843470  8063 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:14:54.845995 13216 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:54.852495 13404 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:14:54.815632 13370 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:54.855284 13370 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:54.855569 13384 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:54.855996 13370 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:14:54.858191 13396 log.cc:927] Time spent T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.786s	user 0.000s	sys 0.006s
I20260709 13:14:54.846364 13199 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:54.871709  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 721813us
I20260709 13:14:54.880081  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:54.883352 13080 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:54.760682 13290 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:54.853341 13216 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:14:54.903124  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 115760us
I20260709 13:14:54.905484  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:54.908231  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 130784us
I20260709 13:14:54.850449 12914 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:54.850629 13230 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:54.870990 13307 raft_consensus.cc:3055] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:54.927611  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:54.929018 13395 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:54.929327 13393 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:54.929647 13028 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.930663  8101 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:54.975672  8138 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:54.976935  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 46335.4us
W20260709 13:14:54.978281  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 51634.1us
I20260709 13:14:54.995870  8724 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:14:54.933245 13403 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:54.901427 13427 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
W20260709 13:14:54.987733  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 82358.6us
I20260709 13:14:54.995043  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:54.924925 13402 consensus_queue.cc:1059] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
W20260709 13:14:54.993366  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 254269us
I20260709 13:14:55.006388  8696 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:14:54.971791  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 283318us
I20260709 13:14:54.880538 13225 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:54.932984 13307 raft_consensus.cc:740] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:55.007853 13218 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:55.008540 13267 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:55.009095 13307 consensus_queue.cc:260] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.009130  8101 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:54.990634 13364 consensus_meta.cc:287] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.525s	user 0.004s	sys 0.006s
I20260709 13:14:54.985642 13376 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:54.989706 12914 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:54.992828 13265 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:14:55.033799  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 230159us
I20260709 13:14:55.027469  8138 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:14:55.048797  8060 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:14:55.049252  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:14:55.011854  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.018949 13394 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.050457  8136 raft_consensus.cc:1813] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Handling vote request from an unknown peer d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:55.050995  8136 raft_consensus.cc:2368] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 14.
I20260709 13:14:55.051761 13394 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:55.019486 13274 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:55.017174  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:54.981608 13028 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.004740 13347 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:54.978435 13393 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.011891 13307 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:55.029137 13402 consensus_queue.cc:1243] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:14:55.081138 13347 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:55.073333  8059 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:55.082180  8696 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:55.085096  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:55.033782 13403 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:55.091977  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:55.145530  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:55.146284  8138 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
W20260709 13:14:55.091440  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 76491.7us
W20260709 13:14:55.122234  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 182392us
I20260709 13:14:55.148118  8029 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:55.148523  8175 leader_election.cc:400] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18.
I20260709 13:14:55.149432  8175 leader_election.cc:400] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:14:55.029942  8730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:50008
I20260709 13:14:54.908031 13391 consensus_queue.cc:237] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.072708 13401 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.033304 13364 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.129283  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:55.132268 13423 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.072s
I20260709 13:14:55.132586 13387 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.187350 13364 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:55.134121 13369 consensus_meta.cc:287] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 1.047s	user 0.007s	sys 0.004s
I20260709 13:14:55.187988 13369 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:55.188117 13444 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:55.135933 13370 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
I20260709 13:14:55.188714 13444 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:55.137653 13410 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:14:55.189095 13444 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:55.189677 13457 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:55.190150 13457 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:55.098627 13347 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:55.170738 13427 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:55.190922 13347 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:55.191263  8209 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:55.192058 13441 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.163s
I20260709 13:14:55.193105 13441 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:55.193418 13441 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:55.194561 13415 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:55.195292 13412 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:55.149807  8175 leader_election.cc:403] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 13:14:55.144063  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 63282us
W20260709 13:14:55.125806 13367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.020s	user 0.011s	sys 0.004s
I20260709 13:14:55.157044 13323 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.157191 13417 consensus_queue.cc:1059] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:14:55.157461 13393 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.161888 13219 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:55.129874 13400 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:55.162850  8132 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15. Ops: []
W20260709 13:14:55.163118 13450 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:14:55.163654 13406 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:55.164693  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:55.167339 13028 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.127058 13267 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.233451 13267 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.222663 13261 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:14:55.129938 12920 consensus_meta.cc:287] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.777s	user 0.009s	sys 0.001s
I20260709 13:14:55.172279  8138 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:55.176354  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:55.186092 13461 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:55.210196 13370 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
W20260709 13:14:55.135248 13236 consensus_meta.cc:287] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.771s	user 0.008s	sys 0.001s
I20260709 13:14:55.246196 13236 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.243938 13461 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:55.248198 13461 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:55.248488 13236 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:55.249080 13236 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:55.249519 13236 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:55.189774 13030 consensus_meta.cc:287] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.781s	user 0.006s	sys 0.003s
I20260709 13:14:55.250197 13369 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.250559 13030 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.250752 13369 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:55.252060 13389 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.796s	user 0.001s	sys 0.001s
I20260709 13:14:55.252687 13030 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.253043 13369 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:55.253561 13369 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:55.253700  8210 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 17: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:55.059902 13395 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.254388 13376 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.254901 13376 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.256937  8133 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:55.258379  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:55.258744 13456 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:55.259114  8206 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:14:55.259886  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1628) took 1124 ms. Trace:
I20260709 13:14:55.219151 13218 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:55.202348  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 53067.9us
I20260709 13:14:55.177191 13278 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:55.223755  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2174) took 1113 ms. Trace:
W20260709 13:14:55.205197 13367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.100s	user 0.012s	sys 0.005s
I20260709 13:14:55.261984 13367 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:55.235405 13430 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:55.225477  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.150794 13436 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.128s
I20260709 13:14:55.187043 13387 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:55.245183  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 68939.8us
I20260709 13:14:55.256983 13376 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:55.152279  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 67651.5us
I20260709 13:14:55.260514  8204 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:55.260730 13218 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.237520 13406 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:55.260933  8132 rpcz_store.cc:276] 0709 13:14:54.110079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.723868 (+613789us) service_pool.cc:224] Handling call
0709 13:14:54.724202 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:55.162675 (+438473us) spinlock_profiling.cc:243] Waited 53.9 ms on lock 0x7b5400380f20. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf8ddc0 00007f6dbb0181c0 00007f6dbb019231 00007f6dbb015d8f 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c
0709 13:14:55.223415 (+ 60740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.223671 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123992448}
I20260709 13:14:55.263684  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:55.216248  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 166552us
I20260709 13:14:55.260279  8133 rpcz_store.cc:276] 0709 13:14:54.135976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.731669 (+595693us) service_pool.cc:224] Handling call
0709 13:14:54.731997 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:55.257852 (+525855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.259521 (+  1669us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b54001f1dd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:55.259820 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1206778112}
I20260709 13:14:55.266542  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:55.268311  8205 raft_consensus.cc:1240] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:55.212076 13364 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:14:55.261200  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:55.235580 13267 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:55.274489  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:55.312332  8133 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:14:55.313768  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:55.302057  8208 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 10.
I20260709 13:14:55.318042  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:55.277380  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3080) took 1129 ms. Trace:
I20260709 13:14:55.279314  8696 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:55.272586  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:14:55.273940  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 219468us
I20260709 13:14:55.263054 13430 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:55.311144  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 50008.3us
I20260709 13:14:55.321228  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:55.319803  8062 rpcz_store.cc:276] 0709 13:14:54.148445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.267475 (+119030us) service_pool.cc:224] Handling call
0709 13:14:54.275701 (+  8226us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:54.537742 (+262041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.537783 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:54.561914 (+ 24131us) log.cc:844] Serialized 26 byte log entry
0709 13:14:54.935879 (+373965us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:55.276318 (+340439us) log.cc:844] Serialized 14 byte log entry
0709 13:14:55.276875 (+   557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.276955 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.277062 (+   107us) raft_consensus.cc:1685] finished
0709 13:14:55.277077 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.277329 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33979,"prepare.queue_time_us":239604,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":85,"raft.queue_time_us":132988,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":36982,"thread_start_us":828960,"threads_started":3,"wal-append.queue_time_us":672113}
I20260709 13:14:55.316526  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:55.281733  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:55.273728 13030 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.229035 13214 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.327556  8204 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 13: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:55.382783  8138 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:55.383199  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:55.181293 13401 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.322539  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:55.254050 13369 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:55.384181  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 61740.7us
W20260709 13:14:55.323575  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44380.9us
I20260709 13:14:55.384994  8175 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:55.385283 13401 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:55.385917 13216 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:55.263013 13367 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:55.372018  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 47818.1us
I20260709 13:14:55.323866 13370 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:14:55.378854  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 106355us
I20260709 13:14:55.371549 13079 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:55.329560  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:55.194190 13441 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.385310 13369 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:55.317071 13224 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:55.243752 12920 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.391706  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:55.404052 13198 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.335s	user 0.009s	sys 0.046s
I20260709 13:14:55.265413 13398 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:55.377931  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 59563.3us
I20260709 13:14:55.400978  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1624) took 1303 ms. Trace:
I20260709 13:14:55.420696  8135 rpcz_store.cc:276] 0709 13:14:54.097434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.097830 (+   396us) service_pool.cc:224] Handling call
0709 13:14:55.400801 (+1302971us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b540036e554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:55.400927 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317743488}
I20260709 13:14:55.237803 13403 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:55.379458  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 49704.2us
I20260709 13:14:55.402680  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:55.431046  8205 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:55.431629  8026 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:55.432220  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:55.400852  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:55.433588 13463 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:55.434188 13463 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:55.312237 13406 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.434799 13463 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:55.434799 13422 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:55.435473  8204 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:55.436116 13437 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:55.436647 13437 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 14 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:55.412104  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:55.393635  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:55.437206 13406 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:55.393419 13402 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:55.418577  8133 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 11: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:14:55.438478 13438 log.cc:927] Time spent T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:14:55.439350  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:55.439549 13440 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:55.440177 13440 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.440775  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:55.441766  8133 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:55.421942  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:55.326830 13423 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:55.409212 13265 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:55.423542  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.433732 13431 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.434218 13429 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.437507 13437 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.407657 13079 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:55.417590 13198 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.349s	user 0.010s	sys 0.046s
I20260709 13:14:55.470067 13079 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:55.470369 13376 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.470521  8721 raft_consensus.cc:3055] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:55.471025  8721 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:55.470935 13376 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:55.471974 13425 log.cc:927] Time spent T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:14:55.471894  8721 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.472856  8721 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:55.399669 13332 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:14:55.393162 13030 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.438459  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:55.417605 13369 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.450176  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:55.408478  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 78976us
I20260709 13:14:55.455031 13219 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:55.434969 13435 consensus_queue.cc:1059] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.070s
W20260709 13:14:55.457718 13460 log.cc:927] Time spent T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:14:55.458767 13454 log.cc:927] Time spent T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:14:55.458359 13265 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.433943  8104 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:55.464706  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:55.514672  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:55.515456  8132 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:14:55.415354 13436 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:55.468824  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:55.475654  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 89114.1us
I20260709 13:14:55.466059  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:55.438468 13402 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:55.412631 13456 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:14:55.490866 13332 raft_consensus.cc:3055] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:14:55.494577  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 82550.7us
I20260709 13:14:55.435663 13422 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:55.502087  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 51984.8us
I20260709 13:14:55.467671 13384 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:55.522480  8057 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.516391  8721 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:14:55.504536 13219 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:55.520602  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 86763.5us
I20260709 13:14:55.516431  8135 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:14:55.517901  8133 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:14:55.562249  8057 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:55.560772  8029 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:55.503311 13406 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:55.579406  8175 leader_election.cc:336] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.534s (SENT)
W20260709 13:14:55.548117  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 116543us
I20260709 13:14:55.439374 13467 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.125s
I20260709 13:14:55.507190 13440 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:55.562022 13384 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:55.570109  8104 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:55.500298 13369 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.522851 13393 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:55.520120 13236 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:55.523607 13402 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:55.607740  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:55.608295  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 47638.9us
I20260709 13:14:55.609417  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:55.603729  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.641882  8132 raft_consensus.cc:2410] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:14:55.518527 13347 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.546137 13332 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:55.581736  8175 leader_election.cc:336] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.580s (SENT)
I20260709 13:14:55.584097 13219 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:55.475033 13431 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.665196  8205 raft_consensus.cc:1217] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Deduplicated request from leader. Original: 8.5->[10.6-15.9]   Dedup: 10.6->[11.7-15.9]
I20260709 13:14:55.665733  8205 pending_rounds.cc:85] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 6
I20260709 13:14:55.612041 13364 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:55.564162 13479 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:55.641901  8135 raft_consensus.cc:2368] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:14:55.674372  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:55.675256 12920 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.676316  8730 raft_consensus.cc:606] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:55.676694  8730 raft_consensus.cc:614] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:55.677420 13347 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:55.677271 13466 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.224s
I20260709 13:14:55.679021 13481 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:55.679469 13481 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:55.679800 13481 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:55.680495  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 177495us
W20260709 13:14:55.680706 13469 log.cc:927] Time spent T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:14:55.636111 13218 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.475162 13429 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:55.602787 13420 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 13:14:55.645464  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 63773.9us
W20260709 13:14:55.645712  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 62134.1us
I20260709 13:14:55.607478 13384 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:55.665970  8205 pending_rounds.cc:107] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:14:55.753216  8205 raft_consensus.cc:2889] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:55.549971 13412 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:55.642097  8057 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:14:55.757961  8057 pending_rounds.cc:85] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 8
I20260709 13:14:55.758278  8057 pending_rounds.cc:107] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:14:55.758566  8057 raft_consensus.cc:2889] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:55.682533  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:55.637584 13214 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.686902  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:55.687804  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:55.689404  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:55.761535  8724 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 11.
I20260709 13:14:55.635777 13387 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.740768  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:55.745260 13430 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.764178  8725 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 10: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:55.764554  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3079) took 1611 ms. Trace:
I20260709 13:14:55.642838 13477 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:55.671303 13236 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:55.673514 13364 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:55.762601  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 79220.8us
I20260709 13:14:55.762818  8132 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:55.746982  8175 leader_election.cc:336] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.654s (SENT)
I20260709 13:14:55.765021  8057 rpcz_store.cc:276] 0709 13:14:54.148141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.265276 (+117135us) service_pool.cc:224] Handling call
0709 13:14:54.337831 (+ 72555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:55.522113 (+1184282us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b4c0008f2c0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db92e2a20 00007f6db92f3a29 00007f6db92b9644 00007f6db92e0f2d 00007f6db92e0b93 00007f6dbb01118b 00007f6dbb011cb5 00007f6dbb00cf30 00007f6dbb00e7de 00007f6dbb018b49 00007f6dbb019416
0709 13:14:55.759131 (+237018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.759370 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64250112}
W20260709 13:14:55.760159 13471 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
I20260709 13:14:55.761279 13412 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:55.773866  8175 leader_election.cc:336] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.757s (SENT)
W20260709 13:14:55.766460 12914 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:14:55.801622  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:55.802254  8726 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:55.798995  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:55.759979 13199 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:55.804414  8175 leader_election.cc:336] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.760s (SENT)
I20260709 13:14:55.805521  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:55.796921  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:55.774895 13364 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:55.789315 13398 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.790175 13466 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:14:55.797683 13436 raft_consensus.cc:697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 14 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:55.807765 13370 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:55.772058 13079 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.819576 13447 raft_consensus.cc:3055] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:55.820041 13486 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
W20260709 13:14:55.821170 13475 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:14:55.808962 13467 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:55.834472 13030 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:55.828291 13265 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.774544 12920 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:14:55.770643 13477 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.798153 13387 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:14:55.813462 13456 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.807926 13216 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:55.836755 13472 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 13:14:55.838162 13079 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.838856 13431 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.799940 13332 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.842137  8132 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:14:55.870194  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:55.870637 13332 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:55.871248  8132 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:55.872117 13477 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:55.872467  8132 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:55.872614 13387 consensus_queue.cc:1243] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:55.810940 13199 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:55.846680 13482 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:14:55.835654 13447 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:55.849285 13467 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:55.821864 13474 log.cc:927] Time spent T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 13:14:55.851259  8723 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 14 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:55.876765 13447 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.852490 13481 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.855600 13487 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:55.797616 13422 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.851032  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:55.776393 13236 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.851974 13219 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.884438  8722 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:14:55.890944 13219 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:55.883927 13493 consensus_queue.cc:1059] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:14:55.892992 13493 consensus_queue.cc:1243] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:14:55.893499  8206 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:14:55.894953 13481 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:55.895526 13481 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:55.895874 13481 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:55.896008  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:55.896723 13490 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:55.897022 13236 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:55.898525 13487 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.900677 13487 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:55.901278 13461 consensus_meta.cc:287] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.653s	user 0.007s	sys 0.003s
I20260709 13:14:55.901816 13461 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.902397 13199 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:55.902849 13429 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:55.887619 13307 consensus_meta.cc:287] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.795s	user 0.009s	sys 0.007s
W20260709 13:14:55.903455 13444 consensus_meta.cc:287] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.714s	user 0.010s	sys 0.000s
I20260709 13:14:55.903851 13444 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.905004  8727 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 17: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:55.905064 13467 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.905860 13444 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.873126 13456 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.906970 13216 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.924535  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:55.925303  8728 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:55.926124  8135 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:14:55.926450  8133 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:55.933094 13406 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:55.931701 13481 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.878010 13370 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.938696 13456 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:55.927261 13436 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:55.945787 13461 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.945533 13422 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:55.960490 13481 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:55.975387  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:55.978667  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:55.978776  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:55.979302  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:55.979533  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3078) took 1831 ms. Trace:
I20260709 13:14:55.979928  8056 rpcz_store.cc:276] 0709 13:14:54.147858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.264727 (+116869us) service_pool.cc:224] Handling call
0709 13:14:54.264935 (+   208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:55.158924 (+893989us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b5400268520. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf8ddc0 00007f6dbb0181c0 00007f6dbb019231 00007f6dbb015d8f 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c
0709 13:14:55.159121 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:14:55.197519 (+ 38398us) log.cc:844] Serialized 14 byte log entry
0709 13:14:55.411797 (+214278us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:55.411843 (+    46us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:55.411887 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.979343 (+567456us) spinlock_profiling.cc:243] Waited 79.5 ms on lock 0x7b540028b9d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:55.979384 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.979485 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38321,"prepare.queue_time_us":67836,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":20468,"raft.queue_time_us":10572,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":8709,"spinlock_wait_cycles":261688576,"thread_start_us":203681,"threads_started":1,"wal-append.queue_time_us":297044}
I20260709 13:14:55.980991  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:55.981271 13195 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:55.981698 13195 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:55.982157  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1633) took 1706 ms. Trace:
I20260709 13:14:55.982542  8138 rpcz_store.cc:276] 0709 13:14:54.276075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.264572 (+988497us) service_pool.cc:224] Handling call
0709 13:14:55.264850 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:55.383276 (+118426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.981955 (+598679us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b540023cdd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:55.982108 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270689024}
I20260709 13:14:55.982960  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:55.983811  8061 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 16: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:55.983641  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:55.984174  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1639) took 1434 ms. Trace:
I20260709 13:14:55.984412  8059 raft_consensus.cc:2368] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12.
I20260709 13:14:55.984508  8136 rpcz_store.cc:276] 0709 13:14:54.549668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.697490 (+147822us) service_pool.cc:224] Handling call
0709 13:14:55.983994 (+1286504us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b5400042ed4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:55.984124 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":751385856}
I20260709 13:14:55.985175  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:55.985965 13497 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:55.986498 13497 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:55.987046  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:55.987551  8208 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:55.988483 13497 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:55.989524  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:55.990188  8055 raft_consensus.cc:2393] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:55.990882  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:55.994864  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55942 (request call id 902) took 1603 ms. Trace:
I20260709 13:14:55.995239  8210 rpcz_store.cc:276] 0709 13:14:54.391247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.392518 (+  1271us) service_pool.cc:224] Handling call
0709 13:14:55.994649 (+1602131us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b54002beb54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:55.994786 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1945723648}
I20260709 13:14:55.996411  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:55.997133  8210 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:55.997859  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:56.003376  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:56.004024  8204 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:56.004858  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:56.005373  8204 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
W20260709 13:14:56.006691  8131 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1637) took 1639 ms (client timeout 1644 ms). Trace:
W20260709 13:14:56.007067  8131 rpcz_store.cc:269] 0709 13:14:54.367485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.377105 (+  9620us) service_pool.cc:224] Handling call
0709 13:14:56.006539 (+1629434us) spinlock_profiling.cc:243] Waited 93.6 ms on lock 0x7b540008cd54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:56.006640 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215206784}
I20260709 13:14:56.008715  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:56.009255  8211 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:56.009976  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:56.010598  8134 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:56.011579  8130 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:56.012228 13368 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:56.013495  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2180) took 1901 ms. Trace:
I20260709 13:14:55.949427 13230 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:56.014079 13230 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.016248 13230 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:56.018772 13462 log.cc:927] Time spent T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.603s	user 0.003s	sys 0.000s
W20260709 13:14:56.020535 13468 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.571s	user 0.001s	sys 0.000s
I20260709 13:14:56.021648  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3065) took 1896 ms. Trace:
W20260709 13:14:55.961982 13463 consensus_meta.cc:287] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.000s
I20260709 13:14:56.022482 13463 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:56.023037 13376 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:56.024585 13463 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:56.021983  8058 rpcz_store.cc:276] 0709 13:14:54.125306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.126091 (+   785us) service_pool.cc:224] Handling call
0709 13:14:54.126478 (+   387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:55.164654 (+1038176us) spinlock_profiling.cc:243] Waited 6.64 ms on lock 0x7b18004469e0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbb00e060 00007f6dbb00ebfb 00007f6dbb01964b
0709 13:14:55.164820 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:55.164857 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:55.165476 (+   619us) log.cc:844] Serialized 52 byte log entry
0709 13:14:55.220323 (+ 54847us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:55.220401 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.220552 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:56.020021 (+799469us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:56.021442 (+  1421us) raft_consensus.cc:1685] finished
0709 13:14:56.021466 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:56.021594 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28847,"raft.queue_time_us":116220,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":64563,"spinlock_wait_cycles":26511104,"thread_start_us":243918,"threads_started":2,"wal-append.queue_time_us":228739}
I20260709 13:14:56.026784  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:56.027889  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.028878  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:56.029551  8058 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:56.030776  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.031353  8059 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:14:56.032142  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:56.032299 13384 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:56.032867  8206 raft_consensus.cc:2393] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 12: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:56.034439 13457 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.035471  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:56.035787 13457 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:56.036002  8056 raft_consensus.cc:2376] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16. Re-sending same reply.
I20260709 13:14:56.036856  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.037592  8056 raft_consensus.cc:2410] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:14:56.038890  8056 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:56.039851  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:56.042522 13402 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.044010 13422 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.045421 13402 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:56.048341  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:56.050448  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:55.976805  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:55.962503 13467 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.990875  8729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 761) took 1249 ms. Trace:
I20260709 13:14:56.053248  8029 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:56.057818  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.072984  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:56.073223  8176 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:56.075093  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.075835  8054 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:14:56.076717  8138 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 13:14:56.078032  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.066512  8211 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:56.030269  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.013856  8130 rpcz_store.cc:276] 0709 13:14:54.111579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.730222 (+618643us) service_pool.cc:224] Handling call
0709 13:14:56.011424 (+1281202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.011979 (+   555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.013329 (+  1350us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400042ed0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:56.013446 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2944917888}
I20260709 13:14:56.032142  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.079176  8728 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:56.063308  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:56.068094  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:56.067565  8210 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:56.083349  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:56.083896  8210 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:56.068645  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.070175  8729 rpcz_store.cc:276] 0709 13:14:54.741077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.741211 (+   134us) service_pool.cc:224] Handling call
0709 13:14:55.990659 (+1249448us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b5400694e54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:55.990798 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67398912}
I20260709 13:14:56.006146  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:56.085074  8204 raft_consensus.cc:2393] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 16: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:56.046079 13422 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:56.087750  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:56.088279  8211 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:14:56.090648  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:56.091411  8132 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 10: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:14:56.092253 13459 log.cc:927] Time spent T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.923s	user 0.001s	sys 0.000s
I20260709 13:14:56.092197  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:56.093051  8129 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 12: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:56.093889 13412 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.095027  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2167) took 1990 ms. Trace:
I20260709 13:14:56.095413  8137 rpcz_store.cc:276] 0709 13:14:54.104316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.105872 (+  1556us) service_pool.cc:224] Handling call
0709 13:14:54.803446 (+697574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:54.817415 (+ 13969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.094860 (+1277445us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b54003c87d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:56.094976 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1598477568}
I20260709 13:14:55.978215 13323 raft_consensus.cc:3055] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:56.034962  8206 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:56.086761  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:56.078063 13420 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:55.987358  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:56.102484  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:56.135545  8135 raft_consensus.cc:2368] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12.
I20260709 13:14:56.103565  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.067243  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:56.103899  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 50729.7us
W20260709 13:14:56.107237  8209 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 3240) took 1976 ms (client timeout 1851 ms). Trace:
I20260709 13:14:56.115442  8729 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:14:56.108973  8721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 944) took 1320 ms. Trace:
I20260709 13:14:56.118917  8726 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:56.122850  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:56.125260  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:56.148837  8130 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:56.149693  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:56.125494  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.125824  8132 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:14:56.153477  8130 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:14:56.155601  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:56.109334  8207 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:14:56.126065  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:56.139760  8208 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:56.131494  8061 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:56.102576  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:56.158030  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:56.158692  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:56.158998  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:56.159328  8208 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:56.160038  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:56.158648  8206 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:56.161917  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.125900  8136 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:56.126472  8131 raft_consensus.cc:2393] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:56.164503  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:56.126253  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.154799  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:56.114722  8102 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:56.143544  8209 rpcz_store.cc:269] 0709 13:14:54.130873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.130978 (+   105us) service_pool.cc:224] Handling call
0709 13:14:56.107036 (+1976058us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54002be654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:56.107145 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2436152704}
I20260709 13:14:56.145874  8721 rpcz_store.cc:276] 0709 13:14:54.788443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.788567 (+   124us) service_pool.cc:224] Handling call
0709 13:14:54.788846 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:55.562770 (+773924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.108763 (+545993us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b5400653954. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:56.108921 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1567392512}
W20260709 13:14:56.170800  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 97684.6us
I20260709 13:14:56.131134  8058 raft_consensus.cc:2368] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16.
I20260709 13:14:56.172930  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:56.175925  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.176803  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:56.177338  8056 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:56.178138  8029 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:56.178429  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.180990  8176 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:56.183513  8061 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 12: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:14:56.184204  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 69547us
I20260709 13:14:56.184904  8211 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:56.186120  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:56.186702  8211 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:56.187484  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:56.188556  8130 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:56.189010 13367 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:56.190260  8130 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:14:56.190696  8725 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:56.190881  8724 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:56.192721  8102 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:56.193221  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:56.197957  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:56.198529  8135 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:56.199894  8102 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:56.200300  8060 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.061s	sys 0.110s
I20260709 13:14:56.200004  8211 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:14:56.200064  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:56.106688 13417 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:56.119459 13080 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.096416 13412 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:56.190495  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:56.205389  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.283968  8204 raft_consensus.cc:2410] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 13:14:56.200608  8060 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.061s	sys 0.110s
I20260709 13:14:56.116834 13369 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:56.273775  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:56.126655 13420 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:56.202399  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.205168  8026 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:56.103220 13323 raft_consensus.cc:740] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:56.207147  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:56.219972  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:56.200714  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:56.190541 13265 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:56.103626 13084 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.174s
I20260709 13:14:56.290939  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:56.319877  8209 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:56.104137 13199 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.173s
I20260709 13:14:56.330322  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:56.330878  8204 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:56.126253 13384 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:56.183117 13400 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:56.106834 13407 consensus_queue.cc:1059] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.169s
I20260709 13:14:56.281997 13487 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:14:56.307276  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 102204us
I20260709 13:14:56.309192  8130 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:56.201668  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:56.345180  8211 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:14:56.270474 13428 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:56.182883 13027 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.302024  8056 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:14:56.198550 13429 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.134s
I20260709 13:14:56.103765 13195 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:56.317688 13265 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:56.367849  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:56.369593  8209 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:14:56.355688  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.329377 13217 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:56.202741 13030 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:56.202926 13456 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:56.275559 13261 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:56.277566 13337 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:56.272186 13417 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:56.328043  8131 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:56.101007 13364 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:56.334758  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.102219 13423 consensus_queue.cc:1059] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.175s
I20260709 13:14:56.101688 13467 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.336683  8724 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:14:56.338822  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:56.340116  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.290760 13420 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:56.271011 13410 consensus_queue.cc:1059] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:14:56.270663 13398 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:14:56.312084 13403 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:14:56.316605  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 96736.8us
I20260709 13:14:56.315960  8132 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:56.372622  8135 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:14:56.407832  8029 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:56.374274  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:56.286865 13080 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:56.377381  8101 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:56.365999  8204 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:14:56.379101  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:56.389833  8057 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:56.414026  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:56.414260  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:56.414731  8204 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:56.374068 13265 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:56.385048 13417 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:56.384904 13402 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:56.424060  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:56.425081 13199 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:56.424103 13431 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:56.396596  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3247) took 1128 ms. Trace:
W20260709 13:14:56.403129  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 64385.8us
I20260709 13:14:56.427201  8696 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:56.375373 13217 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:56.428159 13217 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:56.376199 13261 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:56.429268 13261 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:56.352341 13387 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:14:56.407544 13410 consensus_queue.cc:1243] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 16.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:14:56.430909 13496 log.cc:927] Time spent T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 13:14:56.431442 13199 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:56.376071 13456 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:56.431959 13199 raft_consensus.cc:697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:56.432325 13456 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:56.402628 13440 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:56.377274 13428 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:56.432919 13199 consensus_queue.cc:237] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.416663  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:56.390275  8131 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:56.353986 13487 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.373114 13027 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:56.395467  8209 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:14:56.492126  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:56.492889  8209 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:14:56.494765  8132 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:14:56.419268  8101 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:56.495672  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 76484.4us
I20260709 13:14:56.422705  8133 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:14:56.401866  8211 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:56.426301  8212 rpcz_store.cc:276] 0709 13:14:55.268084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.271572 (+  3488us) service_pool.cc:224] Handling call
0709 13:14:55.312921 (+ 41349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:55.313092 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:14:55.375213 (+ 62121us) log.cc:844] Serialized 14 byte log entry
0709 13:14:56.001664 (+626451us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b1800497320. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb623a 00007f6dbb020a2a
0709 13:14:56.396301 (+394637us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:56.396348 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:56.396396 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.396454 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:56.396549 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12328,"prepare.queue_time_us":57801,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":2363,"raft.queue_time_us":5986,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":5360,"spinlock_wait_cycles":4965760,"thread_start_us":388422,"threads_started":1}
I20260709 13:14:56.427721 13224 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:56.380123 13337 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:56.410825 13489 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:56.436821  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:14:56.445235 13495 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:14:56.409407  8029 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:56.485651  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58514.9us
I20260709 13:14:56.487630  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 913) took 1239 ms. Trace:
I20260709 13:14:56.487988  8210 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:14:56.417135 13436 raft_consensus.cc:2764] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:14:56.499531  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3253) took 1230 ms. Trace:
I20260709 13:14:56.496668  8101 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:56.502651  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.504413  8203 rpcz_store.cc:276] 0709 13:14:55.248509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.261911 (+ 13402us) service_pool.cc:224] Handling call
0709 13:14:55.262236 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:55.262717 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:55.262748 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:55.263193 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 13:14:55.392308 (+129115us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:14:56.424489 (+1032181us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b18004da280. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf
0709 13:14:56.434211 (+  9722us) log.cc:844] Serialized 14 byte log entry
0709 13:14:56.434666 (+   455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.434749 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:56.434797 (+    48us) raft_consensus.cc:1685] finished
0709 13:14:56.487446 (+ 52649us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:56.487575 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24947,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":45,"raft.queue_time_us":2672,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":12538,"spinlock_wait_cycles":3787520,"thread_start_us":134557,"threads_started":2,"wal-append.queue_time_us":332315}
W20260709 13:14:56.507575  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 98263.6us
I20260709 13:14:56.501957 13274 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:56.509296  8057 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:56.512526  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:56.510803  8061 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:14:56.508383  8206 rpcz_store.cc:276] 0709 13:14:55.269574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.182802 (+913228us) service_pool.cc:224] Handling call
0709 13:14:56.183042 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.183231 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:56.292485 (+109254us) log.cc:844] Serialized 14 byte log entry
0709 13:14:56.499135 (+206650us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:56.499183 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:56.499240 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.499381 (+   141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:56.499478 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":104981,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":124,"thread_start_us":206316,"threads_started":1,"wal-append.queue_time_us":214601}
I20260709 13:14:56.515578  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:56.516012  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.516607  8057 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:56.505304 13337 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:56.509764  8029 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:56.500345 13387 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:56.460971 13402 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:56.542291  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3252) took 1273 ms. Trace:
I20260709 13:14:56.543071  8029 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:56.521345  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:56.544003  8057 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:56.502470 13364 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:56.520483 13274 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:56.537381  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:56.520087 13410 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:14:56.522333  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:56.590175  8209 raft_consensus.cc:2410] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:14:56.534616 13080 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:56.542680  8208 rpcz_store.cc:276] 0709 13:14:55.269319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.181998 (+912679us) service_pool.cc:224] Handling call
0709 13:14:56.201267 (+ 19269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:56.201592 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:56.201624 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:56.202021 (+   397us) log.cc:844] Serialized 26 byte log entry
0709 13:14:56.492841 (+290820us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:56.492880 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.492969 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:56.542102 (+ 49133us) raft_consensus.cc:1685] finished
0709 13:14:56.542119 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:56.542235 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":505,"raft.queue_time_us":80,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":72283,"thread_start_us":290517,"threads_started":1,"wal-append.queue_time_us":338348}
I20260709 13:14:56.549957  8029 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:56.529268  8204 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
I20260709 13:14:56.524436  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3254) took 1254 ms. Trace:
I20260709 13:14:56.591419  8210 rpcz_store.cc:276] 0709 13:14:55.269786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.523360 (+1253574us) service_pool.cc:224] Handling call
0709 13:14:56.523830 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.524042 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:56.524078 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:56.524132 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:56.524176 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.524244 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:56.524373 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:56.551621  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3089) took 1514 ms. Trace:
I20260709 13:14:56.512225 13398 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:14:56.563688  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.598927  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3258) took 1326 ms. Trace:
W20260709 13:14:56.590399 13500 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:14:56.608150  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:56.552004 13429 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.551038 13384 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:56.607252  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3255) took 1337 ms. Trace:
I20260709 13:14:56.609369  8209 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:14:56.609822  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 430190us
I20260709 13:14:56.601428  8057 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:56.594285  8060 rpcz_store.cc:276] 0709 13:14:55.037222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.049557 (+ 12335us) service_pool.cc:224] Handling call
0709 13:14:55.049855 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:55.562818 (+512963us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:14:55.637384 (+ 74566us) log.cc:844] Serialized 14 byte log entry
0709 13:14:56.391588 (+754204us) log.cc:844] Serialized 14 byte log entry
0709 13:14:56.403067 (+ 11479us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:56.411904 (+  8837us) log.cc:844] Serialized 26 byte log entry
0709 13:14:56.412029 (+   125us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:56.412060 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.412119 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:56.536035 (+123916us) spinlock_profiling.cc:243] Waited 6.33 ms on lock 0x7b18003484c0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:14:56.551408 (+ 15373us) raft_consensus.cc:1685] finished
0709 13:14:56.551432 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:56.551559 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36429,"prepare.queue_time_us":262242,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":6898,"raft.queue_time_us":87300,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":4698,"spinlock_wait_cycles":14550272,"thread_start_us":1241929,"threads_started":3,"wal-append.queue_time_us":773144}
I20260709 13:14:56.522058  8206 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 17: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:14:56.603688 13403 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:56.648523  8056 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:14:56.652197  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 917) took 1135 ms. Trace:
I20260709 13:14:56.662604  8209 rpcz_store.cc:276] 0709 13:14:55.517261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.601646 (+1084385us) service_pool.cc:224] Handling call
0709 13:14:56.609006 (+  7360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.651953 (+ 42947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.652143 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:56.652767  8208 rpcz_store.cc:276] 0709 13:14:55.270005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.597519 (+1327514us) service_pool.cc:224] Handling call
0709 13:14:56.606667 (+  9148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.606879 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:56.606913 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:56.606966 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:56.607008 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.607075 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:56.607187 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:56.653677 13429 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:56.656157 13504 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:14:56.603454  8204 rpcz_store.cc:276] 0709 13:14:55.272357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.598124 (+1325767us) service_pool.cc:224] Handling call
0709 13:14:56.598420 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.598611 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:56.598637 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:56.598679 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:56.598725 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.598786 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:56.598881 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:56.508168 13428 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.595947 13398 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:56.550827 13407 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:56.684464  8208 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:14:56.685006  8208 pending_rounds.cc:85] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 8
I20260709 13:14:56.667202 13447 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:56.669765 13493 raft_consensus.cc:3055] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:14:56.687929  8138 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.010s	sys 0.045s
I20260709 13:14:56.688011 13493 raft_consensus.cc:740] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:56.689204  8057 raft_consensus.cc:1240] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:56.656953 13431 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.675865  8696 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:14:56.676133  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.679865  8204 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:56.685281  8208 pending_rounds.cc:107] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:14:56.692313  8208 raft_consensus.cc:2889] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:56.693079  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3261) took 1124 ms. Trace:
I20260709 13:14:56.686800  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 918) took 1169 ms. Trace:
W20260709 13:14:56.688449  8138 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.010s	sys 0.045s
I20260709 13:14:56.658363  8104 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:56.674782  8206 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:56.694042  8209 rpcz_store.cc:276] 0709 13:14:55.517545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.663519 (+1145974us) service_pool.cc:224] Handling call
0709 13:14:56.686275 (+ 22756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.686477 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:56.686503 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:56.686541 (+    38us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:56.686573 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.686647 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:56.686752 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:56.693562  8208 rpcz_store.cc:276] 0709 13:14:55.568553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.663882 (+1095329us) service_pool.cc:224] Handling call
0709 13:14:56.684087 (+ 20205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.692794 (+  8707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.693010 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:56.695194  8206 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:56.672375 13467 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:56.549257 13387 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.696939  8206 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:56.683441 13398 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:56.670161 13261 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:56.693755 13503 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:14:56.696086 13224 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:56.644222 13274 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 15.6, Last appended by leader: 6, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:56.700445 13457 consensus_meta.cc:287] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.664s	user 0.012s	sys 0.004s
I20260709 13:14:56.661765 13403 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:56.700511 13219 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:14:56.720993  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.697552 13423 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:14:56.696485 13431 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:56.721577 13487 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:56.737494 13502 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:14:56.718078 13224 raft_consensus.cc:697] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:56.720543 13403 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:56.720731 13219 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:56.742937  8722 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 947) took 1056 ms. Trace:
W20260709 13:14:56.745556  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:56.735116  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 185230us
I20260709 13:14:56.743350  8722 rpcz_store.cc:276] 0709 13:14:55.686550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.686675 (+   125us) service_pool.cc:224] Handling call
0709 13:14:56.742711 (+1056036us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b5400505fd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:56.742854 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283516672}
I20260709 13:14:56.692867 13493 consensus_queue.cc:260] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.750304  8029 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:56.755350 13224 consensus_queue.cc:237] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.763296 13213 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:56.763906 13213 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.765419 13487 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:56.765908 13487 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:56.766418 13487 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:56.766085 13213 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:56.767782  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:56.775679 13431 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:56.776751 13487 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:56.779184 13487 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:56.783008 13217 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:56.785466 13217 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:56.787055 13261 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:56.788097  8029 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:56.788585 13387 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:56.791121 13261 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:56.793731  8029 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:56.796458 13387 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:56.797145 13387 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:56.798180 13387 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:56.739077 13402 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:56.802865  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:56.806528 13402 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:56.807060 13429 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:56.807142 13402 raft_consensus.cc:697] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:56.747236 13219 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:56.808426 13402 consensus_queue.cc:237] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.820700  8203 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
W20260709 13:14:56.829180  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 130622us
W20260709 13:14:56.895133  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 62480.7us
I20260709 13:14:56.835573  8104 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:14:56.908201  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.835588 13265 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:56.917575  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 82115.6us
I20260709 13:14:56.919561  8104 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:56.922984  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:56.923559  8724 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:56.947088  8054 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:56.949586  8054 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:14:56.949975  8054 pending_rounds.cc:85] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 11
I20260709 13:14:56.950242  8054 pending_rounds.cc:107] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:14:56.950685  8054 raft_consensus.cc:2889] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:56.834293 13386 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:14:56.835790 13261 consensus_queue.cc:1059] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
W20260709 13:14:56.924026 13420 consensus_meta.cc:287] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.515s	user 0.006s	sys 0.005s
I20260709 13:14:56.896390 13337 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:56.836822 13431 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:14:56.914301 13265 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:56.974517  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:56.947110 13417 consensus_meta.cc:287] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.513s	user 0.005s	sys 0.004s
I20260709 13:14:56.930227 13441 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:56.975268  8203 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:14:56.903224 13217 consensus_queue.cc:1059] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:14:56.900357 13517 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:14:56.910953 13519 consensus_queue.cc:1059] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
W20260709 13:14:56.976094 13456 consensus_meta.cc:287] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.543s	user 0.018s	sys 0.000s
I20260709 13:14:56.986690 13456 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:57.003269 13514 log.cc:927] Time spent T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:14:57.005128 13456 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:56.957877 13521 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:56.985400  8723 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:14:57.054097  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:57.090363  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 170837us
I20260709 13:14:57.091266 13521 raft_consensus.cc:697] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:57.092154 13521 consensus_queue.cc:237] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.093009 13398 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.097811  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:57.102186 13217 raft_consensus.cc:3055] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:57.105180  8724 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:14:56.983996 13402 raft_consensus.cc:3055] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:57.106233  8104 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:57.107393  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:57.099326 13487 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.977226 13420 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.096593 13431 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:57.104420 13217 raft_consensus.cc:740] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:56.981305 13417 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.101300 13456 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:57.110642 13034 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:14:57.112622 13487 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.106540 13517 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:57.117379 13517 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:57.117442 13307 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:57.117847 13517 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:57.118328 13487 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:57.125362  8730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:50008 (request call id 2) took 2120 ms. Trace:
I20260709 13:14:57.125730  8730 rpcz_store.cc:276] 0709 13:14:55.006053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.006184 (+   131us) service_pool.cc:224] Handling call
0709 13:14:57.125155 (+2118971us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54001793d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb010853 00007f6dbffde60a 00007f6db95647d4 00007f6db9564762 00007f6db9564691 00007f6db956460c 00007f6db95638c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:57.125272 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1209843456}
I20260709 13:14:57.114256 13402 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:57.126225 13337 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:57.126981 13402 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.112017 13431 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.140187 13431 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:57.141723 13489 consensus_meta.cc:287] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.000s
I20260709 13:14:57.142202 13489 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:57.145293 13517 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.149165 13517 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:57.150700 13398 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:57.121809 13456 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.130518  8138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b4358ace86545339036f21610735b47"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:14:57.131879  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 2037) took 1076 ms. Trace:
I20260709 13:14:57.162585  8209 rpcz_store.cc:276] 0709 13:14:56.055406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.698860 (+643454us) service_pool.cc:224] Handling call
0709 13:14:56.700185 (+  1325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:56.704364 (+  4179us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:56.704392 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:56.707698 (+  3306us) log.cc:844] Serialized 26 byte log entry
0709 13:14:56.898747 (+191049us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:56.899710 (+   963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.901564 (+  1854us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:57.129805 (+228241us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:57.131552 (+  1747us) raft_consensus.cc:1685] finished
0709 13:14:57.131572 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:57.131815 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1068,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"spinlock_wait_cycles":4137344,"thread_start_us":174651,"threads_started":1,"wal-append.queue_time_us":266113}
I20260709 13:14:57.164602  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:57.165148  8209 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:57.134387 13217 consensus_queue.cc:260] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.135649 13417 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:57.136500  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:57.136958 13420 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:57.138793 13403 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.166960 13217 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:57.168450  8056 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:57.166957  8203 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:57.161367  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.176205 13403 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:57.178794  8203 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:57.180383  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:57.182124  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:57.182353  8055 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:57.183049  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:57.183624  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1867) took 1193 ms. Trace:
I20260709 13:14:57.184104  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:57.184010  8055 rpcz_store.cc:276] 0709 13:14:55.990161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.990665 (+   504us) service_pool.cc:224] Handling call
0709 13:14:57.183452 (+1192787us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540027a954. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:57.183550 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2738710272}
I20260709 13:14:57.185199  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:57.185561  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:57.185678  8211 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:57.197536  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.185709  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 809) took 1013 ms. Trace:
I20260709 13:14:57.201264  8055 rpcz_store.cc:276] 0709 13:14:56.172436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.185023 (+1012587us) service_pool.cc:224] Handling call
0709 13:14:57.185268 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:57.185432 (+   164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:57.185457 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:57.185496 (+    39us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:57.185531 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.185578 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:57.185665 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:57.204281  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:57.205623  8203 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:14:57.209064  8209 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:14:57.210369  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:57.128126 13402 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:57.126120 13219 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.188179 13456 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:57.225878  8211 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:14:57.226204  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:57.227900 13265 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.230571 13265 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.223938 13415 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:14:57.236991  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:57.234846  8138 raft_consensus.cc:606] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:57.234009  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:57.223219 13420 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:57.239099  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:57.239574  8211 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:14:57.224542 13274 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.224776 13429 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:57.242954  8203 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:57.243769  8209 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:14:57.245056  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:57.246565  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3265) took 1080 ms. Trace:
I20260709 13:14:57.246902  8209 rpcz_store.cc:276] 0709 13:14:56.166283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.243147 (+1076864us) service_pool.cc:224] Handling call
0709 13:14:57.243442 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:57.246321 (+  2879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.246513 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:57.249298  8209 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:57.250084 13274 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.252115 13274 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:57.236543 13393 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.254653 13393 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.256704 13393 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:57.259086 13219 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:57.260040 13429 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.262007  8209 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:57.264331  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:57.265269 13525 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.257s	user 0.000s	sys 0.000s
I20260709 13:14:57.266532  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:57.267797  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 2038) took 1212 ms. Trace:
I20260709 13:14:57.272284  8029 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:57.268301  8206 rpcz_store.cc:276] 0709 13:14:56.055715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.698874 (+643159us) service_pool.cc:224] Handling call
0709 13:14:56.699335 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:56.699739 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:56.699780 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:56.700458 (+   678us) log.cc:844] Serialized 26 byte log entry
0709 13:14:56.916439 (+215981us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:56.916479 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.916542 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:57.264750 (+348208us) spinlock_profiling.cc:243] Waited 87.2 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:57.267528 (+  2778us) raft_consensus.cc:1685] finished
0709 13:14:57.267547 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:57.267742 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":406,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":8279,"spinlock_wait_cycles":200572160,"thread_start_us":174987,"threads_started":1,"wal-append.queue_time_us":256991}
I20260709 13:14:57.266892 13265 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:57.246136 13456 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:57.262152 13429 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:57.274243 13489 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:57.287470  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:57.288108  8724 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:57.288686  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 1865) took 1299 ms. Trace:
I20260709 13:14:57.288786  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:57.289603  8730 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:57.274699 13217 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:57.290616 13217 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:57.291004 13217 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:57.291556 13489 raft_consensus.cc:697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:57.292412 13489 consensus_queue.cc:237] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.248322 13415 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:57.294948 13415 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.295630  8723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 898) took 1248 ms. Trace:
I20260709 13:14:57.295969  8723 rpcz_store.cc:276] 0709 13:14:56.047362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.049680 (+  2318us) service_pool.cc:224] Handling call
0709 13:14:56.743990 (+694310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:57.094816 (+350826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.295013 (+200197us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b5400505fd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:57.295462 (+   449us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b5400505fd0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:57.295579 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2150533888}
I20260709 13:14:57.298364 13431 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.289323  8058 rpcz_store.cc:276] 0709 13:14:55.989694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.030048 (+ 40354us) service_pool.cc:224] Handling call
0709 13:14:57.288530 (+1258482us) spinlock_profiling.cc:243] Waited 50.8 ms on lock 0x7b54001e82d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:57.288634 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116946176}
I20260709 13:14:57.300663  8206 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:14:57.299037 13456 raft_consensus.cc:697] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:57.308367 13517 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:14:57.316867 13456 consensus_queue.cc:237] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:57.354808  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 55086.1us
I20260709 13:14:57.354178 13027 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:57.373742  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3097) took 1265 ms. Trace:
I20260709 13:14:57.374140  8057 rpcz_store.cc:276] 0709 13:14:56.108446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.650162 (+541716us) service_pool.cc:224] Handling call
0709 13:14:56.650449 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.689589 (+ 39140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.373518 (+683929us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b54003ba454. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:57.373687 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88644864}
I20260709 13:14:57.362614 13402 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.380517 13402 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:57.381139  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.386510  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:14:57.387131 13528 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.380s	user 0.002s	sys 0.000s
I20260709 13:14:57.387784  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:57.392699  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:57.394677  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:57.394686  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:57.398149 13534 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:57.398872 13534 consensus_queue.cc:1243] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:14:57.401054  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.408785  8062 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:57.409344  8203 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:57.409989  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:57.411375  8724 raft_consensus.cc:3055] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:57.411718  8724 raft_consensus.cc:740] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:57.412609  8724 consensus_queue.cc:260] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.413460  8724 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:57.415153 13534 consensus_queue.cc:1059] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:57.416141 13534 raft_consensus.cc:3055] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:57.416460 13534 raft_consensus.cc:740] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:57.417246 13534 consensus_queue.cc:260] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.418062  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 916) took 1901 ms. Trace:
I20260709 13:14:57.418715 13534 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:57.418478  8210 rpcz_store.cc:276] 0709 13:14:55.516948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.599324 (+1082376us) service_pool.cc:224] Handling call
0709 13:14:56.599599 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:56.599922 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:56.599946 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:57.286414 (+686468us) spinlock_profiling.cc:243] Waited 6.13 ms on lock 0x7b2c00116ca8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db990fa11 00007f6db991f372 00007f6dbb051a01 00007f6dbb01643a 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2
0709 13:14:57.303545 (+ 17131us) log.cc:844] Serialized 26 byte log entry
0709 13:14:57.336675 (+ 33130us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:57.416918 (+ 80243us) log.cc:844] Serialized 14 byte log entry
0709 13:14:57.417715 (+   797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.417785 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:57.417826 (+    41us) raft_consensus.cc:1685] finished
0709 13:14:57.417904 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:57.418013 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2027,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":3660,"spinlock_wait_cycles":14091008,"thread_start_us":97291,"threads_started":2,"wal-append.queue_time_us":57549}
I20260709 13:14:57.420575  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:57.421296  8724 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:14:57.422015  8206 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: []
I20260709 13:14:57.422063  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:57.423230  8027 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:57.423151  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:57.425829  8210 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:14:57.425370  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:57.390578 13217 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.428315  8027 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:57.429157  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:57.428978  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:57.430006  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:57.421763 13407 consensus_queue.cc:1059] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:14:57.408455 13440 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.438320 13440 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.439170  8056 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3100) took 1330 ms. Trace:
I20260709 13:14:57.439515  8056 rpcz_store.cc:276] 0709 13:14:56.109172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.664460 (+555288us) service_pool.cc:224] Handling call
0709 13:14:56.687351 (+ 22891us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:57.168805 (+481454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.438962 (+270157us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b540028afd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:57.439116 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105945472}
I20260709 13:14:57.440335 13440 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:57.434937  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:57.441043  8203 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:57.422106 13369 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.441642 13369 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.442657 13431 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:57.443297  8206 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:14:57.433319 13456 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:57.433833 13481 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:57.444829  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:57.445075  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:57.445884  8209 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:57.446661  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2179) took 3335 ms. Trace:
I20260709 13:14:57.447261  8209 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:57.447012  8134 rpcz_store.cc:276] 0709 13:14:54.111324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.727816 (+616492us) service_pool.cc:224] Handling call
0709 13:14:56.010429 (+1282613us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.010962 (+   533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.446467 (+1435505us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540008cd50. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:57.446585 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2949105920}
I20260709 13:14:57.432711 13532 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:14:57.430840 13402 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:57.451424 13384 raft_consensus.cc:3055] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:57.451913 13384 raft_consensus.cc:740] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:57.452845 13384 consensus_queue.cc:260] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.447305 13369 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:57.451555 13532 raft_consensus.cc:3055] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:57.458490 13532 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:57.425827 13027 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:14:57.459695 13027 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:14:57.460057 13027 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:57.460831 13027 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.461920 13027 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:57.459280 13532 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.466150 13532 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:57.468071  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:57.468580 13526 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.462s	user 0.002s	sys 0.000s
I20260709 13:14:57.469523  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:57.470057  8138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e397a319b11487fbbacec77723b5739"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:14:57.470603  8138 raft_consensus.cc:606] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:57.470892  8138 raft_consensus.cc:614] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:57.445372  8206 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:57.444479 13440 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.474577 13440 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.477635  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:57.479154 13384 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:57.439673 13217 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:57.481859 13440 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:57.486670  8211 pending_rounds.cc:85] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 9
I20260709 13:14:57.487041  8211 pending_rounds.cc:107] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:14:57.487315  8211 raft_consensus.cc:2889] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:57.487663  8211 pending_rounds.cc:107] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:14:57.488003  8211 raft_consensus.cc:2889] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:57.488369  8211 pending_rounds.cc:107] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:14:57.488696  8211 raft_consensus.cc:2889] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:57.488585  8206 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:14:57.465243  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 817) took 1098 ms. Trace:
I20260709 13:14:57.490272  8206 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:14:57.490552  8062 rpcz_store.cc:276] 0709 13:14:56.373897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.293769 (+919872us) service_pool.cc:224] Handling call
0709 13:14:57.294040 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:57.409161 (+115121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.464957 (+ 55796us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540028aad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:57.465174 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":263509376}
I20260709 13:14:57.491415  8206 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:14:57.494246  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 57293.3us
I20260709 13:14:57.501981  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:57.496937 13217 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
W20260709 13:14:57.511515  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 1.70193e+06us
W20260709 13:14:57.512185  8175 leader_election.cc:336] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 1.941s (SENT)
I20260709 13:14:57.512552  8175 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:57.513257 13440 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:57.515285  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:57.515657  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:57.516181  8062 raft_consensus.cc:2393] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:57.516956  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 908) took 3081 ms. Trace:
I20260709 13:14:57.517303  8207 rpcz_store.cc:276] 0709 13:14:54.432993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.433162 (+   169us) service_pool.cc:224] Handling call
0709 13:14:56.109041 (+1675879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.157320 (+ 48279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.514098 (+1356778us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54002be650. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:57.514233 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3802493696}
W20260709 13:14:57.518231  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.518685  8207 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:57.520002  8207 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:14:57.520045 13027 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:57.521221 13027 raft_consensus.cc:3055] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:57.521570 13027 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:57.521755 12914 consensus_queue.cc:1059] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:57.522506 13027 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.523576 13027 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:14:57.528120 13527 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.521s	user 0.001s	sys 0.000s
I20260709 13:14:57.528156  8207 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:57.529882 13354 raft_consensus.cc:3055] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:57.530264 13354 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:57.531105 13354 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.533039 13323 consensus_queue.cc:260] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:57.535723  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:57.539109 13529 log.cc:927] Time spent T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.528s	user 0.000s	sys 0.002s
I20260709 13:14:57.540994  8102 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:57.541620  8129 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:57.542208 13534 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:57.542744 13431 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:57.543151 13431 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.543495 13431 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:57.549747  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:57.550299  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:57.551815  8132 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:14:57.554065  8104 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:57.555408  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:57.542541 13307 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:57.568542  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:57.569085  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:57.557049  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3248) took 2288 ms. Trace:
I20260709 13:14:57.557605  8204 raft_consensus.cc:1217] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Deduplicated request from leader. Original: 8.5->[10.6-14.9]   Dedup: 13.8->[14.9-14.9]
I20260709 13:14:57.570801  8204 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15. Ops: [10.6-14.9]
I20260709 13:14:57.558799  8134 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:57.542078  8102 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:57.564788  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.542879 13534 consensus_queue.cc:1243] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 14.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:14:57.549464  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:57.578938  8131 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 17: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:57.569427  8059 raft_consensus.cc:2368] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12.
I20260709 13:14:57.566992  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:57.576694  8026 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:57.576725  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1851) took 1599 ms. Trace:
I20260709 13:14:57.580503  8062 raft_consensus.cc:2376] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12. Re-sending same reply.
I20260709 13:14:57.580657  8063 rpcz_store.cc:276] 0709 13:14:55.977092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.977219 (+   127us) service_pool.cc:224] Handling call
0709 13:14:55.977589 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:56.421467 (+443878us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:56.421496 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:56.421921 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 13:14:56.689866 (+267945us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:14:56.689909 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.549988 (+860079us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b540009c254. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:57.550026 (+    38us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:57.550065 (+    39us) raft_consensus.cc:1685] finished
0709 13:14:57.550076 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:57.576668 (+ 26592us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":256,"raft.queue_time_us":943,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":9342,"spinlock_wait_cycles":1019392896,"thread_start_us":245631,"threads_started":1,"wal-append.queue_time_us":331546}
I20260709 13:14:57.577733  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:57.569923  8205 rpcz_store.cc:276] 0709 13:14:55.268375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.524443 (+256068us) service_pool.cc:224] Handling call
0709 13:14:55.524941 (+   498us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:14:55.767128 (+242187us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:55.790437 (+ 23309us) log.cc:844] Serialized 14 byte log entry
0709 13:14:55.919878 (+129441us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:14:55.920685 (+   807us) log.cc:844] Serialized 78 byte log entry
0709 13:14:55.920869 (+   184us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:55.921173 (+   304us) log.cc:844] Serialized 14 byte log entry
0709 13:14:55.926975 (+  5802us) log.cc:844] Serialized 14 byte log entry
0709 13:14:55.927300 (+   325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.041744 (+114444us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b1800135520. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f29522 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:14:57.525330 (+1483586us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b1800135520. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ef72a6 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419
0709 13:14:57.526255 (+   925us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54000ba4d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:57.526298 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:57.556848 (+ 30550us) raft_consensus.cc:1685] finished
0709 13:14:57.556866 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:57.556999 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5183,"prepare.queue_time_us":44107,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":2091,"raft.queue_time_us":5747,"raft.run_cpu_time_us":474,"raft.run_wall_time_us":14805,"spinlock_wait_cycles":282158592,"thread_start_us":100389,"threads_started":1,"wal-append.queue_time_us":150604,"wal-append.run_cpu_time_us":4331,"wal-append.run_wall_time_us":1583049}
I20260709 13:14:57.585098  8134 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 12: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:57.585995  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:57.590791  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:57.591598  8057 raft_consensus.cc:2410] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:14:57.592280  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:57.592931  8057 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 12: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:57.594504  8059 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:57.595484  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:57.596181  8059 raft_consensus.cc:2410] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:14:57.597115  8063 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:57.598096  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:57.599056  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:57.534253 13323 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:57.600965  8730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:50008
I20260709 13:14:57.606483  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:57.586817 13456 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:57.585888 13274 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:57.607236  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:57.607929  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:57.610009 13524 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.595s	user 0.000s	sys 0.001s
I20260709 13:14:57.611382  8054 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3110) took 1483 ms. Trace:
I20260709 13:14:57.613667  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:57.649629  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:57.648392  8211 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:57.647091  8054 rpcz_store.cc:276] 0709 13:14:56.128145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.958938 (+830793us) service_pool.cc:224] Handling call
0709 13:14:57.186525 (+227587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:57.186743 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:57.186780 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:57.186830 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:57.186880 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.186950 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:57.611232 (+424282us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540027a950. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:57.611331 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522613504}
I20260709 13:14:57.576076 13429 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:14:57.658993  8134 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:57.657390  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:57.660760  8054 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:57.661300  8725 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:57.614156  8063 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:57.615445  8062 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:14:57.600019 13453 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.617057  8130 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:57.664911  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:57.665580  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:57.621603 13407 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:57.625237  8056 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:14:57.625859  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 62872.1us
I20260709 13:14:57.625981 13420 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:57.627239 13476 raft_consensus.cc:3055] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:57.627156 13080 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:14:57.601385 13447 consensus_queue.cc:1059] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:14:57.710212  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:57.645784 13274 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:57.634030 13379 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:57.635211  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:57.722553  8138 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:57.636277 13427 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:14:57.636490  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 94474.4us
I20260709 13:14:57.714586  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:57.636793 13423 consensus_queue.cc:1059] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:14:57.637992  8205 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:57.636233  8104 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:57.666286 13400 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:57.728814  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 92648.2us
I20260709 13:14:57.637830 13533 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:57.622682 13433 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
W20260709 13:14:57.640456  8061 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 3109) took 1512 ms (client timeout 1887 ms). Trace:
I20260709 13:14:57.658707 13429 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.600144 13364 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:14:57.643925  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:57.644538  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:57.661791  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.662940  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:57.601114  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:57.623409  8026 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:57.659828  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:57.668422  8101 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:57.622291 13198 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:14:57.707013 13476 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:57.630640 13437 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:57.711915  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:57.712347  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:57.839471  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:57.582103 13397 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:57.735105  8054 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:57.711154  8135 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:14:57.700379 13453 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.827168  8730 raft_consensus.cc:606] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:14:57.733556  8061 rpcz_store.cc:269] 0709 13:14:56.127917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.291211 (+163294us) service_pool.cc:224] Handling call
0709 13:14:57.640306 (+1349095us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b5400333e54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:57.640403 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282236672}
I20260709 13:14:57.837479  8137 raft_consensus.cc:2376] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13. Re-sending same reply.
W20260709 13:14:57.741044  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.824646  8210 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 6. Preceding OpId from leader: term: 17 index: 7. (index mismatch)
I20260709 13:14:57.838438  8134 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:14:57.839614  8063 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:14:57.726863  8102 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:57.839723  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 171367us
W20260709 13:14:57.859254  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 235915us
I20260709 13:14:57.859545  8101 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:57.840698  8175 leader_election.cc:336] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Timed out: RequestConsensusVote RPC to 127.6.198.65:45919 timed out after 2.192s (SENT)
W20260709 13:14:57.842355  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.843477  8056 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:14:57.805608  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:14:57.860584  8175 connection.cc:256] RPC call timeout handler was delayed by -1.4616399639999145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
W20260709 13:14:57.753958  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 44745.3us
W20260709 13:14:57.860946  8175 connection.cc:256] RPC call timeout handler was delayed by -1.4562399639999057s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
I20260709 13:14:57.861119  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:57.848973  8062 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:57.652882 13456 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.852559  8138 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:14:57.737308 13274 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:57.856551  8730 raft_consensus.cc:614] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:14:57.743359  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.861532  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:57.716383 13489 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.129s
I20260709 13:14:57.736888 13393 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:57.739162 13089 raft_consensus.cc:3055] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:57.836874 13429 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:57.868642  8175 leader_election.cc:400] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:14:57.853848  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:57.863543  8101 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:57.862453 13034 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.865844  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:57.867239  8026 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:57.870133  8175 leader_election.cc:403] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 13:14:57.867995  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.869868  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:57.846997 13397 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.880653  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:57.880808  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:57.881450 13522 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.874s	user 0.002s	sys 0.007s
I20260709 13:14:57.882035 13440 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:14:57.882190 13028 raft_consensus.cc:3055] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:57.882666 13028 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:57.883126  8060 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3099) took 1774 ms. Trace:
I20260709 13:14:57.883400 13028 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.883491  8060 rpcz_store.cc:276] 0709 13:14:56.108929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.657630 (+548701us) service_pool.cc:224] Handling call
0709 13:14:56.657950 (+   320us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:56.676446 (+ 18496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:56.676491 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:56.686848 (+ 10357us) log.cc:844] Serialized 52 byte log entry
0709 13:14:56.903474 (+216626us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:14:56.903513 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.903573 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:57.880902 (+977329us) spinlock_profiling.cc:243] Waited 87.9 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:14:57.882794 (+  1892us) raft_consensus.cc:1685] finished
0709 13:14:57.882813 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:57.883041 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":114,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":16087,"spinlock_wait_cycles":202096000,"thread_start_us":201356,"threads_started":1,"wal-append.queue_time_us":277895}
I20260709 13:14:57.884688 13028 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:57.885344 13431 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.881510 13235 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:14:57.886698  8055 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:14:57.887393 13397 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:57.887645  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:57.888396  8130 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:14:57.888516  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 810) took 1701 ms. Trace:
I20260709 13:14:57.889181 13195 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:57.889252  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:57.890569  8136 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:14:57.881610  8135 raft_consensus.cc:2393] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:57.881433  8061 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:14:57.887384 13431 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:57.888911  8055 rpcz_store.cc:276] 0709 13:14:56.186931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.204167 (+1017236us) service_pool.cc:224] Handling call
0709 13:14:57.886464 (+682297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:57.887107 (+   643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.888344 (+  1237us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b540028a5d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:57.888465 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1513921408}
I20260709 13:14:57.881636 13453 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:57.894109 13453 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:57.894045  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:57.894819  8055 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:57.895654  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:57.897054  8055 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:14:57.866724 13217 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:57.897603  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:57.885651  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:57.898880 13521 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:57.899351 13265 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:57.899506 13447 raft_consensus.cc:3055] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:57.901124 13413 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.901203 13393 raft_consensus.cc:697] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:57.902207 13034 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.902823 13436 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:14:57.737836 13400 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.904917  8058 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 7. (term mismatch)
I20260709 13:14:57.841884 13420 consensus_queue.cc:1059] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:14:57.870798 13089 raft_consensus.cc:740] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:57.906898  8059 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:14:57.907025  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 43547.8us
I20260709 13:14:57.971232  8059 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []
W20260709 13:14:57.918973 13554 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:14:57.892550  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:57.912019  8208 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:57.912828  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:57.914742  8725 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:57.976758  8061 raft_consensus.cc:2410] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 15 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:14:57.917112  8134 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:14:57.917261  8205 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:14:57.978752  8027 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:57.870587 13261 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:57.990489 13261 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:57.918529  8724 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:57.874550 13369 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:14:57.921875  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 157198us
I20260709 13:14:57.992172 13369 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:57.992585 13369 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:57.924687  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:57.993942  8056 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:57.994987  8056 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:57.995713 13034 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:57.996001  8056 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:57.996680 13034 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:57.998481  8056 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 17 index: 7. (index mismatch)
I20260709 13:14:57.972563 13420 consensus_queue.cc:1243] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:14:57.927008  8055 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:57.805259 13379 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:57.924573 13447 raft_consensus.cc:740] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:57.999977 13089 consensus_queue.cc:260] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.904588  8129 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:14:58.001153 13447 consensus_queue.cc:260] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.001713  8129 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:14:58.002274 13447 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:58.002933  8129 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:58.003445 13456 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:57.868312 13423 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:14:58.004259 13423 consensus_queue.cc:1243] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:14:58.005198 13090 consensus_queue.cc:1059] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:58.006870  8130 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:14:57.924398 13265 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.008925 13265 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:58.008621 12920 consensus_queue.cc:1059] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:58.009929 12920 raft_consensus.cc:3055] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:58.011705 13379 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:57.939100  8062 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:14:57.962281 13413 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.016227 13413 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:57.964550 13400 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:57.965495  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:57.978463 13521 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:14:57.998056 13034 consensus_queue.cc:1059] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:14:58.020539 13521 raft_consensus.cc:3055] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:58.020654 13034 raft_consensus.cc:3055] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:58.021042 13521 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:58.021267 13034 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:58.027593  8132 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:57.901162 13532 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:57.968997  8058 pending_rounds.cc:85] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 6
I20260709 13:14:57.971710  8101 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:58.034382  8058 pending_rounds.cc:107] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 15.7
I20260709 13:14:58.034710  8058 raft_consensus.cc:2889] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:58.034780  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 63151.8us
I20260709 13:14:57.907708 13431 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:57.870703 13408 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:14:57.979673  8208 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:14:57.914542 13427 consensus_queue.cc:1059] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:14:57.920326 13217 raft_consensus.cc:697] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:57.979939  8134 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14. Ops: []
W20260709 13:14:57.981472 13551 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:14:57.981022 13393 consensus_queue.cc:237] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:57.982113 13553 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:14:57.982950 13274 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:57.983343 13027 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:57.983837 13402 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.983868 13364 consensus_queue.cc:1059] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:14:57.985451  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.039844  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:58.041000  8203 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:14:58.036336  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:58.070690  8101 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:58.044943 13398 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:58.045256 13369 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.045946 13408 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:14:58.077740  8056 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:58.046638 13431 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:58.048579  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:58.054315  8061 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:58.074095  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:58.082868  8063 raft_consensus.cc:2410] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:14:58.076745  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:58.030390 12920 raft_consensus.cc:740] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:58.084726 12920 consensus_queue.cc:260] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.085666 12920 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:58.037534 13532 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.022251 13034 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.087990 13532 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:58.088357 13034 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:58.090866 13400 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.091513 13400 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.096350 13400 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:58.082031  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:58.097849  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:58.022148 13521 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.110374 13521 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:58.072192 13027 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:58.109647  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 242469us
W20260709 13:14:58.321669  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.322248  8026 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:58.121897  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.325533  8026 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:58.133136  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:58.331286  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 338468us
I20260709 13:14:58.133805  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2190) took 2031 ms. Trace:
I20260709 13:14:58.338761  8133 rpcz_store.cc:276] 0709 13:14:56.102261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.102656 (+   395us) service_pool.cc:224] Handling call
0709 13:14:56.102941 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:56.497305 (+394364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.132947 (+1635642us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b5400130e54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:58.133727 (+   780us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":734840704}
I20260709 13:14:58.341118  8137 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
W20260709 13:14:58.145291  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 418496us
I20260709 13:14:58.155859  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 2034) took 2367 ms. Trace:
I20260709 13:14:58.343329  8204 rpcz_store.cc:276] 0709 13:14:55.788316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.696021 (+907705us) service_pool.cc:224] Handling call
0709 13:14:57.557427 (+861406us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:14:57.571162 (+ 13735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.155593 (+584431us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b54000ba4d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:58.155778 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1980158336}
W20260709 13:14:58.135428  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 260362us
W20260709 13:14:58.243208  8696 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.348762  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:58.350221  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:58.224192  8722 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:58.354166  8722 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:58.153505  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 285423us
I20260709 13:14:58.239106  8203 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14. Ops: []
I20260709 13:14:58.356305  8722 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.357818  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:58.359714  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:58.359925  8722 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:58.343966  8102 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:14:58.362977  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.366945  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:58.117825 13027 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:58.368032  8203 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:58.252185 13028 raft_consensus.cc:3055] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:58.125588 13034 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:58.184329 13080 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:58.188112 13089 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:58.406738 13456 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:14:58.409508  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.410131 13090 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.410513  8101 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:58.411139 13090 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.412683  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:58.137347 13217 consensus_queue.cc:237] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:58.419337  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.423126  8205 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:14:58.426441 13090 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:58.427067  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:58.173672 13219 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.178994 13265 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.175783 13274 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:58.105677 13290 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:14:58.386260 13307 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:14:58.431684  7961 catalog_manager.cc:5697] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 6 to 7, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 7 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:58.160022 13354 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:58.208197 13369 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:58.054595 13397 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.437489 13397 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:58.103864 13402 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.067320 13413 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:14:58.333964  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3111) took 1954 ms. Trace:
I20260709 13:14:58.439219  8058 rpcz_store.cc:276] 0709 13:14:56.379250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.310689 (+931439us) service_pool.cc:224] Handling call
0709 13:14:57.317670 (+  6981us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:58.036835 (+719165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.333278 (+296443us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b5400219654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:58.333677 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1349771136}
I20260709 13:14:58.185607 13493 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:58.248144 13428 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.239s	user 0.002s	sys 0.012s
W20260709 13:14:58.442036 13428 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.433s	user 0.002s	sys 0.012s
I20260709 13:14:58.443229 13428 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:58.443665 13428 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.444069 13428 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:58.158161 13429 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.450877  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:58.147876 13431 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.452826 13437 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:14:58.229169 13476 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.236366 13532 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.457196 13532 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:58.244427 13533 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:58.459203 13533 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.116833 13489 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.476840 13265 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:58.476603 13489 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.122563 13497 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:14:58.139719 13534 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:58.479817  8722 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:14:58.218636 13536 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.847s	user 0.003s	sys 0.009s
W20260709 13:14:58.480717 13423 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
W20260709 13:14:58.177448 13556 log.cc:927] Time spent T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:14:58.379248 13430 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:58.379458  8175 leader_election.cc:400] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 14.
W20260709 13:14:58.484884  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 105778us
I20260709 13:14:58.381392 13440 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:14:58.382189 13564 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:58.385291 13198 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.386730 13364 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:58.387220 13461 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.387527 13569 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:58.387951 13484 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:58.343634  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:58.394315  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.395373 13080 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:58.389946 13028 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:58.430797  8204 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:14:58.482807  8129 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15. Ops: []
W20260709 13:14:58.522966  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.528038 13080 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:58.530930  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 120687us
W20260709 13:14:58.535151  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.513406 13564 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:58.530162 13080 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:58.519285 13569 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:14:58.537088  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 211866us
I20260709 13:14:58.543021 13198 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:58.543292 13224 consensus_queue.cc:1175] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d9cd2999aa314b39997b423ec1ae509e" responder_term: 13 status { last_received { term: 13 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 13 index: 11 } }
I20260709 13:14:58.526947 13440 raft_consensus.cc:697] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:14:58.545161  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:58.546404  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.538658 13564 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:58.539461 13569 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:58.559049 13198 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:58.549479 13430 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:14:58.546247 13219 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.580499 13219 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:58.582171 13430 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:14:58.585934  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.586452 13585 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.587183 13585 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.589843 13585 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:58.553658 13397 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.590729 13397 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.591017 13028 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.592232 13028 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:58.593844 13397 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:58.547042 13461 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.598551 13440 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.601565 13564 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.604990 13219 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.605595 13219 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.607812 13534 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:58.608512 13219 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:58.609009 13534 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.610380 13429 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:58.613664 13461 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:58.602129 13027 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.620075 13489 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:58.631819  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:58.633816  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 85029.1us
I20260709 13:14:58.634685  8721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 913) took 1082 ms. Trace:
I20260709 13:14:58.635207  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:58.635066  8721 rpcz_store.cc:276] 0709 13:14:57.551694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.556741 (+  5047us) service_pool.cc:224] Handling call
0709 13:14:57.557015 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:57.584095 (+ 27080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:57.584139 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:57.602015 (+ 17876us) log.cc:844] Serialized 26 byte log entry
0709 13:14:57.826991 (+224976us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:14:57.827047 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.836705 (+  9658us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:58.627554 (+790849us) raft_consensus.cc:1685] finished
0709 13:14:58.629598 (+  2044us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:58.634479 (+  4881us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26708,"raft.queue_time_us":57454,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":4024,"thread_start_us":188102,"threads_started":1,"wal-append.queue_time_us":484126}
W20260709 13:14:58.640494  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 69709.9us
I20260709 13:14:58.650694  8212 raft_consensus.cc:1240] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 17. Current term is 18. Ops: []
I20260709 13:14:58.653569 13364 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:58.653666  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:58.655663  8059 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:14:58.656042  8059 pending_rounds.cc:85] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 8
I20260709 13:14:58.656070  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1666) took 1020 ms. Trace:
I20260709 13:14:58.656468  8059 pending_rounds.cc:107] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:14:58.656868  8059 raft_consensus.cc:2889] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:58.656646  8129 rpcz_store.cc:276] 0709 13:14:57.635694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.003571 (+367877us) service_pool.cc:224] Handling call
0709 13:14:58.003867 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:58.533412 (+529545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.655766 (+122354us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b54001ab154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:58.655990 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1052325120}
I20260709 13:14:58.663822  8136 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:58.663676  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:58.665555  8026 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:58.666353 13413 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:14:58.666613 13408 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:58.663712  8055 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:14:58.668342  8026 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:58.669358 13290 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:58.669749 13290 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.670074 13290 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:58.670001 13413 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:58.671378 13484 raft_consensus.cc:3055] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:58.672178  8137 raft_consensus.cc:606] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:14:58.672680  8137 raft_consensus.cc:614] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:58.627351 13217 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 13:14:58.666604  8175 leader_election.cc:400] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12.
I20260709 13:14:58.675029  8175 leader_election.cc:403] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:58.676155  8204 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:58.680222 13027 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:58.679939 13570 consensus_queue.cc:1059] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:14:58.681522 13570 raft_consensus.cc:3055] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:58.682255 13261 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.683401 13261 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:58.681546  8208 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:58.671761 13484 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:58.686518  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:58.682075 13570 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:58.693861 13034 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:14:58.694727 13034 consensus_queue.cc:1243] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 14.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:58.696450 13027 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.697113 13027 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.686792 13569 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.698402 13089 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.698635 13564 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:58.688679  8104 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:58.698937 13089 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.699556 13027 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:58.699677  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 50644.2us
I20260709 13:14:58.700054  8723 pending_rounds.cc:85] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Aborting all ops after (but not including) 12
I20260709 13:14:58.700364  8723 pending_rounds.cc:107] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:14:58.700604  8723 raft_consensus.cc:2889] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:58.701439 13533 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:58.701856 13533 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.702193 13533 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:58.697014 13484 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.702522 13364 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:14:58.713372 13089 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:58.714576  8205 raft_consensus.cc:2410] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 18 because replica has last-logged OpId of term: 15 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 6.
I20260709 13:14:58.714758  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:58.715637  8730 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 17: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:14:58.715682 13484 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:58.719705 13569 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:58.722571 13467 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.723121 13467 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.724957  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:58.737426  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:58.734144  8027 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:58.744663  8060 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:58.733747 13585 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12.
I20260709 13:14:58.753774  8208 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
W20260709 13:14:58.764848  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.762814  8029 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:58.764410  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:58.728935 13028 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:14:58.765379 13461 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:14:58.776042  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 88513.5us
I20260709 13:14:58.783514  8730 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:14:58.785559  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:58.776981  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 50453.8us
I20260709 13:14:58.769877 13027 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:58.793042  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:58.793395  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:58.793545 13481 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:58.796211  8026 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:58.796474 13481 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.796988 13481 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:58.799466  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 133049us
W20260709 13:14:58.793045 13595 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:14:58.802574  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:58.802111  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:58.805585  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2211) took 1252 ms. Trace:
I20260709 13:14:58.811447  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:58.813410  8730 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:58.806890  8136 rpcz_store.cc:276] 0709 13:14:57.553164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.892283 (+339119us) service_pool.cc:224] Handling call
0709 13:14:58.028224 (+135941us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b58001c3e98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:14:58.028436 (+   212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:58.664373 (+635937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.794118 (+129745us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54001aac54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:58.805512 (+ 11394us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1500563584}
I20260709 13:14:58.820569 13397 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:58.823809  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:58.824920  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:58.825779 13369 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:14:58.825821  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:58.826436 13369 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:58.826625 13030 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:14:58.827502  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.828281  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:58.827525 13369 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.828961  8056 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:14:58.830137  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.830085  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:58.831008  8057 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:58.826715 13566 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:58.831871 13566 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:58.832787 13566 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.833369 13467 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:58.807691 13429 consensus_queue.cc:1059] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:14:58.819594 13217 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:14:58.835232  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45451.2us
I20260709 13:14:58.817878 13027 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.815073 13532 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:58.838619  8062 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
W20260709 13:14:58.839797 13583 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.269s	user 0.002s	sys 0.000s
I20260709 13:14:58.833777 13428 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:58.852435  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.849711  8056 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:14:58.856058  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:58.858000 13354 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:14:58.858762 13588 log.cc:927] Time spent T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.316s	user 0.002s	sys 0.000s
I20260709 13:14:58.858915  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:58.859790  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:58.865145  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:58.819147 13564 raft_consensus.cc:2764] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:14:58.876259  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.878712  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:58.883679  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:58.879848  8728 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:14:58.887850 13030 raft_consensus.cc:3055] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:58.888106  8026 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:58.888418 13030 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:58.876992 13430 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:14:58.887751  8054 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:58.889683  8026 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:58.883960 13400 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:14:58.890487 13400 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.878739 13216 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:58.806573 13570 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.890769  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:58.890964 13400 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:58.891613 13570 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:58.886044 13428 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:58.879478 13421 consensus_queue.cc:1059] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:14:58.893617  8061 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:58.819249 13585 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:14:58.894145 13421 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:14:58.894651 13421 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:14:58.895414 13589 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.316s	user 0.002s	sys 0.000s
I20260709 13:14:58.891170 13216 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:58.896137  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.895733 13421 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.897214 13421 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:58.897639  8722 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:58.897873 13216 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:58.895412 13030 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.880524  8729 raft_consensus.cc:2410] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:14:58.905326  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.915186  8722 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:14:58.905964  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:58.906783 13027 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:58.887985  8728 pending_rounds.cc:85] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Aborting all ops after (but not including) 10
I20260709 13:14:58.919855  8728 pending_rounds.cc:107] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:14:58.920145  8728 raft_consensus.cc:2889] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:58.909518  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:14:58.895932  8175 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:58.895593  8060 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:14:58.910760  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:58.918263  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:58.894606 13364 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.918673  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:58.924832  8730 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:58.929493  8057 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:58.932322  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:58.928954  8054 pending_rounds.cc:85] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 4
I20260709 13:14:58.933043  8054 pending_rounds.cc:107] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:14:58.933332  8054 raft_consensus.cc:2889] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:58.933684  8054 pending_rounds.cc:107] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 14.6
I20260709 13:14:58.933955  8054 raft_consensus.cc:2889] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:58.939191  8101 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:14:58.940956  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:58.941756  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 79145.5us
I20260709 13:14:58.943496  8129 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 10.
I20260709 13:14:58.945720  8102 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:58.946513 13364 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:58.948249  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:58.948709 13364 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:58.950237  8129 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:58.950609 13090 raft_consensus.cc:2804] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:14:58.951236 13090 raft_consensus.cc:697] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:58.952450 13090 consensus_queue.cc:237] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.930583 13413 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:58.954913  8101 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:58.954921  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:58.956693  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:58.931000 13224 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:58.966926  8212 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:58.902109 13533 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:58.940897 13027 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:14:58.971994  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 82432.9us
I20260709 13:14:58.976485 13534 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:14:58.980664 13592 log.cc:927] Time spent T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.383s	user 0.003s	sys 0.000s
W20260709 13:14:58.980868 13590 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.438s	user 0.002s	sys 0.000s
I20260709 13:14:58.981451 13080 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:58.986276  8101 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:59.001361  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:14:58.966779 13224 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:59.010334  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:14:58.978680 13216 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:14:59.049769  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:58.965544 13481 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:59.052587  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:59.046432  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:59.010178 13364 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:59.011300 13224 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:59.072043 13437 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:59.113418 13398 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:14:59.113998 13398 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:59.114082  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 114663us
I20260709 13:14:59.114403 13398 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:59.115837  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:59.114250  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:59.117578  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 105818us
I20260709 13:14:59.129395  8057 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:59.048349 13354 consensus_queue.cc:1059] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:14:59.070984 13364 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:59.133503  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 157491us
I20260709 13:14:59.130834  8728 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
W20260709 13:14:59.135241  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 134534us
I20260709 13:14:59.151422  8203 raft_consensus.cc:3060] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:14:59.158928  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:59.159368  8176 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:14:59.132187 13566 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:59.162071  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:59.138072 13364 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:59.175086  8101 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:59.180568  8026 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:14:59.166910 13610 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:14:59.178290  8205 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:14:59.137367 13564 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:14:59.168659 13585 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:59.152027 13290 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.200181  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:59.168233  8102 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:59.168816 13440 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:14:59.199309 13585 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:59.260419  8695 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.206498 13484 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:14:59.274674  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 73019.7us
I20260709 13:14:59.211438 13265 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:14:59.279426  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 119570us
I20260709 13:14:59.260792 13585 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:14:59.276924  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:59.289130  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.271936 13533 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:59.307231 13337 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:14:59.307669 13337 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:59.308524 13337 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.311013  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3132) took 1235 ms. Trace:
I20260709 13:14:59.311329  8055 rpcz_store.cc:276] 0709 13:14:58.075496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.077143 (+  1647us) service_pool.cc:224] Handling call
0709 13:14:58.111544 (+ 34401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:58.667946 (+556402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.310747 (+642801us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b540043d354. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:14:59.310950 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1259518720}
I20260709 13:14:59.312863  8055 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:14:59.315513 13476 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:59.290752  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:59.318481  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 182520us
I20260709 13:14:59.290045 13265 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:59.291930 13609 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 13:14:59.293947 13614 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:14:59.293876  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:14:59.299546 13605 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:14:59.300293 13481 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:14:59.301452 13608 log.cc:927] Time spent T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:14:59.275134 13090 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:14:59.320847  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 167765us
I20260709 13:14:59.289189  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:14:59.324118  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 46339.8us
I20260709 13:14:59.330096  8057 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:14:59.330327  8102 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:59.346217 13290 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:59.350814  8061 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:14:59.355103 13627 consensus_queue.cc:1059] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:14:59.356078 13627 raft_consensus.cc:3055] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:59.356431 13627 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:59.357375 13627 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.358680  8061 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:14:59.362488  8101 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:14:59.319715 13265 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:59.367319  8102 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:59.369694  8059 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:14:59.372267 13587 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.168s
I20260709 13:14:59.373425 13587 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:59.370833 13090 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:59.374715 13532 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:59.375061 13379 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:14:59.375456 13533 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:14:59.376106 13533 consensus_queue.cc:1243] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:14:59.377225 13533 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:59.377614 13533 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:14:59.385493 13613 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:14:59.386894 13497 consensus_queue.cc:1059] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:59.387212 13627 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:59.387840 13497 raft_consensus.cc:3055] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:59.388167 13497 raft_consensus.cc:740] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:14:59.389010 13497 consensus_queue.cc:260] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:59.390228  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 3131) took 1315 ms. Trace:
I20260709 13:14:59.390771 13566 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:59.391220  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:59.394775  8102 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:14:59.395797 13400 consensus_meta.cc:287] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.504s	user 0.011s	sys 0.000s
I20260709 13:14:59.395838  8102 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:14:59.396365 13400 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:59.396760  8725 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:59.397536 13397 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:14:59.397785 13618 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:14:59.398059 13532 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:59.398725 13532 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:59.399199 13532 raft_consensus.cc:697] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:59.399844 13402 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:59.400419 13354 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:14:59.398072 13397 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:59.365525 13034 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:14:59.403668 13585 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:59.404618 13593 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.868s	user 0.007s	sys 0.025s
I20260709 13:14:59.405110  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:59.405282  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:59.405907  8728 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:14:59.406039 13440 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:59.402195 13354 raft_consensus.cc:3055] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:14:59.406920 13354 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:14:59.400159 13532 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.290482 13219 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:14:59.402156 13397 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.373961 13587 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:14:59.407397  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:59.414582 13400 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:59.415175 13440 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:59.417374  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:59.407788 13354 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.420918 13354 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:14:59.408193  7956 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34146 (request call id 259) took 31964 ms (client timeout 15000 ms). Trace:
I20260709 13:14:59.411003 13407 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.424355 13407 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:59.411536  8209 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:14:59.427789 13407 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:59.428220  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:59.429454  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:59.430488  8203 raft_consensus.cc:2468] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:14:59.415228 13587 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:59.432349 13587 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:59.423185  7956 rpcz_store.cc:269] 0709 13:14:27.443789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.452371 (+  8582us) service_pool.cc:224] Handling call
0709 13:14:59.407946 (+31955575us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:50.455315 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:50.456156 (+   841us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:50.456214 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:14:50.456240 (+    26us) tablet.cc:662] Decoding operations
0709 13:14:50.457439 (+  1199us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:50.457547 (+   108us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:50.457572 (+    25us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:14:50.463796 (+  6224us) tablet.cc:701] Row locks acquired
0709 13:14:50.463846 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:14:50.463999 (+   153us) write_op.cc:270] Start()
0709 13:14:50.464151 (+   152us) write_op.cc:276] Timestamp: P: 1783602890463937 usec, L: 0
0709 13:14:50.464207 (+    56us) op_driver.cc:348] REPLICATION: starting
0709 13:14:50.465057 (+   850us) log.cc:844] Serialized 3999 byte log entry
0709 13:14:51.021688 (+556631us) op_driver.cc:464] REPLICATION: finished
0709 13:14:51.129630 (+107942us) write_op.cc:301] APPLY: starting
0709 13:14:51.129774 (+   144us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:51.238319 (+108545us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:51.238481 (+   162us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:51.248449 (+  9968us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:51.248766 (+   317us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 13:14:51.248835 (+    69us) write_op.cc:312] APPLY: finished
0709 13:14:51.249237 (+   402us) log.cc:844] Serialized 97 byte log entry
0709 13:14:51.249425 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:51.249553 (+   128us) write_op.cc:454] Released schema lock
0709 13:14:51.249616 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:51.250016 (+   400us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:51.250050 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":605,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":72185,"apply.run_cpu_time_us":14864,"apply.run_wall_time_us":120413,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45927,"num_ops":10,"prepare.queue_time_us":252985,"prepare.run_cpu_time_us":46402,"prepare.run_wall_time_us":187597,"raft.queue_time_us":164762,"raft.run_cpu_time_us":5461,"raft.run_wall_time_us":48141,"replication_time_us":557287,"thread_start_us":516083,"threads_started":4,"wal-append.queue_time_us":348418,"wal-append.run_cpu_time_us":34659,"wal-append.run_wall_time_us":1830403}]]}
W20260709 13:14:59.434628  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 62961.8us
I20260709 13:14:59.434832  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:14:59.435505 13606 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.322s	user 0.003s	sys 0.000s
I20260709 13:14:59.435662  8211 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 18: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:14:59.437203  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.432420  8203 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:14:59.440230  8212 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:59.437604  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:59.444615  8209 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:59.390667  8062 rpcz_store.cc:276] 0709 13:14:58.075072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.076534 (+  1462us) service_pool.cc:224] Handling call
0709 13:14:59.389985 (+1313451us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b540033efd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:14:59.390143 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1734590848}
W20260709 13:14:59.478701  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 61399.2us
I20260709 13:14:59.444223 13440 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:59.414326 13461 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.444083 13413 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:59.491982  8723 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:14:59.496007  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:59.481122 13497 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:59.501535  8063 pending_rounds.cc:85] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 11
I20260709 13:14:59.507179  8063 pending_rounds.cc:107] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:14:59.507510  8063 raft_consensus.cc:2889] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:59.501832 13431 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.004s	sys 0.024s
I20260709 13:14:59.446846 13219 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:14:59.412395 13224 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.495597  8723 pending_rounds.cc:85] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Aborting all ops after (but not including) 6
I20260709 13:14:59.522168  8175 leader_election.cc:400] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12.
W20260709 13:14:59.511673 13431 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.004s	sys 0.024s
I20260709 13:14:59.496879 13413 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:14:59.465441 13533 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.517508  8723 pending_rounds.cc:107] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 10.7
W20260709 13:14:59.628574 13421 consensus_meta.cc:287] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.731s	user 0.010s	sys 0.004s
I20260709 13:14:59.633147  8723 raft_consensus.cc:2917] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Aborting config change with OpId 10.7: Aborted: Op aborted by new leader
W20260709 13:14:59.655491  8208 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.023s	sys 0.023s
W20260709 13:14:59.655997  8208 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.023s	sys 0.023s
I20260709 13:14:59.660887  8723 raft_consensus.cc:2889] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:59.630007 13461 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.665166 13407 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:14:59.675856 13585 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:14:59.703034  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 207670us
I20260709 13:14:59.675014 13090 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:14:59.698541 13427 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.682082 13364 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:59.712493  8723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:38710 (request call id 5) took 11652 ms. Trace:
I20260709 13:14:59.702526 13421 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.717870  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:59.712085 13476 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:14:59.734313 13476 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.736336 13476 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:59.736819 13638 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.737349 13638 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:59.739401 13638 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:59.739665  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:59.729326 13398 consensus_meta.cc:287] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.612s	user 0.009s	sys 0.002s
I20260709 13:14:59.744441  8727 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:14:59.744057  8209 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:59.705968 13407 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.748322  8135 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
W20260709 13:14:59.750052 13632 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:14:59.714856  8723 rpcz_store.cc:276] 0709 13:14:48.048510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.065326 (+ 16816us) service_pool.cc:224] Handling call
0709 13:14:48.253329 (+188003us) log.cc:844] Serialized 494 byte log entry
0709 13:14:59.690359 (+11437030us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35851,"raft.queue_time_us":37496,"raft.run_cpu_time_us":1266,"raft.run_wall_time_us":1265,"thread_start_us":91891,"threads_started":1,"wal-append.queue_time_us":253132,"wal-append.run_cpu_time_us":2086,"wal-append.run_wall_time_us":1022029}
W20260709 13:14:59.754199 13629 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:14:59.754550 13398 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.786525 13398 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:59.786852 13585 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:59.787089 13398 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:59.787724 13398 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.790004 13533 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:59.790513 13398 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:59.791535  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:59.792220  8135 raft_consensus.cc:2393] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:14:59.793016  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:59.758476  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:59.760576  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:59.761200  8131 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:14:59.743172 13421 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:14:59.762897 13631 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:14:59.762997  8132 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 14. Ops: []
I20260709 13:14:59.763118  8134 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:14:59.763278 13224 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:59.764498  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:59.763803 13413 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.795420  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:59.796573  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:59.766775 13090 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:59.770649  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:59.744984 13427 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.859270  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:59.780365 13030 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:14:59.782861 13570 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:59.758435  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.800863  8137 raft_consensus.cc:2410] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:14:59.833555  8212 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:14:59.858764  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:59.812345  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:59.867667  8132 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:59.796451  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:59.819427  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.803696 13364 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:59.808704 13421 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:59.861606  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:14:59.868472  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:59.863927  8136 raft_consensus.cc:2368] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 14. Current term is 15.
I20260709 13:14:59.796769 13585 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:14:59.805035 13461 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:59.794306  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:59.866417  8212 pending_rounds.cc:85] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 6
I20260709 13:14:59.877606  8212 pending_rounds.cc:107] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:14:59.877946  8212 raft_consensus.cc:2889] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:59.864324 13198 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.810639 13407 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:59.876452  8134 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
I20260709 13:14:59.875895  8138 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:14:59.879181  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:59.879786  8724 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:14:59.880831  8203 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:14:59.881989 13198 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:14:59.882369  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:59.884960  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:59.885648  8130 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:59.873365 13587 raft_consensus.cc:2764] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:14:59.889046  8693 leader_election.cc:400] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 14. Current term is 15.
I20260709 13:14:59.887694  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:14:59.814972  7946 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a003b4584d3747a381522d67299aaed2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) due to error UNKNOWN_ERROR; will retry: Aborted: Op aborted by new leader
I20260709 13:14:59.868310 13570 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:14:59.889724  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.877305 13585 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:14:59.910988 13642 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.911144  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:14:59.919881 13642 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:59.921990 13642 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:14:59.922876 13585 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:14:59.908883  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 113549us
I20260709 13:14:59.924458  8134 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:59.885936 13461 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:14:59.889001  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:59.918537  8693 leader_election.cc:403] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:59.890413  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:14:59.889605  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 2089) took 1047 ms. Trace:
I20260709 13:14:59.932926  8135 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
W20260709 13:14:59.933152  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.934834  8130 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
W20260709 13:14:59.935387  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 46542.5us
I20260709 13:14:59.935642  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:14:59.934617  7946 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 119876us
W20260709 13:14:59.960377  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.931874  8208 rpcz_store.cc:276] 0709 13:14:58.842674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.842819 (+   145us) service_pool.cc:224] Handling call
0709 13:14:58.854436 (+ 11617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:58.854613 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:14:59.656386 (+801773us) log.cc:844] Serialized 14 byte log entry
0709 13:14:59.889313 (+232927us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:59.889372 (+    59us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:14:59.889410 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.889466 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:59.889559 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123223,"prepare.queue_time_us":275603,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":84,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":5272,"thread_start_us":904803,"threads_started":2,"wal-append.queue_time_us":222803}
I20260709 13:14:59.943295 13497 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.966130 13585 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:59.967065 13427 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:59.968142 13585 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:14:59.962138 13497 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:14:59.969619  8175 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:14:59.970412 13628 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:59.971024  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:14:59.971789 13628 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:59.972190 13628 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:14:59.972608 13628 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:59.972304  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:59.982033  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:14:59.983093 13413 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:14:59.947362 13587 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:14:59.983676  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:14:59.984336  8722 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:14:59.936595 13636 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:14:59.986157  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:14:59.991122  8026 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:14:59.972749 13369 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:59.996979  8026 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:14:59.999653 13533 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.000198 13533 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.001595 13570 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:00.001922  8725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 968) took 1233 ms. Trace:
I20260709 13:15:00.002518 13481 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:00.010488  8206 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:00.012919  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:00.013496  8722 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:00.002269  8725 rpcz_store.cc:276] 0709 13:14:58.768842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.789618 (+ 20776us) service_pool.cc:224] Handling call
0709 13:15:00.001727 (+1212109us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b5400582854. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:00.001845 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1386798464}
I20260709 13:15:00.022141  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:00.023901  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 146716us
I20260709 13:14:59.989486 13587 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:15:00.025509  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 53308.1us
I20260709 13:15:00.023914  8722 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
W20260709 13:15:00.023605 13028 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.012s	sys 0.006s
I20260709 13:15:00.030730  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:00.033545  8026 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:00.034873  8029 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:00.002223 13533 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:00.038491  8135 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:00.039413  8102 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:00.023054 13567 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:15:00.049268  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 183777us
W20260709 13:15:00.033720 13028 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.012s	sys 0.006s
I20260709 13:15:00.050211 13655 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:59.992865 13497 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:00.053694  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:00.050145  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 2086) took 1363 ms. Trace:
I20260709 13:15:00.054594  8208 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:00.033524 13570 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.064189  8102 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:00.064411  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:00.061029  8206 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:15:00.034224 13407 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:15:00.070057  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:00.067953  8696 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:00.075439 13398 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:00.075877 13398 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:00.076205 13398 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:00.066069 13027 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:00.078888 13650 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:00.069868 13379 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:15:00.080392 13379 consensus_queue.cc:1243] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:15:00.080286 13570 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.060747 13429 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:00.065811 13655 raft_consensus.cc:697] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:00.067955 13652 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:15:00.100615  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:15:00.089180  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 53175.8us
I20260709 13:15:00.096086  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:00.089537  8132 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:15:00.097965  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:00.121554  8057 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:00.097544  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:00.097651  8725 raft_consensus.cc:1275] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:15:00.098440  8062 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
W20260709 13:15:00.118418 13648 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:15:00.150168 13481 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:00.139094 13090 consensus_queue.cc:1048] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.012s
W20260709 13:15:00.150967 13617 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:15:00.147568  8176 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:00.146898  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:00.056839  8204 rpcz_store.cc:276] 0709 13:14:58.686752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.686919 (+   167us) service_pool.cc:224] Handling call
0709 13:14:58.691930 (+  5011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:58.692059 (+   129us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:14:59.829249 (+1137190us) spinlock_profiling.cc:243] Waited 6.66 ms on lock 0x7b180048d120. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf
0709 13:14:59.829426 (+   177us) log.cc:844] Serialized 14 byte log entry
0709 13:15:00.049789 (+220363us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:00.049838 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:00.049889 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:00.049946 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:00.050063 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":285663,"prepare.queue_time_us":436285,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":6714,"raft.queue_time_us":3572,"spinlock_wait_cycles":15323392,"thread_start_us":403548,"threads_started":2,"wal-append.queue_time_us":220353}
I20260709 13:15:00.159624  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 779) took 1147 ms. Trace:
I20260709 13:15:00.160012  8727 rpcz_store.cc:276] 0709 13:14:59.012197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.134477 (+122280us) service_pool.cc:224] Handling call
0709 13:15:00.159442 (+1024965us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b540060de54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:00.159548 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1113210112}
I20260709 13:15:00.144326 13587 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:00.162160 13587 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:00.125861 13570 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:00.162561 13587 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:00.165511  8727 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:15:00.143649  8056 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:15:00.144089  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 54238.9us
I20260709 13:15:00.168201  8060 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:15:00.168946  8176 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:15:00.166808  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 45001.5us
I20260709 13:15:00.085618 13429 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:00.136515  8207 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3311) took 1219 ms. Trace:
I20260709 13:15:00.177229  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:00.061935  8027 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:00.182027  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:00.182637  8063 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:00.190683  8175 leader_election.cc:400] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16.
I20260709 13:15:00.191126  8175 leader_election.cc:403] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:00.175354  8207 rpcz_store.cc:276] 0709 13:14:58.917177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.688925 (+771748us) service_pool.cc:224] Handling call
0709 13:14:59.689337 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:59.703821 (+ 14484us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:14:59.703865 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:59.715794 (+ 11929us) log.cc:844] Serialized 26 byte log entry
0709 13:14:59.943153 (+227359us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:59.943190 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.943250 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:00.136211 (+192961us) raft_consensus.cc:1685] finished
0709 13:15:00.136236 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:00.136356 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11003,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":4178,"thread_start_us":201002,"threads_started":1,"wal-append.queue_time_us":308494}
I20260709 13:15:00.193049  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:00.193931  8056 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
W20260709 13:15:00.195945  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 135123us
I20260709 13:15:00.200706  8060 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:15:00.178719 13585 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:00.181155 13216 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:00.177942 13429 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:00.093793 13654 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:00.178107 13637 raft_consensus.cc:3055] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:00.195814 13659 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:00.097734 13628 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:00.212348  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:00.214088 13628 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:00.214629 13421 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:00.193408 13652 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.215646 13652 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.215183 13421 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:00.217849 13652 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:00.212345 13659 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:00.218869 13659 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:00.219570 13421 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:00.224189 13398 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:00.225729 13587 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.200507 13030 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:00.226770  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 201799us
I20260709 13:15:00.230307  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:00.230641  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:00.247926  8101 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:00.231222  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:00.231513  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:00.215144 13655 consensus_queue.cc:237] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.196010 13440 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16.
I20260709 13:15:00.205447 13481 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:15:00.235231  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:00.236420 13369 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:00.237622 13198 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:00.292562 13421 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:00.293217 13421 raft_consensus.cc:697] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:00.293149 13407 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:00.294370 13421 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:00.237764  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:00.296345  8055 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:00.238397  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:00.183334 13402 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:15:00.238591  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:00.239004 13652 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.230705  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:00.209831 13585 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:00.243328  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:00.244764  8057 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:00.226253 13398 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:00.246369 13636 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 14. Current term is 15.
I20260709 13:15:00.210882 13637 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:00.248394  8027 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:00.174373 13570 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:15:00.280953  8133 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:15:00.287308  8056 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:15:00.288687  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 58118.5us
I20260709 13:15:00.287532  8058 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:15:00.289340  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:00.291733  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:00.291527 13369 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:00.290721  8026 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:15:00.308054  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 60200.7us
I20260709 13:15:00.316893 13216 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:00.211997 13654 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:00.307613  8055 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:15:00.252867 13627 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:00.308728  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:00.316367  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 68049.4us
I20260709 13:15:00.321755  8101 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:00.316367 13397 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:00.292562 13534 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:15:00.317471 13400 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:00.332657  8063 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:00.319476 13407 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:00.328042  8026 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:00.342216  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:00.343139  8026 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:00.315771 13030 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.330551 13413 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:15:00.308672  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 237340us
I20260709 13:15:00.346908  8104 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:00.314498 13652 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.347149  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:00.335822 13654 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:00.349756  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 106525us
I20260709 13:15:00.336189 13430 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:00.341962 13090 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:00.354537  8101 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:00.341342 13397 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:00.346266 13027 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:00.346024 13481 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:00.363929  8026 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:00.350675 13224 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:00.357282 13407 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:00.344111 13627 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:00.363078  8104 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:00.377061  8175 leader_election.cc:400] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12.
I20260709 13:15:00.379894  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:00.349336 13497 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:15:00.380348  8175 leader_election.cc:403] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:00.365296 13027 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:00.388231 13027 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:00.388352 13567 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:00.390470 13440 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 11. Current term is 12.
I20260709 13:15:00.381750 13216 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:00.375061 13431 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:00.384195  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:00.386960  8101 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:00.387686  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:00.397173  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:00.397805  8723 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:00.375842 13413 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:00.396898  8101 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:00.367799 13660 consensus_queue.cc:1059] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:15:00.383837  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:00.383473 13570 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:15:00.413203  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 50199.7us
I20260709 13:15:00.410178 13034 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:00.421094  8101 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:00.409477 13587 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:00.380573 13585 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:00.446874 13030 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:00.449165 13397 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.450074  8057 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:15:00.443392 13265 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:00.454919 13265 raft_consensus.cc:697] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:00.455821 13265 consensus_queue.cc:237] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:00.457319 13652 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:00.413843 13628 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:00.412251 13430 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:00.462878 13430 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:00.463337 13430 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:00.463428 13671 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:15:00.413738 13662 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:00.464634 13662 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:00.464999 13662 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:00.415957 13534 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:00.413777 13570 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:00.425127 13034 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:00.456219 13369 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:00.444552 13627 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:00.527504  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:00.566596  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 152490us
I20260709 13:15:00.567946  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:00.568249  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:15:00.569253  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 181624us
I20260709 13:15:00.472762 13570 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:00.521435 13034 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:00.572541  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:00.581425  8057 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:15:00.573256  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:00.582700  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:00.577730  8728 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:00.583011  8211 raft_consensus.cc:1275] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:15:00.584061  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:00.585182  8055 raft_consensus.cc:2410] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:15:00.560267 13497 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.559358 13532 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:00.593751  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:00.595114 13627 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:00.595893  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:00.596837  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:00.597589  8208 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
W20260709 13:15:00.598627  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 218836us
I20260709 13:15:00.604916  8210 raft_consensus.cc:3055] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:00.605419  8210 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:00.613940  8209 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:15:00.614281  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:00.613730  8205 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 14. (term mismatch)
I20260709 13:15:00.623777  8205 pending_rounds.cc:85] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 13
I20260709 13:15:00.624132  8205 pending_rounds.cc:107] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 14.14
I20260709 13:15:00.624418  8205 raft_consensus.cc:2889] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:00.627195  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:00.647018  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:00.647794  8205 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:00.650880  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:00.652976 13400 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:15:00.655527  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 72915.9us
I20260709 13:15:00.602703 13497 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.658869 13627 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.659415 13627 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:00.661562 13627 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:00.667016 13290 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.667486 13290 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.669641 13290 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:00.671485  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:00.608511 13028 consensus_queue.cc:1048] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:15:00.656838  8210 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.612217 13532 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:00.686411  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 73742us
I20260709 13:15:00.692842  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:00.693594  8138 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:00.694619  8208 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:15:00.695729  8210 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:00.696605  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:15:00.688942 13670 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:15:00.734724  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:00.735371  8204 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:15:00.736213  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:00.736709 13532 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:00.736964  8204 raft_consensus.cc:2410] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:15:00.737939  8204 raft_consensus.cc:3055] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:15:00.689092 13673 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:15:00.650322 13027 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.740784 13027 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:00.709393  8207 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:00.738212  8204 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:00.746173  8204 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.688998 13678 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:00.695499 13682 log.cc:927] Time spent T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:15:00.715605 13497 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:00.719033  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:00.065408  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:00.723349  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:00.724341 13672 log.cc:927] Time spent T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.358s	user 0.000s	sys 0.002s
I20260709 13:15:00.746305  8723 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:00.728096 13431 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:00.731889  8101 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:00.807017  8205 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
W20260709 13:15:00.807597  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 75808.4us
I20260709 13:15:00.757208 13627 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:00.809156  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:00.809708  8132 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:00.810295  8138 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:00.747064  8204 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:00.757019  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 64394.2us
I20260709 13:15:00.770480  8130 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:15:00.749488  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:00.769968  8209 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:15:00.812254  8208 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:00.747990 13659 consensus_meta.cc:287] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.000s
I20260709 13:15:00.817862 13560 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.818326  8205 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:00.808657 13627 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:00.820832  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:00.816293 13638 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:15:00.849464  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:00.896317  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:00.898633  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 78435.4us
I20260709 13:15:00.753729 13678 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.902261  8132 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
W20260709 13:15:00.838835 13429 consensus_meta.cc:287] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.000s
I20260709 13:15:00.902293  8210 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:15:00.905970  8210 pending_rounds.cc:85] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 6
I20260709 13:15:00.906244  8210 pending_rounds.cc:107] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:15:00.906505  8210 raft_consensus.cc:2889] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:00.748885 13430 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:00.834828 13627 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:00.754714 13034 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.909403  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:00.899544 13652 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:00.834698 13560 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:00.893230 13638 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:00.834296 13644 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
W20260709 13:15:00.921941  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:00.830101 13659 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:00.916590 13652 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:00.920528 13638 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:00.911422 13429 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:00.906735 13427 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:15:00.911230 13662 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:00.931077 13028 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.937922 13652 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:00.972514 13659 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:00.973665 13560 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:00.973965 13678 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:00.980494  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:00.981065  8132 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:00.982205  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:01.010993  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:01.015472 13027 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:01.018193 13027 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.020388 13027 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:01.027870 13570 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:01.029067  8137 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 16: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:01.031754 13429 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:01.004797 13654 consensus_meta.cc:287] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.645s	user 0.010s	sys 0.000s
I20260709 13:15:01.037530  8138 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:01.035202  8208 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:01.037276 13216 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:01.032126 13627 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:01.010967 13678 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:00.981878 13585 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:01.012482 13497 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:01.056775 13430 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:01.041889 13627 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:01.057976 13678 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:01.060429 13678 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:01.060571 13662 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:00.979565 13028 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.063545  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:01.067180  8058 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:01.067898  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:01.072332  8132 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
W20260709 13:15:01.072695  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 89567.4us
I20260709 13:15:01.044056 13654 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.066075 13430 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:01.067553 13423 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.095099  8204 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:15:01.095655  8204 pending_rounds.cc:85] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 7
I20260709 13:15:01.095944  8204 pending_rounds.cc:107] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:15:01.096238  8204 raft_consensus.cc:2889] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:01.064139 13497 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.101580 13028 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:01.102562 13652 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:01.104485 13652 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:01.115835  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:01.116935  8061 raft_consensus.cc:2410] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:15:01.114985  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:01.126736  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:01.127429  8055 raft_consensus.cc:2368] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14.
I20260709 13:15:01.128098  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:01.128870  8060 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:15:01.127130  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:01.130558  8062 raft_consensus.cc:1813] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Handling vote request from an unknown peer d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:01.130986  8062 raft_consensus.cc:2368] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:15:01.132627  8056 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:15:01.132961  8056 pending_rounds.cc:85] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 8
I20260709 13:15:01.132938  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:01.133422  8056 pending_rounds.cc:107] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:15:01.133685  8056 raft_consensus.cc:2889] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:01.134577  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:01.089774 13430 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.153393  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:01.091136 13423 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.160250  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:01.163038  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:01.132489 13198 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:01.167583  8057 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:15:01.152347 13570 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:01.170045  8060 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:01.173647  8056 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:01.174239  8054 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:15:01.175298  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:01.175995  8129 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:01.176167 13423 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:01.177034  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:01.177592  8129 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:01.171523 13198 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.184808  8129 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:01.185962 13198 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:01.186403  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:01.186709 13687 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:15:01.186915  8129 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:15:01.187116  8132 raft_consensus.cc:2393] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:01.188238  8138 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:01.189849  8138 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:15:01.191565  8138 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:15:01.192806  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:01.193452 13638 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:01.194478  8138 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:01.195497 13654 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:01.195675  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:01.196369  8138 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:01.197377  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:01.198680  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:01.201092  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:01.201771  8138 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 16: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:01.202577  8129 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:15:01.203929  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:01.205389  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:01.205989  8175 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:01.206908 13216 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:01.210588 13216 consensus_queue.cc:1059] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:01.211486 13400 raft_consensus.cc:3055] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:01.211827 13400 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:01.212574 13400 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:01.213104 13034 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:01.214161 13400 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:01.217038 13224 raft_consensus.cc:3055] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:01.217511 13224 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:01.220598 13407 raft_consensus.cc:3055] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:01.220983 13407 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:01.221834 13407 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.222829 13407 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:01.222920 13337 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:01.224192 13585 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:01.226284 13198 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.226788 13198 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.228705 13198 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:01.233384  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:01.233975  8726 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:01.205417  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:01.190083 13497 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:01.249445  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:01.222983 13400 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 13:15:01.253904 13667 log.cc:927] Time spent T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.896s	user 0.003s	sys 0.000s
I20260709 13:15:01.255635  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:01.256501  8130 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:15:01.262562  8135 raft_consensus.cc:2368] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14.
I20260709 13:15:01.263592  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:01.257011 13400 consensus_queue.cc:1243] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:15:01.272220  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.274619  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:01.275149  8054 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:01.279351  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:01.280115  8728 raft_consensus.cc:2393] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 18: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:01.281399 13430 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:01.281757  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.284126  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:01.284282  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:01.284756  8057 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:01.238813 13627 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:15:01.287045  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:01.287654  8060 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:01.288661 13627 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:01.271445 13224 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.289170 13627 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:01.289851 13224 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:01.291504  8055 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:01.293900  8725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 981) took 1140 ms. Trace:
I20260709 13:15:01.294284  8725 rpcz_store.cc:276] 0709 13:15:00.153715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:00.154009 (+   294us) service_pool.cc:224] Handling call
0709 13:15:00.154321 (+   312us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:00.154681 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:00.154709 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:00.155245 (+   536us) log.cc:844] Serialized 52 byte log entry
0709 13:15:00.304329 (+149084us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:00.304382 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:00.304466 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:01.253349 (+948883us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:15:01.293646 (+ 40297us) raft_consensus.cc:1685] finished
0709 13:15:01.293668 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:01.293850 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6946,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":74,"spinlock_wait_cycles":23278464,"thread_start_us":148527,"threads_started":1,"wal-append.queue_time_us":175404}
I20260709 13:15:01.297821  8058 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:01.300119  8175 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:01.301352  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:01.282317  8102 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:01.303237  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:01.304263  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:01.304687  8208 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:15:01.305321  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:01.305795  8211 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:01.306905  8205 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:01.319478  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:01.321789  8175 leader_election.cc:400] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14.
I20260709 13:15:01.325898  8057 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:01.326100  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:01.323952  8029 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:01.326668  8131 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:01.330894  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:01.331184  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:01.331619  8728 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 18: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:01.331722  8209 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:15:01.333007  8134 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:01.333896  8725 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:15:01.325312  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:01.335140  8207 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:01.311237 13655 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.005s	sys 0.015s
W20260709 13:15:01.339250 13655 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.005s	sys 0.015s
I20260709 13:15:01.340953  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:01.343528  8132 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:15:01.343979  8132 pending_rounds.cc:85] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 12
I20260709 13:15:01.344238  8132 pending_rounds.cc:107] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:15:01.344512  8132 raft_consensus.cc:2889] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:01.349514  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:01.353214  8029 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:01.365984 13712 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:01.366214  8176 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:01.367689  8696 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:01.398422  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:01.318104 13400 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:01.379407  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:01.320250 13570 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:01.380038  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:01.391034  8729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 979) took 1308 ms. Trace:
W20260709 13:15:01.399359  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 46244.2us
W20260709 13:15:01.399580  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 51443.1us
I20260709 13:15:01.400122  8138 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
W20260709 13:15:01.401793  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 73555.1us
I20260709 13:15:01.412474  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:01.405843  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:01.413460  8725 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:01.401525  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:15:01.406800  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.407699  8696 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:01.415802  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:01.340590 13402 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:01.425797  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 108771us
I20260709 13:15:01.426707  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:01.429697  8696 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:01.431063  8054 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:01.409274 13655 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:01.409404 13677 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:01.405645 13716 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:01.410204 13633 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:15:01.584918  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 185657us
I20260709 13:15:01.424258 13570 raft_consensus.cc:697] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:15:01.591208  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 155232us
I20260709 13:15:01.428231 13430 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:01.586396  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:01.599927  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:01.438242 13476 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:01.603636  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:01.444363 13423 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:01.573810 13677 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.616074 13677 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:01.598477 13655 raft_consensus.cc:697] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:01.617595 13655 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:01.575261 13219 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:15:01.431800 13674 consensus_queue.cc:1059] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.172s
I20260709 13:15:01.628863 13674 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:01.429467 13402 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:01.629359 13674 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.599563 13638 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:01.598672 13427 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:01.632575  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:01.632640 13427 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.575587 13633 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:01.633678 13633 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:15:01.634140 13713 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.264s	user 0.000s	sys 0.007s
I20260709 13:15:01.634919 13427 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:01.635452 13427 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:01.635766 13427 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:01.636538 13427 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.637674 13427 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:01.591274 13654 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:01.598989 13430 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.613621 13423 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.623636 13570 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.574302 13716 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.655575 13716 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:01.657642 13674 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:01.658882 13660 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:01.613054 13476 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:01.642827  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:01.643865  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:01.651927  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:01.631198 13402 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:01.639055 13654 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.655086 13678 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:01.662789  8207 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:01.668993  8205 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:15:01.613422 13627 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:01.669031  8208 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:15:01.652429 13423 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:01.680728  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 254120us
I20260709 13:15:01.680996  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:01.683614  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:01.651101 13430 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:01.668627 13216 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:01.674208 13476 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:01.687611  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.679023 13660 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:01.574975 13637 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:01.696146  8175 leader_election.cc:400] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:15:01.705914 13627 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:01.709700 13216 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:01.746448 13637 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:01.705543 13678 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:01.414880  8729 rpcz_store.cc:276] 0709 13:15:00.083135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:00.083255 (+   120us) service_pool.cc:224] Handling call
0709 13:15:00.084612 (+  1357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:00.084797 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:15:00.211236 (+126439us) log.cc:844] Serialized 14 byte log entry
0709 13:15:01.220391 (+1009155us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b180046b220. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb623a
0709 13:15:01.375915 (+155524us) log.cc:844] Serialized 14 byte log entry
0709 13:15:01.390566 (+ 14651us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:01.390639 (+    73us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:15:01.390708 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:01.390795 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:01.390962 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":383481,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":13030,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":3756,"spinlock_wait_cycles":3154304,"thread_start_us":404745,"threads_started":3,"wal-append.queue_time_us":169453}
I20260709 13:15:01.753386 13677 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:01.793727  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:01.804477  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:01.765600 13337 consensus_meta.cc:287] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.000s
I20260709 13:15:01.762868 13413 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:01.764415 13216 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:01.909300  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.794430 13633 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.804177 13627 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:01.986192  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.031213  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3344) took 1438 ms. Trace:
I20260709 13:15:02.034584 13660 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.052558  8206 rpcz_store.cc:276] 0709 13:15:00.592379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:00.596677 (+  4298us) service_pool.cc:224] Handling call
0709 13:15:00.597002 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:00.597211 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:15:00.805208 (+207997us) log.cc:844] Serialized 14 byte log entry
0709 13:15:01.350357 (+545149us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b180047aa60. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f29522 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:15:02.030645 (+680288us) log.cc:844] Serialized 14 byte log entry
0709 13:15:02.030848 (+   203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:02.030898 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:02.030943 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:02.031023 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:02.031151 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1728963,"prepare.queue_time_us":1408440,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":2021,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":2639,"spinlock_wait_cycles":4097280,"thread_start_us":303064,"threads_started":2,"wal-append.queue_time_us":249772}
I20260709 13:15:02.059305 13660 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.060298 13430 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.034683 13585 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:02.062902 13430 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:02.063325 13585 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:02.053086 13413 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.032858 13423 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.090643  8696 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:02.091150 13497 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.091645 13497 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.091888  7957 catalog_manager.cc:5697] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 11 to 14, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 14 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:02.093587 13497 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:02.099555  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:02.103305  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:02.103894  8212 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:02.085785  8725 consensus_meta.cc:287] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.663s	user 0.008s	sys 0.004s
I20260709 13:15:02.109498  8725 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
W20260709 13:15:02.074617  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 378583us
I20260709 13:15:02.079427  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:02.111007  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:02.047852  8724 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:02.036643 13476 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:01.757688 13402 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:15:02.047528  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.053561  8131 consensus_meta.cc:287] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.718s	user 0.006s	sys 0.005s
I20260709 13:15:02.058605 13633 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.058712 13627 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:02.065673  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:02.137565  8131 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:01.765537 13290 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:15:02.156232  8178 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.053s	sys 0.160s
W20260709 13:15:02.171748  8178 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.053s	sys 0.161s
I20260709 13:15:02.070571 13585 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.157557 13654 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.075723 13413 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:02.158119 13476 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.079582 13431 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:02.158938 13716 consensus_meta.cc:287] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.503s	user 0.012s	sys 0.000s
I20260709 13:15:02.209757  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.210839  8135 raft_consensus.cc:2410] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:15:02.211516  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.093600  7957 catalog_manager.cc:5697] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 10 to 13, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 13 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.085673 13423 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:02.160982  8724 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55838 (request call id 785) took 1637 ms (client timeout 1621 ms). Trace:
I20260709 13:15:02.216843  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.218011  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:02.218755  8207 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:02.162370  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 51434us
I20260709 13:15:02.164338 13660 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.164409 13560 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.166145  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:02.127864 13028 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:02.167339  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 88031.5us
I20260709 13:15:02.158119  8104 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:02.225221  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 67168.6us
I20260709 13:15:02.198711 13461 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.209224 13497 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.227839 13461 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.227963 13497 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.205500 13653 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:02.229095 13653 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.229807 13461 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.205292 13407 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.230348 13660 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.230754 13407 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.230916 13660 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.232772 13407 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.232983 13660 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.233271 13028 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.233512 13560 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:02.234493  8176 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.68:44573 timed out after 1.621s (SENT) [suppressed 27 similar messages]
I20260709 13:15:02.235075 13699 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:02.235718 13560 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.236096 13585 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:02.229838 13497 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.237720  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:02.238552 13089 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:02.239359 13674 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:02.201768  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:02.240833  8208 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:15:02.206423  8026 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:02.206642  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:02.200350  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:02.221410  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:02.130612 13421 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:02.216341  8724 rpcz_store.cc:269] 0709 13:15:00.523158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:00.523303 (+   145us) service_pool.cc:224] Handling call
0709 13:15:02.160814 (+1637511us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400581bd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:02.160903 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2919068544}
I20260709 13:15:02.207820  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:15:02.226488  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 135939us
I20260709 13:15:02.186298  8210 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:02.188879 13402 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.170639  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:02.246203  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:02.246588  8212 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:02.246768  8210 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:15:02.247630  8133 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:15:02.249020  8026 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:02.243546  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:02.199442 13627 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.262738  8026 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:02.250085  7957 catalog_manager.cc:5697] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 10 to 13, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 13 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.265097  8728 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:02.220539 13198 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:02.275503  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.276500  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:02.272671  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:02.248145 13219 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:02.278093  8131 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:02.279261  8723 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:02.280282  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:02.285121  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:02.285650  8206 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:15:02.289134  8210 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:02.291128  8207 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:15:02.292356  8026 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:02.224092 13431 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:02.294579  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.295827 13402 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.248173 13674 raft_consensus.cc:697] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:02.223773 13716 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.297430 13674 consensus_queue.cc:237] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.248349 13723 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:02.265658 13497 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:02.299538  7957 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.300627  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.301198  8138 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:15:02.301060  7957 catalog_manager.cc:5697] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 14 to 17, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 17 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.306402 13678 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.306981  7957 catalog_manager.cc:5697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.308614  7957 catalog_manager.cc:5697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.275280 13407 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:02.310061  8696 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:02.312323  8696 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:02.313547 13337 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:02.313675  8205 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:15:02.314158  8205 pending_rounds.cc:85] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 8
I20260709 13:15:02.314469  8205 pending_rounds.cc:107] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:15:02.314739  8205 raft_consensus.cc:2889] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:02.314487 13337 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:02.310519  7957 catalog_manager.cc:5697] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.316112 13337 raft_consensus.cc:697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:02.315536 13585 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:02.317303 13337 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.320060 13198 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 14 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:02.321257 13198 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.306989 13027 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:02.317376  7957 catalog_manager.cc:5697] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 10 to 13, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 13 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.278988 13216 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.325748  7957 catalog_manager.cc:5697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 9 to 15, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 15 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.328672 13216 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:02.329250 13216 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:02.329423  7957 catalog_manager.cc:5697] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 5 to 8, term changed from 9 to 13, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65), VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New cstate: current_term: 13 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:02.334931  7957 catalog_manager.cc:5697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.335656 13677 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.307668 13400 raft_consensus.cc:2764] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:15:02.292507 13627 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.343047 13627 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:02.300086  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:02.345305  7957 catalog_manager.cc:5697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:02.350464  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 58217.4us
I20260709 13:15:02.351733  8026 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:02.364472  8203 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:02.365600 13677 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:02.366078 13677 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.365810  7957 catalog_manager.cc:5697] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 10 to 18, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 18 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:02.366565 13677 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:02.367267  7957 catalog_manager.cc:5697] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 13 to 16, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 16 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.368956  7957 catalog_manager.cc:5697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:02.370388  7957 catalog_manager.cc:5697] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 10 to 13, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 13 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:02.374027  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:02.375191  8209 raft_consensus.cc:2410] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:15:02.378657  8206 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
W20260709 13:15:02.379838  8029 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.012s	sys 0.030s
I20260709 13:15:02.384223  8208 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
W20260709 13:15:02.395265  8029 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.012s	sys 0.030s
W20260709 13:15:02.386524  7961 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34146 (request call id 258) took 57141 ms (client timeout 15000 ms). Trace:
I20260709 13:15:02.225273 13290 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:02.395859  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 712276us
I20260709 13:15:02.381374 13497 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:15:02.419898  8055 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.014s	sys 0.031s
I20260709 13:15:02.403133  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:02.447527  8055 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.015s	sys 0.031s
I20260709 13:15:02.403694  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.404238 13290 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.272536 13421 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.420397  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.402273  8131 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
W20260709 13:15:02.420426  8028 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.006s	sys 0.059s
W20260709 13:15:02.494570  8028 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.008s	sys 0.068s
I20260709 13:15:02.448763  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1902) took 1152 ms. Trace:
I20260709 13:15:02.445678  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.440569  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.495534  8055 rpcz_store.cc:276] 0709 13:15:01.296341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:01.307847 (+ 11506us) service_pool.cc:224] Handling call
0709 13:15:01.325418 (+ 17571us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:01.325612 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:15:01.438561 (+112949us) log.cc:844] Serialized 14 byte log entry
0709 13:15:02.448293 (+1009732us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:02.448362 (+    69us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:15:02.448430 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:02.448515 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:02.448675 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":379625,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":1645,"raft.queue_time_us":83,"thread_start_us":872559,"threads_started":2}
W20260709 13:15:02.445806 13398 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.000s	sys 0.045s
W20260709 13:15:02.446014 13570 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.018s	sys 0.028s
I20260709 13:15:02.443280 13402 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.510071 13660 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.511848 13402 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:02.508808 13398 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.000s	sys 0.045s
W20260709 13:15:02.511209 13570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.019s	sys 0.028s
I20260709 13:15:02.420581 13497 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:15:02.506676 13427 consensus_meta.cc:287] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.869s	user 0.013s	sys 0.005s
W20260709 13:15:02.395908  7961 rpcz_store.cc:269] 0709 13:14:05.238645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.238838 (+   193us) service_pool.cc:224] Handling call
0709 13:15:02.379703 (+57140865us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:58.655035 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:58.655806 (+   771us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:58.655862 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:14:58.655900 (+    38us) tablet.cc:662] Decoding operations
0709 13:14:58.656289 (+   389us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:58.656394 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:58.656427 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:14:58.656781 (+   354us) tablet.cc:701] Row locks acquired
0709 13:14:58.656826 (+    45us) write_op.cc:260] PREPARE: finished
0709 13:14:58.656963 (+   137us) write_op.cc:270] Start()
0709 13:14:58.657092 (+   129us) write_op.cc:276] Timestamp: P: 1783602898656906 usec, L: 0
0709 13:14:58.657126 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:14:58.657839 (+   713us) log.cc:844] Serialized 538 byte log entry
0709 13:14:59.134238 (+476399us) op_driver.cc:464] REPLICATION: finished
0709 13:14:59.306267 (+172029us) write_op.cc:301] APPLY: starting
0709 13:14:59.306383 (+   116us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:59.307942 (+  1559us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:59.308071 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:59.308737 (+   666us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:59.308983 (+   246us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:14:59.309040 (+    57us) write_op.cc:312] APPLY: finished
0709 13:14:59.309287 (+   247us) log.cc:844] Serialized 25 byte log entry
0709 13:14:59.309462 (+   175us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:59.309534 (+    72us) write_op.cc:454] Released schema lock
0709 13:14:59.309608 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:59.310031 (+   423us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:59.310057 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4962481,"reported_tablets":51,"spinlock_wait_cycles":119168,"child_traces":[["op",{"apply.queue_time_us":171699,"apply.run_cpu_time_us":3759,"apply.run_wall_time_us":3760,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8307,"num_ops":1,"prepare.queue_time_us":172390,"prepare.run_cpu_time_us":35325,"prepare.run_wall_time_us":134952,"raft.queue_time_us":255440,"raft.run_cpu_time_us":15734,"raft.run_wall_time_us":86475,"replication_time_us":472122,"spinlock_wait_cycles":55552,"thread_start_us":452242,"threads_started":4,"wal-append.queue_time_us":172890,"wal-append.run_cpu_time_us":20639,"wal-append.run_wall_time_us":1549147}]]}
I20260709 13:15:02.510489 13461 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.543655 13461 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.501632 13725 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.538973 13659 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.542649 13090 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.513854 13727 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.563899 13090 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.572443 13090 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.570716 13726 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.563772 13725 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.576082 13726 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.562855 13497 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.579231 13461 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.557873 13659 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.581386 13659 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.581588 13677 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.567499 13727 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.566497 13407 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:02.547108 13660 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.586300 13027 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:02.592010 13727 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.595096 13090 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:02.621265  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 395371us
I20260709 13:15:02.589401 13413 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.628041  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.637635  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:02.637838  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:02.638731  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:02.655288  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 420794us
I20260709 13:15:02.657761  8205 raft_consensus.cc:2393] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 16: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:15:02.656095  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 362422us
I20260709 13:15:02.636440  8026 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:02.659988  8104 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:02.671298  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.675948  8175 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:02.680794  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.681881  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.678678  8133 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:02.686326  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:02.639699 13407 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.670063 13461 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.679047 13398 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:02.623911 13090 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.673563 13721 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:02.679149 13678 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:02.673928 13723 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:02.678773 13627 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:02.667514 13703 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:15:02.687956 13400 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.708078  8129 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:02.683282 13712 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:15:02.688107 13379 consensus_queue.cc:1059] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:15:02.714829  8029 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:02.694037 13407 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.707875 13570 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:02.708024 13678 raft_consensus.cc:697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:02.702224 13461 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.707432 13627 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.712663 13712 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.741402 13570 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.722239 13727 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:02.745386 13727 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.745798 13727 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:02.748457  8205 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:02.711318 13400 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.707551 13723 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.719129 13402 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:02.720311 13398 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:15:02.742133 13737 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:02.759754 13677 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.763298 13738 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:02.782766 13677 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.783308 13677 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.785355 13677 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:02.760748 13712 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.788105 13712 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:02.788436 13728 log.cc:927] Time spent T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.344s	user 0.002s	sys 0.000s
I20260709 13:15:02.795207 13027 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.796288 13461 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:02.795914 13027 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.798557 13400 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:02.795292 13627 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:02.799614 13090 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.799711 13027 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.763979 13427 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
W20260709 13:15:02.801148 13731 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:15:02.801412 13716 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:02.802718 13726 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.809772 13431 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:02.811468 13723 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:02.815313  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:02.816938 13028 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:02.816988  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.817893  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.818907  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.820324  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.821346  8133 raft_consensus.cc:2393] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:02.832618  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:02.833295  8131 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:02.848289  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.849242  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.822139  8026 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:02.852384  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.860630  8135 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:02.769543 13402 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:15:02.877008  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.859138  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 43899.9us
I20260709 13:15:02.879124  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:02.847642 13027 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.881193 13027 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.882558 13402 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:02.883913 13027 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.862458 13461 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:02.863361 13407 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:02.903064  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:02.864706 13421 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.865139 13427 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:02.779060 13738 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:02.780305 13570 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.780138 13660 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:02.834141 13674 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.848798 13737 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.867295  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.869443 13653 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:02.870883 13734 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:15:02.871881 13413 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:02.873416 13732 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:15:02.874864  8206 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:15:02.932130  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:02.879341  8131 raft_consensus.cc:2410] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:15:02.904011 13461 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:02.880446  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:02.893545  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:02.921855  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.914577  8026 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:02.915434  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:02.949793  8208 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:02.950877  8725 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:15:02.915627  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:02.952099  8696 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:02.949838  8728 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:02.952777  8129 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:02.924799  8723 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:02.926056  8027 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:02.898011 13337 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.849908  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:02.950217  8130 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:02.921788 13379 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:15:02.961007  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 285142us
W20260709 13:15:02.969838  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 55344.6us
I20260709 13:15:02.972851  8026 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:02.932816 13402 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:02.946974 13560 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:02.874958 13678 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:02.916653 13703 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:15:02.974995  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:02.958456 13461 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:02.949442 13090 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.913735 13735 consensus_queue.cc:1059] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:15:02.973543  8175 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:03.008782  8026 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:03.011785  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:02.967350 13707 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:03.016613 13716 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:03.016999  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:02.976207 13677 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:03.014703 13712 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:03.023926  8137 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:15:02.971396 13427 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:15:02.953502 13737 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.026298  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.013203 13402 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:03.028389  8135 raft_consensus.cc:2368] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:15:02.946637 13674 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.021585 13585 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:03.029996 13585 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:03.031188 13585 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.035509 13674 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:03.016172 13633 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:03.036072 13413 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:03.041718 13633 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:03.041930 13413 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.044620 13413 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:03.038836 13745 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:15:03.039436 13740 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:15:03.018219 13407 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:03.047083  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.019326 13677 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.969514 13337 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.053279 13407 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:03.062407  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:15:02.977182 13198 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.003s	sys 0.027s
I20260709 13:15:02.978863 13379 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.042176 13633 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:03.067857  8175 leader_election.cc:400] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14.
I20260709 13:15:03.076498  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:03.017975 13090 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:03.083396  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 414183us
W20260709 13:15:03.066635 13198 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.003s	sys 0.027s
I20260709 13:15:03.055581 13677 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:03.087442  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.098881  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:03.099583  8204 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:15:03.099893  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.100734  8133 raft_consensus.cc:2368] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
I20260709 13:15:03.101572  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.088721  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:03.096915  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.068434 13407 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:03.103148  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.103708  8203 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:15:03.078251 13627 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.106155  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:03.108943  8129 raft_consensus.cc:2393] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:03.108793  8204 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:15:03.067144 13721 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:03.107722  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.107735  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.112783  8210 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:15:03.106804  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.109676  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.114148  8208 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:03.109439  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3361) took 1006 ms. Trace:
I20260709 13:15:03.115059  8211 rpcz_store.cc:276] 0709 13:15:02.102737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.107144 (+  4407us) service_pool.cc:224] Handling call
0709 13:15:02.107405 (+   261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:02.140533 (+ 33128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:02.140562 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:02.224427 (+ 83865us) log.cc:844] Serialized 26 byte log entry
0709 13:15:02.672824 (+448397us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:02.672862 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:02.672922 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:03.109236 (+436314us) raft_consensus.cc:1685] finished
0709 13:15:03.109255 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:03.109384 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":50863,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":105,"thread_start_us":432704,"threads_started":1,"wal-append.queue_time_us":532077}
I20260709 13:15:03.115083  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.117220  8203 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:03.018188 13738 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.110736  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.095456 13716 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:03.132587  8175 leader_election.cc:400] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:15:03.132911  8175 leader_election.cc:403] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:03.133440  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.133752  8175 leader_election.cc:400] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
I20260709 13:15:03.134140  8175 leader_election.cc:403] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:03.124032 13400 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:03.104928 13725 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:03.118734 13627 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.122618 13703 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:15:03.123950 13716 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:03.135087 13655 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.008s	sys 0.014s
W20260709 13:15:03.151932  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 272853us
I20260709 13:15:03.158144  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.153717  8061 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:03.159844  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:03.163106  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:03.163573  8063 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:03.122056 13721 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.161824  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:03.120445 13727 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.165630  8134 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:03.158319  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:03.166004  8133 raft_consensus.cc:3055] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:03.121188 13750 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:15:03.168049  8133 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:03.168069  8058 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:03.168463  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.177040  8057 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
W20260709 13:15:03.177649  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 195581us
I20260709 13:15:03.189244 13723 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
W20260709 13:15:03.162771 13655 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.008s	sys 0.015s
I20260709 13:15:03.179582  8133 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.200690  8133 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:03.197589  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:03.198776  8206 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:03.195996  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.201957  8131 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
W20260709 13:15:03.186074 13743 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:15:03.149324 13746 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:15:03.177760 13741 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:15:03.243309  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.149143 13400 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.207175  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 945) took 1042 ms. Trace:
I20260709 13:15:03.247099  8727 rpcz_store.cc:276] 0709 13:15:02.164574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.165902 (+  1328us) service_pool.cc:224] Handling call
0709 13:15:03.207016 (+1041114us) spinlock_profiling.cc:243] Waited 887 ms on lock 0x7b5400198f54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:03.207125 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2040598784}
I20260709 13:15:03.207414  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:03.247846  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:03.248399  8724 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:15:03.207995  8060 raft_consensus.cc:1240] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:03.209537  8027 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:03.205040  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.245085  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:03.246656  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 112969us
I20260709 13:15:03.250355  8208 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:03.252614  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:03.254170  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:03.256458  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 47045.6us
I20260709 13:15:03.262598  8207 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 16: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:03.212613 13674 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:03.265115  8133 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:15:03.268132  8133 pending_rounds.cc:85] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 12
I20260709 13:15:03.268481  8133 pending_rounds.cc:107] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:15:03.268808  8133 raft_consensus.cc:2889] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:03.266462  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.263573  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.244684 13752 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.270442  8208 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 18: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:03.265383  8027 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:15:03.265792  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 145237us
I20260709 13:15:03.287441  8104 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.253898 13413 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.288822  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.291697  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.244099 13726 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 13:15:03.314277  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:03.320201  8208 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:03.288276 13748 log.cc:927] Time spent T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.237s	user 0.001s	sys 0.001s
W20260709 13:15:03.284924 13638 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:15:03.263041 13747 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:03.311986 13703 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:03.334087  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:03.290673 13674 raft_consensus.cc:697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:03.291232 13761 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.291057 13737 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:03.313097 13337 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:03.339025  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.315620 13660 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:03.319635 13653 consensus_queue.cc:1059] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:15:03.353967  8729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 990) took 1588 ms. Trace:
I20260709 13:15:03.323465 13379 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:03.341254 13570 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:03.255273 13746 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.361164  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:03.368824  8208 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:03.337427 13747 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:03.357928  8729 rpcz_store.cc:276] 0709 13:15:01.765592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:01.765944 (+   352us) service_pool.cc:224] Handling call
0709 13:15:02.043216 (+277272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:02.150540 (+107324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:02.150599 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:02.250953 (+100354us) log.cc:844] Serialized 26 byte log entry
0709 13:15:02.713274 (+462321us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:02.713316 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:02.713791 (+   475us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:03.185409 (+471618us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:15:03.345664 (+160255us) raft_consensus.cc:1685] finished
0709 13:15:03.345690 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:03.353911 (+  8221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":105403,"raft.queue_time_us":83635,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":45898,"spinlock_wait_cycles":6199040,"thread_start_us":330962,"threads_started":1,"wal-append.queue_time_us":637862}
I20260709 13:15:03.342681 13760 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:03.358421 13742 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:15:03.312702 13413 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.386518  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.343291 13750 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 12. Current term is 13.
I20260709 13:15:03.290541 13752 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.357971 13737 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.374054 13712 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:03.397442  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.398651 13627 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:03.424104 13400 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:03.424252 13659 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:03.424813 13659 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.387810 13747 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:03.407473  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.410593 13407 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.420502  8208 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:03.420737  8062 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
W20260709 13:15:03.459744  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 170950us
I20260709 13:15:03.423652 13028 raft_consensus.cc:3055] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:03.427481  8058 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:03.428479  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:03.405440  8209 raft_consensus.cc:2393] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:15:03.453507  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 134181us
I20260709 13:15:03.462137  8059 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:03.417835 13633 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:03.418874 13755 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:15:03.463200  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.348201 13761 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.464196  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:03.471652  8057 raft_consensus.cc:2368] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:15:03.466689  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.472216  8062 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:15:03.476598  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.477185  8062 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:03.477975  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.478737  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.479831  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.459491 13660 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:03.483803  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.463863 13727 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:03.461661 13028 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:03.466089 13716 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:03.519012  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:03.519781  8062 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:03.486277 13407 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:03.487229 13413 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:03.487579 13752 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:03.459940 13674 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.488441 13461 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.427781 13737 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.455526 13751 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:15:03.390942 13760 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:03.491427 13754 log.cc:927] Time spent T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
W20260709 13:15:03.492275 13762 log.cc:927] Time spent T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 13:15:03.492810  8176 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:03.565766  8056 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:03.493507  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.493806 13660 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:03.494442 13759 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
W20260709 13:15:03.568181  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 75458.2us
I20260709 13:15:03.484601  8209 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:03.484251  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.570693  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:03.498184  8063 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:15:03.457616 13653 consensus_queue.cc:1059] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:15:03.485945  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.520609  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.571936  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.572151  8058 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:15:03.572950  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.522941  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.523092  8101 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:03.523509  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:03.532336  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69246.6us
I20260709 13:15:03.528077 13570 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:03.502110 13659 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:03.545471 13737 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:03.595986  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.596277  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:03.597203  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 113026us
I20260709 13:15:03.597298 13198 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:03.556030 13633 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:03.559191 13090 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:03.559484 13746 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:03.564599  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:03.609345  8723 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:15:03.609802 13751 raft_consensus.cc:3055] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:03.610328 13751 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:15:03.573791  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 50774.5us
I20260709 13:15:03.576359  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:03.576884  8211 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:03.611330 13751 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.577572 13660 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:03.578528  8029 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:03.613564  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.613929 13760 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:03.565073 13028 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.614884 13028 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:03.580901 13738 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:03.622452  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.623245  8057 raft_consensus.cc:2410] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:15:03.623625  8131 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:03.623962  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.623899  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.627900  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.562621 13753 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:03.584218 13461 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:03.585798 13761 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:03.586125 13756 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.114s	user 0.001s	sys 0.001s
I20260709 13:15:03.531812 13774 consensus_queue.cc:1059] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:15:03.590543  8059 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:15:03.590644  8058 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
I20260709 13:15:03.591111 13779 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:03.591385  8104 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:03.591722  8055 raft_consensus.cc:2393] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:03.592221  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:03.592653  8730 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:15:03.593055  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 848) took 1349 ms. Trace:
I20260709 13:15:03.593218  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:03.599815 13723 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:03.645167  8056 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:15:03.642776 13768 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:15:03.647975  8056 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:03.649756  8056 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:15:03.652122  8056 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:03.594558  8729 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:15:03.595386  8726 raft_consensus.cc:2368] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:15:03.599273  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:03.598939  8062 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:03.603325  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:03.603180 13198 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:03.605625  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.606436 13337 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
W20260709 13:15:03.663305  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 57791.2us
I20260709 13:15:03.663424 13768 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:03.627936  8131 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:03.663794 13198 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:03.628641  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.663811 13768 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:03.614140  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:03.664776  8059 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:15:03.665416  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 51350.2us
I20260709 13:15:03.666124  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.665566 13768 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.667241  8029 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.667305 13768 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:03.668372  8029 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:03.581002 13677 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.670521 13677 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:03.636370  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.671721  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.672322  8211 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:15:03.672406  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:03.672911  8726 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:15:03.639387  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.641999  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.642058  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:03.590061 13747 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.679380  8057 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:03.635672 13660 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.628752  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.674752  8204 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:15:03.685706  8056 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:03.673540  8205 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:15:03.676020  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.633566 13746 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:03.686951 13760 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:03.687633 13746 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:03.688073 13746 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:03.690536  8204 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:03.657440  8727 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:03.692564  8060 raft_consensus.cc:2393] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 16: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:03.694517 13726 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:03.694950 13726 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:03.694712 13660 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.695322 13726 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:03.695665 13219 raft_consensus.cc:3055] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:03.696175 13219 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:03.696831 13660 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:03.699257  8063 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:03.699491  8693 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:03.700837  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:03.701548  8727 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:03.702558  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:03.706575  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:03.644508  8136 rpcz_store.cc:276] 0709 13:15:02.244085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.244288 (+   203us) service_pool.cc:224] Handling call
0709 13:15:02.275107 (+ 30819us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:02.654814 (+379707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:02.654857 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:02.676234 (+ 21377us) log.cc:844] Serialized 26 byte log entry
0709 13:15:03.119660 (+443426us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:03.130826 (+ 11166us) log.cc:844] Serialized 14 byte log entry
0709 13:15:03.131259 (+   433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:03.187930 (+ 56671us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b540036ef54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:03.187991 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:03.592762 (+404771us) raft_consensus.cc:1685] finished
0709 13:15:03.592872 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:03.592981 (+   109us) inbound_call.cc:177] Queueing success response
0709 13:15:03.611904 (+ 18923us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b18001ae540. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
Metrics: {"mutex_wait_us":32786,"prepare.queue_time_us":10375,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":25436,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":168,"spinlock_wait_cycles":68132480,"thread_start_us":782007,"threads_started":2,"wal-append.queue_time_us":565261}
I20260709 13:15:03.699592  8027 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:03.714448  8029 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:03.717124  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 125821us
I20260709 13:15:03.717798  8104 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:03.682147 13461 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:03.682291 13757 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:03.724906  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.728891  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:03.729677  8063 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:03.643049 13413 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:15:03.692750 13747 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:03.698491 13727 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:03.733834  8104 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:03.731312 13765 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:03.739832  8029 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:03.697237 13219 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.635000 13753 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.734416 13398 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:03.712978 13421 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:03.746965 13421 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:03.747226 13751 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:03.747263  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.721063 13560 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:03.723883 13655 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:03.747359 13421 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:03.733898 13637 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:03.721244 13779 raft_consensus.cc:697] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:03.727308 13707 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:03.733249 13757 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:03.733385 13677 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:03.738863 13660 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.763890  8055 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:03.772939  8208 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:03.775365  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.776796  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.777407  8210 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:15:03.777498  8208 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:03.758811  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.765594  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:03.779655  8211 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:03.749531 13570 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:03.716856 13723 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:15:03.753742 13758 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:03.753613 13407 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:03.770579  8729 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:03.787063  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:03.786545  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:03.754812 13560 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:03.790995  8727 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:03.772192 13219 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:03.770608 13707 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:03.772876 13757 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:03.815351  8027 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:03.796411  8721 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:03.753116 13703 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
W20260709 13:15:03.821913  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 82187.5us
I20260709 13:15:03.823784  8176 leader_election.cc:400] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:15:03.824088  8176 leader_election.cc:403] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:03.824133  8029 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.789947 13758 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:03.798166 13560 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:03.847656  8029 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.845984  8176 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:03.822230 13707 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:03.786502 13660 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.821127 13769 raft_consensus.cc:3055] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:03.825444 13407 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:03.822675 13723 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.822862 13716 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.859125 13716 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.861841 13716 leader_election.cc:290] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:03.862514 13407 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:03.844919 13712 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:15:03.863561 13407 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.821525 13398 raft_consensus.cc:697] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:03.856365 13769 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:03.865779 13398 consensus_queue.cc:237] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.866724 13769 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.867908 13769 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:03.855556  8029 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:03.827260 13585 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:15:03.824879 13737 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:03.858359 13723 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:03.845707 13758 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:03.846052 13216 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
W20260709 13:15:03.878681  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.879456 13413 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:15:03.858379 13786 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:15:03.881023 13461 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:03.882791  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.883430  8061 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:03.883868  8059 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:15:03.852279 13678 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:03.854398 13677 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:03.887921  8204 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
W20260709 13:15:03.909660  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:03.886760  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 275714us
I20260709 13:15:03.910804  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.911399  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:03.887288  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:03.890643  8696 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:03.912973  8137 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:03.871603  8029 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:03.889050  8104 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.854539 13750 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:03.920225  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:03.922184  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:03.857348 13400 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:03.907583  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:03.925740  8204 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:15:03.919723  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:03.931634  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 60117.9us
I20260709 13:15:03.920658  8104 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:03.932376  8029 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.929351  8134 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:15:03.940528  8209 raft_consensus.cc:2410] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:15:03.941354  8101 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:03.941962  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.942022  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.918473 13637 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:03.906911 13413 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:03.944201  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:03.921906 13738 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:03.931783 13427 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:15:03.975337  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 54783.8us
I20260709 13:15:03.979312  8101 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:03.904291 13737 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:03.988005  8726 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:03.940479 13570 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:03.988667  8029 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.926867 13735 consensus_queue.cc:1059] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
W20260709 13:15:04.039598  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 51044us
I20260709 13:15:04.043294  8029 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:03.980921 13678 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:04.046998  8027 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:03.975980 13198 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:03.945755 13795 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:03.976511 13765 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:03.945393 13655 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:03.988521 13379 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:04.028721  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:03.931914 13750 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:04.054147  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 66991.7us
I20260709 13:15:03.886037 13779 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.029218 13738 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:15:04.151680  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.989041 13751 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:04.054663 13427 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:04.050531 13727 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:04.129319 13795 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:04.130666 13655 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:04.136186 13379 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:04.039585 13570 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.186326 13738 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:15:04.211719 13797 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 13:15:04.202129 13795 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:04.187065 13753 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:04.216615 13427 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.203429 13655 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:04.218469 13792 log.cc:927] Time spent T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.240s	user 0.000s	sys 0.000s
W20260709 13:15:04.223989 13805 log.cc:927] Time spent T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:15:04.224153 13337 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:04.226703 13765 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.228662 13660 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:04.231418 13737 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:04.235872  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:04.149410 13796 raft_consensus.cc:3055] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:04.371783 13796 raft_consensus.cc:740] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:15:04.269263 13726 consensus_meta.cc:287] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.573s	user 0.007s	sys 0.000s
W20260709 13:15:04.272785 13746 consensus_meta.cc:287] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.584s	user 0.007s	sys 0.000s
I20260709 13:15:04.122882 13802 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.150s
I20260709 13:15:04.272863 13400 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.274571 13750 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:04.194444 13768 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:04.300143  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:04.205067 13379 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:04.346374  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:04.120936 13633 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:04.136528 13637 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.263922 13758 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:04.382853  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 339605us
I20260709 13:15:04.297832 13738 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:04.134296 13659 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:15:04.402653  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 458484us
I20260709 13:15:04.407618  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:04.408895  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 429641us
W20260709 13:15:04.411405  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 468000us
W20260709 13:15:04.396335 13219 consensus_meta.cc:287] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.000s
I20260709 13:15:04.415609  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:04.450201  8131 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:15:04.198915 13413 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.457564  8062 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:15:04.415455 13400 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.462548  8133 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:04.462093 13817 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:04.423185 13402 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:04.423638  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3209) took 1217 ms. Trace:
I20260709 13:15:04.461728  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:04.372828 13337 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.457638  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:04.446856  8102 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:04.295497 13421 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.413939 13723 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:04.463694  8055 rpcz_store.cc:276] 0709 13:15:03.206048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.645999 (+439951us) service_pool.cc:224] Handling call
0709 13:15:03.763710 (+117711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:03.780372 (+ 16662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:04.423443 (+643071us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540028a5d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:04.423576 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":269976320}
I20260709 13:15:04.436632 13633 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:04.466490 13793 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:15:04.479339  8137 consensus_meta.cc:287] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.000s
I20260709 13:15:04.483067  8137 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
W20260709 13:15:04.460552 13398 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.004s	sys 0.020s
I20260709 13:15:04.408010 13726 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.413710 13746 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.484794  8102 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:04.489825 13400 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:04.491007 13726 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:04.486753 13398 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.004s	sys 0.020s
I20260709 13:15:04.495911 13758 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:04.497900 13769 consensus_meta.cc:287] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.000s
I20260709 13:15:04.499030 13337 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:04.500944  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:04.466200 13796 consensus_queue.cc:260] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.502207 13796 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:04.428383 13768 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.504518 13768 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:04.513583 13769 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.514200 13769 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.515173  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:04.516345  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:04.513888  8176 leader_election.cc:304] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:04.467015 13402 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:15:04.538408  8132 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.022s	sys 0.025s
W20260709 13:15:04.538933  8132 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.022s	sys 0.025s
I20260709 13:15:04.486193 13633 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:04.487171 13735 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:04.532296 13768 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.532848 13653 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:04.530380 13793 raft_consensus.cc:697] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:04.549453 13400 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:04.568037  8722 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
W20260709 13:15:04.553200 13028 consensus_meta.cc:287] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.938s	user 0.008s	sys 0.003s
I20260709 13:15:04.533358 13769 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:04.557761 13735 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:04.561106 13779 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:15:04.565738 13757 consensus_meta.cc:287] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.737s	user 0.008s	sys 0.002s
I20260709 13:15:04.581259 13402 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.568271 13753 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.590129 13400 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.593444 13750 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:04.595472 13750 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.596592 13746 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:04.597546 13750 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:04.583072 13653 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.598747 13653 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.593402 13753 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.602312 13753 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:04.594972 13822 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:15:04.597061 13793 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.578651 13735 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:04.579656 13779 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:15:04.591606 13804 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:15:04.564679 13768 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:04.602236 13560 consensus_meta.cc:287] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.745s	user 0.007s	sys 0.000s
W20260709 13:15:04.612584  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 155782us
W20260709 13:15:04.593899 13677 consensus_meta.cc:287] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.679s	user 0.007s	sys 0.004s
W20260709 13:15:04.616693  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 181170us
I20260709 13:15:04.628530  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:15:04.623663  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 140983us
W20260709 13:15:04.617061  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 132320us
I20260709 13:15:04.588311  8175 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:04.614166 13402 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:04.631747  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:15:04.633788  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45520.2us
I20260709 13:15:04.615483 13779 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:04.640712  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:04.641767  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:04.642443  8204 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 16: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:04.632058  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:04.643165  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:04.643882  8203 raft_consensus.cc:2393] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:04.637732 13767 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:04.635807 13746 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:04.645007 13746 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:04.644963 13767 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.645566 13746 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:04.638520 13659 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:04.646730 13659 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:04.647092 13659 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:04.635468 13769 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:15:04.652383 13707 consensus_meta.cc:287] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.786s	user 0.003s	sys 0.005s
I20260709 13:15:04.652814 13707 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.589954 13757 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.658164 13757 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:04.636549 13752 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.658918 13752 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.659552 13767 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:04.638123  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:04.659744 13337 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:04.660274  8210 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:04.660959 13752 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:04.661933 13768 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:04.594658 13400 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.663868 13400 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:04.637321  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:04.666965  8212 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:04.668752  8210 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
W20260709 13:15:04.674765  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:04.682528  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.683146  8055 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
W20260709 13:15:04.659363 13825 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:15:04.689528  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.636742 13028 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:04.691215 13659 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.691773 13028 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.692776  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.693303 13659 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:04.693653 13028 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:04.627593 13560 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.694455 13769 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:04.694592  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:04.694948 13769 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.695559  8725 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:04.695830  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.696106  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:04.695842 13560 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:04.696934 13769 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:04.697187 13421 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:04.697754 13421 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:04.697865  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:04.698966 13707 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:04.700318  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:04.702629  8212 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:15:04.703334  8210 raft_consensus.cc:1240] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:04.703438  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:04.704521  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.705922  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:04.706499  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:04.706679  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:04.706845  8206 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:15:04.707275  8204 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:04.707602  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 50565us
I20260709 13:15:04.631078 13677 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:04.710372  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:04.710464  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:04.710893  8062 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:04.712177  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:04.715415  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:04.634131 13765 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:04.723176  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:04.724666  8175 leader_election.cc:400] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:15:04.724951  8175 leader_election.cc:403] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:15:04.727267  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:04.704669 13757 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:04.738215  8204 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:04.739797  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.741462  8062 raft_consensus.cc:1813] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Handling vote request from an unknown peer d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:04.741914  8062 raft_consensus.cc:2368] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:15:04.741977  8696 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:04.643383 13427 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:04.715196 13712 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:04.720342 13767 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:04.720366 13560 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:04.720960 13829 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:04.725535 13721 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:04.727108 13753 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:04.725731 13737 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:15:04.748068  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 71652.1us
I20260709 13:15:04.750658  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:04.755517  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.751475  8104 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:04.743064 13723 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:15:04.773545  8721 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:04.774133  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:04.774674  8060 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:04.774691  8057 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:04.727741 13765 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.782500  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.747718 13767 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:15:04.778785  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:04.780876  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.784546  8057 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:04.726742 13779 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.789448 13738 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.745618 13757 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.770780 13752 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:15:04.809561  8056 raft_consensus.cc:1240] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:04.812366  8054 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:04.827322  8057 raft_consensus.cc:1240] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:04.751920 13560 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.831804  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.828197  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.832949  8209 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 3376) took 1073 ms. Trace:
I20260709 13:15:04.786975 13767 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:04.835124  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.838112 13812 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:04.838761 13812 raft_consensus.cc:697] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:04.834744  8209 rpcz_store.cc:276] 0709 13:15:03.743083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.758587 (+ 15504us) service_pool.cc:224] Handling call
0709 13:15:04.816286 (+1057699us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b5400110154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:04.816461 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":371590400}
I20260709 13:15:04.842679  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1710) took 1115 ms. Trace:
I20260709 13:15:04.843068  8133 rpcz_store.cc:276] 0709 13:15:03.727193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.727340 (+   147us) service_pool.cc:224] Handling call
0709 13:15:03.727621 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:04.462910 (+735289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:04.842461 (+379551us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b5400174154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:04.842622 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1689515904}
I20260709 13:15:04.845317  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.846254  8057 raft_consensus.cc:2393] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 12: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:04.847667  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.848260  8054 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:04.848799  8057 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:04.829746 13398 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.768857 13839 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:15:04.853180  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.858120  8060 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:04.752431 13829 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.786660 13637 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:15:04.861837  8056 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:15:04.868810  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.869390  8056 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:15:04.884558  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.885311  8056 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:04.839783 13812 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.893591  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.879702 13834 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:04.864862 13398 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:04.901687 13840 log.cc:927] Time spent T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:15:04.902717 13807 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:04.903152 13807 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:04.903509 13807 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:04.914877  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 130919us
I20260709 13:15:04.916476  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.920060 13431 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:04.920558 13431 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.920996 13800 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.921468 13800 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.922277 13807 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.923267  8136 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:15:04.924515 13807 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:04.925036  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:04.928774 13431 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:04.929394  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:04.931222  8133 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:15:04.931455  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:04.930608  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:04.932062  8209 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:04.930032  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:04.932598  8207 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:04.933130  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:04.933933 13858 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:04.934310 13858 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:04.934707 13858 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:04.932201  8136 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:04.938045  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:04.938486 13800 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:04.938969  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:04.939519  8135 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:04.942267  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:04.942020 13858 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.943953  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:04.944206 13858 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:04.945359  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:04.945752  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:04.946669  8137 raft_consensus.cc:2393] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 16: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:15:04.947988  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:04.948515  8209 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:04.948568  8132 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:04.950315  8207 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:15:04.886473 13379 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.958657  8135 raft_consensus.cc:2368] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 16. Current term is 17.
I20260709 13:15:04.959612 13379 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:04.961582 13655 consensus_meta.cc:287] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.589s	user 0.007s	sys 0.000s
I20260709 13:15:04.966878 13757 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:04.969630 13850 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:15:04.970021 13847 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:15:04.966935 13738 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:04.968312  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.952284  8104 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:04.971617  8136 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:04.970654  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:04.979876  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:04.982669 13735 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:04.986114  8133 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:04.988116 13735 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:04.970222 13655 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:04.989930  8061 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:15:04.992106 13655 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:04.992978  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:04.993775  8722 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:04.996220  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:04.889998 13796 consensus_queue.cc:1059] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:15:04.998279  8209 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:04.999832  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3383) took 1248 ms. Trace:
I20260709 13:15:05.032001  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:05.032758  8060 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:05.030936  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:04.982024  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:05.003366  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:15:05.033950  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 52014.9us
I20260709 13:15:05.031667  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:05.035945  8029 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:05.034127  8209 raft_consensus.cc:2368] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15.
I20260709 13:15:05.037381  8029 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:05.037621  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:05.039006  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:05.039711  8209 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:04.997880 13379 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:04.995069 13560 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:05.041858  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:05.043610  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:05.005352 13753 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:04.909446 13834 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.034991 13461 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:05.065408  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:05.070647  8207 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:05.043165 13655 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:05.074927 13655 raft_consensus.cc:697] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:05.075764 13655 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.080714  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3382) took 1336 ms. Trace:
I20260709 13:15:05.080904 13829 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:05.082996  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:05.083674  8056 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:05.055164 13707 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:05.081228  8211 rpcz_store.cc:276] 0709 13:15:03.744592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.781128 (+ 36536us) service_pool.cc:224] Handling call
0709 13:15:04.036772 (+255644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:04.037026 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:04.131757 (+ 94731us) log.cc:844] Serialized 14 byte log entry
0709 13:15:04.365875 (+234118us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:04.365946 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:04.366009 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:04.366101 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:05.080514 (+714413us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b540019ebd0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:05.080647 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":85619,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":10765,"raft.queue_time_us":559,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":6261,"spinlock_wait_cycles":587260288,"thread_start_us":233123,"threads_started":1,"wal-append.queue_time_us":290175}
W20260709 13:15:05.068737 13795 consensus_meta.cc:287] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.700s	user 0.010s	sys 0.000s
I20260709 13:15:05.137192  8209 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:05.153673  8104 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:05.157511  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:05.084379 13379 raft_consensus.cc:697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:05.162833  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:05.063237 13753 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.165400  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:05.170895 13379 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.150969 13707 raft_consensus.cc:697] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:05.179021 13653 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:05.181320 13834 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:05.142197 13765 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:05.182219 13633 consensus_meta.cc:287] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.001s
I20260709 13:15:05.187656  8134 raft_consensus.cc:3055] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:05.190045  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:05.194536  8206 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:05.192808  8134 raft_consensus.cc:740] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:05.195506  8211 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:15:05.198465 13633 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.193475  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.200618 13633 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:05.151578 13746 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.151407 13795 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.197593  8134 consensus_queue.cc:260] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.205443  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:05.187783 13800 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:05.188139 13757 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:05.216552  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:05.221203 13707 consensus_queue.cc:237] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.221588  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:05.187443 13461 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:15:05.222236 13821 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:05.226959 13795 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:05.227802  8063 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:05.228466  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:05.228708 13746 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:05.253186  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:05.255594  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:05.204175 13769 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.257202  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:05.266794  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:05.267805  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:05.268574  8057 raft_consensus.cc:2393] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:05.268514  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:05.273967  8057 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:05.225565 13753 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:05.292960 13585 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:05.256588 13712 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:05.257076 13400 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:05.294148  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:05.276093  8026 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:05.256723 13723 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:05.234409 13767 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.305967  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:05.269192 13800 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:05.270689 13716 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:05.271198 13821 raft_consensus.cc:697] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:05.304553 13400 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:05.310987 13400 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:05.313393  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3387) took 1557 ms. Trace:
I20260709 13:15:05.275350 13746 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:15:05.314735 13761 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:05.315222 13761 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.317576  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:05.317687  8026 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:05.313773  8205 rpcz_store.cc:276] 0709 13:15:03.756220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.881975 (+125755us) service_pool.cc:224] Handling call
0709 13:15:03.882194 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:03.985368 (+103174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:03.985398 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:03.985439 (+    41us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:03.985472 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:05.313160 (+1327688us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54002c8b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:05.313205 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:05.313325 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236859648}
I20260709 13:15:05.321444  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:05.321931  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:05.323707  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:05.302794 13585 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.237670 13757 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.271039 13769 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.344755 13829 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:05.345329 13829 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.322613 13660 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:05.346287 13769 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:05.346758 13660 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:05.347218 13660 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:05.348637  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:05.350497 13585 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.352028 13757 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:05.352538 13585 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:05.335687 13746 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:05.353586 13746 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:05.353933 13746 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:05.355003  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:05.341914 13828 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.352962 13829 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:05.338433 13761 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:05.358620 13828 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.361110 13828 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:05.366529 13846 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:05.367161 13846 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.370708  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:05.371593 13846 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:05.334225  8026 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:05.372743  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:05.373714  8057 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:05.371966  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:05.375312  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:05.375976  8056 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:05.303069 13753 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:15:05.390460  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:05.386556 13461 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:05.388717 13779 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:05.391122  8726 raft_consensus.cc:2368] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16.
I20260709 13:15:05.395594  8102 leader_election.cc:400] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16.
I20260709 13:15:05.395874  8102 leader_election.cc:403] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:15:05.392660  8026 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:05.397261  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:05.322170 13763 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:05.409586  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:05.411083  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:05.377815 13337 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:05.424408  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 149345us
I20260709 13:15:05.425493  8175 leader_election.cc:400] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:15:05.426388  8175 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:05.426482  8056 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:05.428141  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:05.429569  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:05.430325  8056 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:05.431378  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:05.426963  8057 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:05.432657  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:05.433372  8057 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:05.435278  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:05.443182  8175 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:05.443342  8056 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:05.390753 13090 raft_consensus.cc:3055] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:05.445937  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:05.396297 13757 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:05.412232 13402 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:05.412563 13678 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:15:05.453087  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:05.413671 13793 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:05.413846 13407 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:05.418982 13379 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:05.451706  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:05.424093 13585 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:05.470224  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:05.471879 13585 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.472316  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:05.473929 13585 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:05.474471 13585 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:05.474851 13585 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:05.447075 13821 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.475317 13712 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:05.475715 13712 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:05.476109 13712 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:15:05.476459 13813 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.997s	user 0.001s	sys 0.000s
I20260709 13:15:05.477855 13585 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:05.481334 13868 log.cc:927] Time spent T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:15:05.483386  8056 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:05.483994  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:05.484725  8059 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:05.485177  8057 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:05.471071  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:05.447149 13769 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:05.471129 13216 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:05.445331 13337 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.403139 13747 consensus_queue.cc:1059] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:15:05.450026 13716 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:05.497138 13767 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:05.487969  8204 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:15:05.452219 13090 raft_consensus.cc:740] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:05.422679 13746 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.468182 13723 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:05.419864 13763 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.493579  8056 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:05.495682  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:05.508325  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:05.499951  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 87454.5us
I20260709 13:15:05.490747 13216 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:05.503438  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:05.510058  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:05.500041  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:05.511761  8693 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:05.520928  8101 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:05.447234 13800 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.505019 13723 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:05.468474 13757 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.518671 13216 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:05.561919  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:05.563676  8101 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:05.522975 13678 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:05.549108 13774 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:15:05.556190 13820 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.964s	user 0.002s	sys 0.000s
I20260709 13:15:05.556003 13723 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:05.516049 13400 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.588413 13400 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:05.566905 13219 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:05.549667 13809 consensus_queue.cc:1059] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:15:05.589165 13703 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:05.576960 13774 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:05.560122 13795 consensus_queue.cc:1059] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:15:05.641131  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:05.569523 13752 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:15:05.560500 13779 consensus_queue.cc:1059] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 13:15:05.624444 13851 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.726s	user 0.004s	sys 0.000s
I20260709 13:15:05.574358 13712 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:05.634824 13866 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.372s	user 0.001s	sys 0.000s
I20260709 13:15:05.637344 13400 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:05.030439  8208 rpcz_store.cc:276] 0709 13:15:03.751676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.781262 (+ 29586us) service_pool.cc:224] Handling call
0709 13:15:03.781628 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:03.781819 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:15:03.820139 (+ 38320us) log.cc:844] Serialized 14 byte log entry
0709 13:15:04.841337 (+1021198us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b180049f6c0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb623a 00007f6dbb020a2a
0709 13:15:04.999451 (+158114us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:04.999515 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:04.999579 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:04.999656 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:04.999781 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2090,"prepare.queue_time_us":38171,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":3469,"raft.queue_time_us":2217,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":28636,"spinlock_wait_cycles":3678976,"thread_start_us":155661,"threads_started":1}
W20260709 13:15:05.648080  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 55005us
I20260709 13:15:05.631225 13763 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:05.645790 13757 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:05.642230 13774 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:05.656498 13400 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.661806 13400 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:05.663057 13752 raft_consensus.cc:3055] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:05.663441 13752 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:05.664151 13752 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.665248 13752 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:05.666813 13337 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:05.667307 13703 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:15:05.669428  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 175540us
I20260709 13:15:05.576381 13090 consensus_queue.cc:260] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.680922 13090 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:05.682314 13751 consensus_queue.cc:1175] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" responder_term: 12 status { last_received { term: 11 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:15:05.683283 13398 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:05.684260  8130 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:15:05.666687  8104 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:05.672874  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 161177us
I20260709 13:15:05.673003 13198 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:05.675207  8208 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:48152
I20260709 13:15:05.692912  8208 raft_consensus.cc:606] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:05.693354  8208 raft_consensus.cc:614] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:05.670596 13765 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
W20260709 13:15:05.678711  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:05.680070 13427 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:05.690058  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:05.719261  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:15:05.725065  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58443.1us
I20260709 13:15:05.730424  8061 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:15:05.685288 13750 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:15:05.735890  8693 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:05.723637 13757 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:05.739859  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:05.740908  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.741170  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.741266  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:05.744184  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:05.741900 13421 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:05.744431 13757 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.745007  8136 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:05.738670 13216 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.746589 13757 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:05.747465 13660 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.747963 13216 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:05.749518 13660 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:05.751814  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:05.752600  8726 raft_consensus.cc:2410] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:15:05.753326  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:05.755336  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:05.760000  8138 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:15:05.738262  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:05.763280  8693 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:05.744973 13421 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:05.774576  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.774638  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:05.770186  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 78019.4us
I20260709 13:15:05.771245  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:05.761039 13883 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:15:05.778281  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.742041 13735 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:05.776634  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:05.781528  8725 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 1005) took 2015 ms. Trace:
I20260709 13:15:05.782935  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.783655  8137 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:05.783874 13821 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:05.784248 13821 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:05.784574 13821 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:05.781908  8725 rpcz_store.cc:276] 0709 13:15:03.766382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.766718 (+   336us) service_pool.cc:224] Handling call
0709 13:15:03.786736 (+ 20018us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:04.695945 (+909209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:05.781328 (+1085383us) spinlock_profiling.cc:243] Waited 909 ms on lock 0x7b54006aead4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:05.781478 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2089674112}
I20260709 13:15:05.789132 13779 raft_consensus.cc:3055] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:05.789523 13779 raft_consensus.cc:740] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:05.790313  8101 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:05.790235 13779 consensus_queue.cc:260] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.793257  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:05.795729 13402 raft_consensus.cc:3055] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:05.796142 13402 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:05.796905 13402 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.797605 13723 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.798002 13402 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:05.767504 13817 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:05.766806 13780 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:05.804461  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.770593 13737 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.807401  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.778931 13421 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:05.810981  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.777199 13765 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:05.812680  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.782054 13763 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:05.805779 13198 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:05.823688  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.806288 13809 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:05.826431  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.827961  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.828547  8136 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:05.812126 13812 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:15:05.816994 13828 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:15:05.815016 13817 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:05.833689 13817 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.813122 13795 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:05.825580  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:15:05.858631 13838 log.cc:927] Time spent T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.977s	user 0.004s	sys 0.000s
I20260709 13:15:05.860220  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55838 (request call id 798) took 1345 ms. Trace:
I20260709 13:15:05.860561  8728 rpcz_store.cc:276] 0709 13:15:04.514483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:04.514623 (+   140us) service_pool.cc:224] Handling call
0709 13:15:04.514934 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:04.515248 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:04.515288 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:04.515703 (+   415us) log.cc:844] Serialized 26 byte log entry
0709 13:15:04.644000 (+128297us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:15:04.771653 (+127653us) log.cc:844] Serialized 14 byte log entry
0709 13:15:04.772577 (+   924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:04.773557 (+   980us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:05.295552 (+521995us) spinlock_profiling.cc:243] Waited 944 us on lock 0x7b18002f1d60. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ea3c0a 00007f6dc15e3219 00007f6dbeeea58f 00007f6db7f29571 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd
0709 13:15:05.755397 (+459845us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:15:05.860018 (+104621us) raft_consensus.cc:1685] finished
0709 13:15:05.860038 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:05.860165 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12443,"prepare.queue_time_us":215347,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":108344,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":22448,"spinlock_wait_cycles":8301696,"thread_start_us":253873,"threads_started":2,"wal-append.queue_time_us":330160}
I20260709 13:15:05.858661  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.800082 13735 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.817095 13751 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
W20260709 13:15:05.910717 13877 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 13:15:05.911303 13880 log.cc:927] Time spent T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:15:05.917225  8728 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:50008
I20260709 13:15:05.917654 13832 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:05.921265 13712 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:05.923174 13723 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:05.818164 13737 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.937350  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.816891 13779 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:05.946508  8210 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3398) took 1275 ms. Trace:
I20260709 13:15:05.885636 13809 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:05.947024  8693 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:05.824815 13198 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:05.952832  8210 rpcz_store.cc:276] 0709 13:15:04.671014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:04.671153 (+   139us) service_pool.cc:224] Handling call
0709 13:15:04.703176 (+ 32023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:04.703709 (+   533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:05.946279 (+1242570us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x7b54002c9050. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:05.946434 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73076992}
I20260709 13:15:05.853816 13802 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:05.952996  8132 raft_consensus.cc:2410] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 13:15:05.956915  8693 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:05.961835  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:05.926090 13774 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:05.967260  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 144915us
I20260709 13:15:05.824368 13765 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.853057 13839 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 13:15:05.971937  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:05.977087  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:05.977712  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:05.980167  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:05.953193 13761 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:15:05.981458 13712 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:05.981693 13760 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:15:05.982753 13876 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.348s	user 0.002s	sys 0.000s
I20260709 13:15:05.983610 13761 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:05.984072 13761 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:05.984967 13761 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:05.986927 13379 consensus_meta.cc:287] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.501s	user 0.007s	sys 0.002s
I20260709 13:15:05.987337 13379 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16.
W20260709 13:15:05.987566 13585 consensus_meta.cc:287] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.002s
I20260709 13:15:05.987978 13585 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:05.989143  8210 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:05.989914 13585 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:05.990473  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:05.996171  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.997355  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:05.997939 13752 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:05.986192 13761 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:05.999028 13752 raft_consensus.cc:3055] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:05.999426  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:05.999058  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:06.000541  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:06.000118 13846 raft_consensus.cc:3055] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:06.001477 13846 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:06.001541  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:06.002480 13846 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:06.000854  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:06.000368 13760 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:06.004830 13865 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:06.007768  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3396) took 1340 ms. Trace:
I20260709 13:15:06.004243  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:06.009016  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:06.008102  8203 rpcz_store.cc:276] 0709 13:15:04.667421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:04.667567 (+   146us) service_pool.cc:224] Handling call
0709 13:15:04.667843 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:04.698684 (+ 30841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:04.698715 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:04.699148 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 13:15:05.212881 (+513733us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b48002301d8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:15:06.007254 (+794373us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b18003f4de0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1
0709 13:15:06.007430 (+   176us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:06.007479 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:06.007566 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:06.007605 (+    39us) raft_consensus.cc:1685] finished
0709 13:15:06.007615 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:06.007709 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49527,"raft.queue_time_us":40921,"raft.run_cpu_time_us":2642,"raft.run_wall_time_us":497715,"spinlock_wait_cycles":15812224,"thread_start_us":15475,"threads_started":1,"wal-append.queue_time_us":49585,"wal-append.run_cpu_time_us":4968,"wal-append.run_wall_time_us":1242691}
I20260709 13:15:06.009717 13760 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:06.010418 13760 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:06.010785 13760 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:06.011089 13760 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:06.012571  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:06.013149  8205 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:06.013857  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:05.963783 13198 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:06.015048  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:06.019520  8104 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:06.021577 13735 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:06.022562 13735 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:06.022935 13735 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:06.023185  8211 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:06.024008  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1920) took 1321 ms. Trace:
I20260709 13:15:06.024374  8055 rpcz_store.cc:276] 0709 13:15:04.702530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:04.702932 (+   402us) service_pool.cc:224] Handling call
0709 13:15:04.703174 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:05.886724 (+1183550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:05.886754 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:05.886793 (+    39us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:15:05.886822 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:06.023734 (+136912us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540009c254. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:06.023812 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:06.023934 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2721572736}
I20260709 13:15:06.025928 13737 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:06.027344  8722 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:15:06.017767 13760 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.999434 13752 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:06.029016 13752 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:05.966143 13812 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:06.023305 13735 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:06.032307 13461 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:15:06.036659  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 56602.2us
I20260709 13:15:06.037418  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:06.038548  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:15:06.040073 13884 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:15:06.042575 13765 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:06.058200  8175 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:06.070685  8175 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:06.044097 13774 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:05.967885 13809 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:06.047626 13219 raft_consensus.cc:3055] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:06.048539 13798 raft_consensus.cc:3055] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:06.050323  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:06.035827 13560 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:15:06.051764  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.052517  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:06.064033  8204 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:15:06.089105  8204 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:06.065546  8133 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:06.056380 13712 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:06.051816 13585 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:06.067173  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:06.096014  8207 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 19: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:06.067061  8203 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:06.096623  8029 leader_election.cc:400] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15.
I20260709 13:15:06.097050  8029 leader_election.cc:403] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 13:15:06.071877  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.029484 13760 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:06.052826 13737 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:06.078156  8205 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:06.075539 13753 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:06.080906  8211 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:15:06.082795  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 194240us
I20260709 13:15:06.013952 13821 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.082185 13750 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:06.107161 13660 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15.
I20260709 13:15:06.131422 13751 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:06.105718 13585 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:06.132259 13751 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:06.085439  8210 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:06.132524 13585 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:06.098201  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:06.135195  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:06.137073  8029 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:06.137742 13413 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:06.079941  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:06.084435 13798 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:15:06.138651  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 58802us
I20260709 13:15:06.138788 13798 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.140121 13798 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:06.140398 13753 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:06.140763 13753 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:06.141103 13753 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:06.141137 13821 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:06.082396 13219 raft_consensus.cc:740] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:06.142554  8137 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 19: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:06.146314  8055 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:06.146910  8054 raft_consensus.cc:1240] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:06.083119 13752 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:06.055068 13379 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:06.095018 13402 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.178601 13402 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.181317 13402 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:06.151651  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:06.142881 13219 consensus_queue.cc:260] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.187728 13727 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:06.153110  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:06.188153 13219 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:06.189428  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44125.1us
I20260709 13:15:06.132681 13751 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:06.190117  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:06.190305 13746 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:06.190912 13751 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.191419  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:06.154279 13407 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:06.174269  8211 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:06.187088 13757 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:06.187849  8059 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:06.174047  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:06.199641  8060 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:06.192258  8029 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:06.082803 13421 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:06.194458  8210 raft_consensus.cc:2410] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:15:06.192042 13751 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:06.222183  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:06.224575  8101 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.230535  8059 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
I20260709 13:15:06.232061  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:06.232882  8207 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:06.215075 13497 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:06.191437 13746 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:06.241767  8060 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:06.244161  8207 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:15:06.238627  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 466607us
I20260709 13:15:06.256844  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:06.238897 13461 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:06.257431  8210 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:06.217377 13750 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:15:06.291951  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69878.1us
I20260709 13:15:06.266659  8210 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:06.292966  8134 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:06.232055 13737 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.292315  8101 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:06.299953  8696 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:06.197352 13379 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:06.244690 13497 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:06.303562  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:06.305186  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:06.306012  8055 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:06.306397  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:06.306200  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:06.307431  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:06.307997  8728 raft_consensus.cc:606] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:06.308095  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:06.308789  8059 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:06.308043  8136 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:06.311062  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:06.311403  8210 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:15:06.306783  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:06.314327  8055 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 19: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:06.261088 13796 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.312839  8101 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:15:06.315716  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 305521us
I20260709 13:15:06.314092  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:06.324270  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:06.334648  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:06.301424 13800 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:15:06.300822 13750 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.341763 13750 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:06.261792 13809 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.342365 13796 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.342747  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:06.311894 13497 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:06.345494 13815 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:15:06.348167 13878 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.722s	user 0.009s	sys 0.003s
I20260709 13:15:06.325596 13821 consensus_queue.cc:1059] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:15:06.299149 13402 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:06.351788  8203 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:15:06.354768  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:06.350026  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:06.356247  8129 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:06.357776  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:06.361060  8721 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 975) took 1013 ms. Trace:
I20260709 13:15:06.316655 13760 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:06.370695 13747 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:06.373152 13421 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:06.377364 13198 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:06.312181 13763 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
W20260709 13:15:06.383059 13892 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:15:06.361558  8721 rpcz_store.cc:276] 0709 13:15:05.347471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:05.349534 (+  2063us) service_pool.cc:224] Handling call
0709 13:15:05.349812 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:05.350120 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:05.350145 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:05.351634 (+  1489us) log.cc:844] Serialized 26 byte log entry
0709 13:15:05.572401 (+220767us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:15:05.572439 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:05.572502 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:06.347555 (+775053us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:15:06.351479 (+  3924us) raft_consensus.cc:1685] finished
0709 13:15:06.351506 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:06.360988 (+  9482us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1238,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":64,"spinlock_wait_cycles":27396864,"thread_start_us":220492,"threads_started":1,"wal-append.queue_time_us":274397}
I20260709 13:15:06.389672  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:06.390260  8129 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:06.393728  8059 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:06.395521  8059 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:15:06.404628  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:06.312333 13660 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:15:06.305261 13737 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:06.431286  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 118512us
I20260709 13:15:06.432065  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:06.334817 13570 consensus_queue.cc:1059] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:15:06.442121  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:06.385528 13893 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:06.350739 13795 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:15:06.450739  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 2294) took 1505 ms. Trace:
I20260709 13:15:06.433620 13750 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:06.451141  8135 rpcz_store.cc:276] 0709 13:15:04.941885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:04.942002 (+   117us) service_pool.cc:224] Handling call
0709 13:15:06.440610 (+1498608us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b54001f1dd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:06.447136 (+  6526us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35808512}
I20260709 13:15:06.452700 13750 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.440881 13398 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:06.460057  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:06.393898 13402 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:15:06.456946  8134 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:15:06.443396 13779 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:15:06.465045 13028 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:15:06.465777  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2308) took 1279 ms. Trace:
I20260709 13:15:06.455804 13821 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:06.474972  8135 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:06.477115 13750 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:06.477707  8133 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:06.468915 13779 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:06.478973 13881 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.845s	user 0.001s	sys 0.004s
I20260709 13:15:06.479856 13090 raft_consensus.cc:3055] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:06.480268 13090 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:06.480562 13779 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:06.481065 13090 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.482247 13090 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:06.466307  8134 rpcz_store.cc:276] 0709 13:15:05.186570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:05.186770 (+   200us) service_pool.cc:224] Handling call
0709 13:15:05.187549 (+   779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:06.292489 (+1104940us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b58007fa3d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf720aa 00007f6dbaff4447 00007f6dbb00cfe7 00007f6dbb00e7de 00007f6dbb018b49 00007f6dbb019416 00007f6dbb015d8f 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4
0709 13:15:06.464824 (+172335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:06.465706 (+   882us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1359211392}
I20260709 13:15:06.485919 13585 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.486910  8134 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:15:06.487345  8134 pending_rounds.cc:85] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 6
W20260709 13:15:06.487058 13894 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.149s	user 0.002s	sys 0.000s
I20260709 13:15:06.488267 13585 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:06.491302  8134 pending_rounds.cc:107] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:15:06.491698  8134 raft_consensus.cc:2889] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:06.463857 13570 consensus_queue.cc:1243] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:15:06.509383  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:06.520689  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:06.521363  8134 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:06.530709  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 227642us
I20260709 13:15:06.507454 13839 raft_consensus.cc:3055] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:06.502671 13893 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:06.507303 13398 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:06.541498  8134 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
W20260709 13:15:06.544620  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 218419us
I20260709 13:15:06.555788  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:06.556591  8134 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:15:06.548864  8026 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:06.569523  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:15:06.570259  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 239147us
W20260709 13:15:06.578915 13897 log.cc:927] Time spent T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:15:06.579905 13198 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:06.507050 13821 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:06.535022 13839 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:06.587195  8026 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:06.586091  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:06.588604  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:06.589226  8203 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:15:06.593504  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:06.574849 13402 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:06.600186  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:06.604776  8026 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:06.568519 13570 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:15:06.569130 13752 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:15:06.591462 13216 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:06.601168 13723 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:06.586241 13753 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.601382 13660 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:06.607414 13760 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:06.645025 13402 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:06.648584 13910 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:06.650601  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3254) took 1281 ms. Trace:
I20260709 13:15:06.650952  8057 rpcz_store.cc:276] 0709 13:15:05.369286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:05.491858 (+122572us) service_pool.cc:224] Handling call
0709 13:15:06.067746 (+575888us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b5800000c98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:15:06.067911 (+   165us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:06.068254 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:06.068293 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:06.078882 (+ 10589us) log.cc:844] Serialized 52 byte log entry
0709 13:15:06.503160 (+424278us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:06.519062 (+ 15902us) log.cc:844] Serialized 14 byte log entry
0709 13:15:06.528650 (+  9588us) log.cc:844] Serialized 14 byte log entry
0709 13:15:06.528855 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:06.528943 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:06.650374 (+121431us) raft_consensus.cc:1685] finished
0709 13:15:06.650396 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:06.650545 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1428,"prepare.queue_time_us":59710,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":80,"raft.queue_time_us":9698,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":11209,"spinlock_wait_cycles":9300736,"thread_start_us":348603,"threads_started":1,"wal-append.queue_time_us":516368}
I20260709 13:15:06.623291 13461 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:15:06.654523  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 285444us
I20260709 13:15:06.632094 13379 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:06.678248  8203 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 20.
I20260709 13:15:06.653883 13737 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:06.701556 13737 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.702051 13737 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.702658 13821 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:06.702785 13497 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.704125 13737 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:06.704830  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:06.705603  8130 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:06.680145 13216 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:06.707871  8134 raft_consensus.cc:2410] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:15:06.680677 13906 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.631464 13839 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.631304 13398 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:06.696637 13735 consensus_meta.cc:287] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.665s	user 0.009s	sys 0.001s
I20260709 13:15:06.686725 13760 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:06.674023 13461 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:06.731143  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:06.628309 13893 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.739861  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:06.768420  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:06.769754  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:06.773969  8130 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:06.778957  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:06.779665  8130 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:06.781268  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 159295us
I20260709 13:15:06.781467  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:06.781528  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:06.782871  8056 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 16: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:06.782364  8057 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 17: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:15:06.787528  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 214847us
I20260709 13:15:06.731657 13735 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.730290 13906 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:06.803464  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 203326us
I20260709 13:15:06.807139  8130 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:06.811003  8136 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:15:06.787147 13402 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:15:06.790891 13821 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 17.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:15:06.819418  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:06.801843 13753 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:06.820611  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:06.823675  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:06.826092  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:06.815362 13570 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.831081  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:06.821679 13402 consensus_queue.cc:1243] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:15:06.826443 13216 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:06.828052 13750 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:06.843875  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:06.842170 13216 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:06.850461 13216 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:06.851118 13889 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:06.851552 13889 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:06.852321 13889 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:06.844892 13585 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:06.853550 13889 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:06.843895  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58712.5us
I20260709 13:15:06.824750 13924 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:06.860535 13917 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:15:06.864342 13846 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:06.869030  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:15:06.877115 13902 log.cc:927] Time spent T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.186s	user 0.002s	sys 0.000s
W20260709 13:15:06.849977 13904 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:15:06.893503  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 49728.9us
I20260709 13:15:06.894506  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:06.895864  8063 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:06.837093 13497 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:06.903141  8063 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:06.904126  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:06.905709  8063 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:06.899173  8060 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:15:06.908371  8060 pending_rounds.cc:85] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 8
I20260709 13:15:06.908581  8060 pending_rounds.cc:107] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:15:06.908804  8060 raft_consensus.cc:2889] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:06.830698 13911 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 13:15:06.847141 13753 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:06.911449  8054 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:15:06.908823  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
W20260709 13:15:06.912755  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 121789us
I20260709 13:15:06.848510 13750 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:06.898487  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:06.921517  8055 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:15:06.924016  8063 raft_consensus.cc:2468] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:06.922101  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:06.838088 13570 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:06.933072  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:06.934527  8134 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 16: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:06.902976 13585 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:06.942667  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1719) took 1132 ms. Trace:
I20260709 13:15:06.943161  8137 rpcz_store.cc:276] 0709 13:15:05.810357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:05.810807 (+   450us) service_pool.cc:224] Handling call
0709 13:15:06.938223 (+1127416us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b540015b654. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:06.942594 (+  4371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":751277952}
I20260709 13:15:06.929344 13750 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:06.980173 13570 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:06.982769 13735 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:06.965312  8729 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 1008) took 2197 ms. Trace:
I20260709 13:15:06.966243  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:06.984946  8059 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:15:06.968037 13929 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:15:06.968397 13809 raft_consensus.cc:3055] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:06.993506  8137 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: []
W20260709 13:15:06.974900 13920 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:15:06.975198 13585 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:06.990083  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 877) took 2229 ms. Trace:
I20260709 13:15:06.995527  8062 rpcz_store.cc:276] 0709 13:15:04.754587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:04.754724 (+   137us) service_pool.cc:224] Handling call
0709 13:15:04.755087 (+   363us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:04.755480 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:15:05.141192 (+385712us) log.cc:844] Serialized 14 byte log entry
0709 13:15:05.234139 (+ 92947us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:06.471756 (+1237617us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b2c00037918. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db990fa11 00007f6db991f372 00007f6dbb051a01 00007f6dbb01643a 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2
0709 13:15:06.472403 (+   647us) log.cc:844] Serialized 52 byte log entry
0709 13:15:06.623137 (+150734us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:06.623177 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:06.623440 (+   263us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:06.958252 (+334812us) raft_consensus.cc:1685] finished
0709 13:15:06.984045 (+ 25793us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:06.984236 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":431557,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":173,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":32046,"spinlock_wait_cycles":45998336,"thread_start_us":575064,"threads_started":3,"wal-append.queue_time_us":461532,"wal-append.run_cpu_time_us":1298,"wal-append.run_wall_time_us":1061668}
I20260709 13:15:06.990423  8055 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:06.984234  8729 rpcz_store.cc:276] 0709 13:15:04.759534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:04.759642 (+   108us) service_pool.cc:224] Handling call
0709 13:15:04.759869 (+   227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:04.882320 (+122451us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:15:04.891040 (+  8720us) log.cc:844] Serialized 14 byte log entry
0709 13:15:05.299047 (+408007us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:05.299100 (+    53us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:05.299146 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:05.477089 (+177943us) spinlock_profiling.cc:243] Waited 7.13 ms on lock 0x7b4800031ed8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:15:06.956491 (+1479402us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400730854. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:06.956536 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:06.956651 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":613692,"prepare.queue_time_us":26532,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":21624,"raft.queue_time_us":389935,"raft.run_cpu_time_us":735,"raft.run_wall_time_us":168105,"spinlock_wait_cycles":297514624,"thread_start_us":25472,"threads_started":1,"wal-append.queue_time_us":26102,"wal-append.run_cpu_time_us":1037,"wal-append.run_wall_time_us":1001336}
I20260709 13:15:07.001416 13028 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:07.003727 13028 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:06.959280 13921 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:06.933516 13923 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:15:07.022760  8208 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:07.024417  8693 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:07.026786  8059 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 20.
W20260709 13:15:07.019811  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 111075us
I20260709 13:15:07.028650  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:07.028431  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:07.029898  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:06.992148 13809 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:06.984540 13798 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:15:07.002601 13924 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:07.019227 13585 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:07.000741 13832 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:15:07.027971 13906 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:07.031777 13763 raft_consensus.cc:2804] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:07.034142 13921 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:07.051918  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:07.060201  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:07.020843 13427 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:15:07.063910  8059 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:07.063974  8062 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:07.071787 13924 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:07.064774  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:07.074988  8055 raft_consensus.cc:2410] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:15:07.031917 13712 consensus_queue.cc:1059] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:15:07.064621 13811 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:15:07.069491 13925 log.cc:927] Time spent T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:15:07.069417 13216 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:07.071395 13918 log.cc:927] Time spent T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:15:07.104357  8062 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
W20260709 13:15:07.107251  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 78945us
I20260709 13:15:07.035790 13337 consensus_queue.cc:1059] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:15:07.047405 13933 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
W20260709 13:15:07.123672 13930 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:15:07.112486 13920 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:15:07.063501 13753 raft_consensus.cc:697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:07.103615 13763 raft_consensus.cc:697] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
W20260709 13:15:07.150787 13931 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:15:07.150856 13922 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 13:15:07.117823 13811 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 20 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:07.113229 13809 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.158084 13946 raft_consensus.cc:3055] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:07.155478 13750 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.165714 13763 consensus_queue.cc:237] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.166566 13800 raft_consensus.cc:3055] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:07.173398 13216 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:07.164589 13585 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.183579 13750 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:07.184983 13811 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.186847 13949 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:07.187367 13949 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.190156 13949 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:07.190915 13809 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:07.202270  8136 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:15:07.202886  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.172124 13800 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:07.206506 13800 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.207034  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:07.207459 13800 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:07.208778  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:07.211433  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:07.212262  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:07.213130  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:07.170066 13946 raft_consensus.cc:740] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:07.214219 13946 consensus_queue.cc:260] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.215382 13946 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:07.216920 13461 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:07.213287  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:07.218454  8063 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 17: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:07.219255  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:07.219487  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:07.220741  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:07.221421 13921 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:07.223572  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:07.225492  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:07.225437  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:07.221987  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:07.229267  8133 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:07.232710  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:07.233332  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:07.233815 13750 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:07.234306 13750 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:07.234501 13216 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:07.236150 13753 consensus_queue.cc:237] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.241323  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:07.200901 13737 raft_consensus.cc:3055] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:07.200462 13829 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.260070 13949 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:07.258659 13737 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:07.259291 13921 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:07.243672 13938 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:07.323112  8136 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:07.270749 13832 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.359038 13938 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:07.314954 13747 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.352830 13737 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.334127 13952 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 13:15:07.272210 13829 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.235251 13750 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.360975 13796 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:07.348507  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:15:07.385962  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 58346.9us
W20260709 13:15:07.338457 13930 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:15:07.364971 13921 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.389097 13921 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:07.387442 13753 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:07.381258 13723 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:07.380507 13938 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:07.444892  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.440843  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:07.443779 13812 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.410493 13753 raft_consensus.cc:697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:07.444938  8722 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:07.447189  8137 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:15:07.462890  8061 raft_consensus.cc:3055] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:07.463369  8061 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:07.465256  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2318) took 1219 ms. Trace:
I20260709 13:15:07.465641  8135 rpcz_store.cc:276] 0709 13:15:06.245674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.476744 (+231070us) service_pool.cc:224] Handling call
0709 13:15:06.477059 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:06.600710 (+123651us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:06.600756 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:06.600811 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:06.600858 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:07.464998 (+864140us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b540040a1d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:07.465057 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:07.465201 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283891968}
W20260709 13:15:07.467454  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.468799  8061 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.469760  8061 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:07.470584 13839 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:07.473492  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:07.463549  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:07.323689 13760 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:07.475915 13753 consensus_queue.cc:237] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.479230  8026 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:07.479714  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 136935us
I20260709 13:15:07.488813 13427 raft_consensus.cc:993] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:07.384486 13747 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.494316  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:07.495229  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:07.495723  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:07.498695  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.510164  8057 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:07.511432  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 873) took 1548 ms. Trace:
I20260709 13:15:07.511448  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:07.512184  8730 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:15:07.503531  8176 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:07.506224  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:07.440172 13751 raft_consensus.cc:3055] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:07.516022  8062 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:15:07.383852 13832 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.508796  8102 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:07.517486  8056 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:07.519572  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:07.522994  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:07.524590 13427 raft_consensus.cc:1081] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:07.501140 13911 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.505831 13940 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:07.511917  8138 rpcz_store.cc:276] 0709 13:15:05.962659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.097162 (+134503us) service_pool.cc:224] Handling call
0709 13:15:06.097503 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:06.350529 (+253026us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:15:06.384533 (+ 34004us) log.cc:844] Serialized 14 byte log entry
0709 13:15:06.782609 (+398076us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:06.782679 (+    70us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:15:06.782734 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:07.511237 (+728503us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b540008cad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:07.511277 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:07.511373 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34283,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":2779,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":3395,"spinlock_wait_cycles":581323648,"thread_start_us":397738,"threads_started":1,"wal-append.queue_time_us":518330}
I20260709 13:15:07.531944 13765 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:07.533012  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3256) took 1388 ms. Trace:
I20260709 13:15:07.535656  8138 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:15:07.515345 13637 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.561910 13637 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.515077 13216 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:07.543071 13911 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.567376 13911 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:07.532462 13796 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:15:07.533377  8058 rpcz_store.cc:276] 0709 13:15:06.144497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.144639 (+   142us) service_pool.cc:224] Handling call
0709 13:15:06.144936 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:06.212146 (+ 67210us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:06.245322 (+ 33176us) log.cc:844] Serialized 14 byte log entry
0709 13:15:06.591778 (+346456us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:06.592223 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 13:15:06.592358 (+   135us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:06.592393 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:07.518217 (+925824us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b5400333954. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:07.518260 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:07.518324 (+    64us) raft_consensus.cc:1685] finished
0709 13:15:07.532836 (+ 14512us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:07.532960 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32134,"prepare.queue_time_us":33050,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":4212,"raft.queue_time_us":44168,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":22860,"spinlock_wait_cycles":80881280,"thread_start_us":287749,"threads_started":1,"wal-append.queue_time_us":505075}
I20260709 13:15:07.454058 13812 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:07.549835 13962 log.cc:927] Time spent T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:15:07.551817 13747 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:07.558581  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.535247 13940 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:07.516163 13751 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:07.447785 13723 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.607421  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:50008 (request call id 4) took 1686 ms. Trace:
I20260709 13:15:07.607856  8728 rpcz_store.cc:276] 0709 13:15:05.903051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:05.903358 (+   307us) service_pool.cc:224] Handling call
0709 13:15:07.589346 (+1685988us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b54002e1154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb010853 00007f6dbffde60a 00007f6db95647d4 00007f6db9564762 00007f6db9564691 00007f6db956460c 00007f6db95638c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:07.589459 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":826567936}
I20260709 13:15:07.584338  8102 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:07.616339  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:07.514371 13760 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.608099  8058 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:37156
I20260709 13:15:07.617439  8056 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:15:07.622804 13812 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:07.623272 13760 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:07.624959 13723 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:07.625608 13751 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.7, Last appended by leader: 7, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.626623 13751 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:07.629590 13956 log.cc:927] Time spent T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 13:15:07.631044 13960 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:15:07.607295 13940 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:07.595155 13865 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:07.642514  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 58247.2us
I20260709 13:15:07.606710 13427 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:15:07.621493 13927 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:07.662007  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 147574us
I20260709 13:15:07.614358 13497 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
W20260709 13:15:07.663213  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 47952.8us
I20260709 13:15:07.663214  8722 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:50008
I20260709 13:15:07.666455  8722 raft_consensus.cc:606] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:07.666945  8722 raft_consensus.cc:614] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:07.644618 13779 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:07.673918  8027 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:07.677682  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:07.660668 13753 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:15:07.681625  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 137106us
I20260709 13:15:07.681663  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:07.644985 13865 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.679536  8212 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:07.690582  8203 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:07.682129  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:07.689066  8061 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:07.692622  8061 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3259) took 1545 ms. Trace:
I20260709 13:15:07.695537  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:07.695968  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:07.693056  8061 rpcz_store.cc:276] 0709 13:15:06.146991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.147124 (+   133us) service_pool.cc:224] Handling call
0709 13:15:06.191736 (+ 44612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:07.462804 (+1271068us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400263a20. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf8ddc0 00007f6dbb0181c0 00007f6dbb019231 00007f6dbb015d8f 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c
0709 13:15:07.692314 (+229510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:07.692555 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234734336}
I20260709 13:15:07.679888 13921 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.676393 13723 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:07.699712  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:07.700485 13407 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:15:07.706661  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:07.708240  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:07.682772 13949 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:07.681733 13585 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:07.702881  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:07.703563  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:07.714847  8204 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:07.716360  8204 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:07.716710  8208 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:07.679966 13966 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:07.720088  8209 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 19: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:07.665354 13927 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.709715  8211 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:48152
I20260709 13:15:07.722024  8058 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:07.693325 13779 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.711441  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:07.722498  8058 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:07.726552  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.726697  8061 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:15:07.698011 13938 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.719475 13966 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:07.729463  8212 raft_consensus.cc:1240] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:15:07.732900  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:07.687511 13753 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.736931  8056 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:15:07.738595  8204 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:15:07.741535  8058 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.747716  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:07.753736  8057 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 17: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:07.727816 13216 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:07.758586  8209 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:07.760574  8212 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:07.709506 13921 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.763098  8204 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:15:07.764853 13967 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:15:07.767586 13821 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.776837 13969 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.776964 13461 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:07.777835 13932 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:07.777850 13780 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:07.778733 13753 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:07.780124  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:07.782080 13927 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:07.743366 13966 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:07.714013 13723 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.786787  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:07.786932 13865 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:07.787317 13216 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.730211 13779 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:07.792799  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 62451us
I20260709 13:15:07.722661 13949 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.806140  8212 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:07.859354  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:07.859920  8212 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:15:07.861086  8212 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:07.810632  8104 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:07.859841 13949 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:07.862607 13949 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:15:07.862635  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 52117.4us
I20260709 13:15:07.863970  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:07.722786 13585 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.814860  8205 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:07.853824  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:07.861100  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:07.865103  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:07.865978  8204 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:07.861883  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:07.868662  8057 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:07.869534  8104 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:07.796216 13924 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:15:07.877724 13924 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:07.813292 13398 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:07.879938  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:07.881353  8138 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:07.871145 13765 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:07.811633 13932 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.883419 13765 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:07.883852 13982 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:07.876257 13751 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:07.867998 13800 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:07.893746 13946 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:07.894405 13946 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.895745 13921 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:07.897290  7957 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37774 (request call id 142) took 10256 ms. Trace:
I20260709 13:15:07.897420 13216 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:07.850639 13753 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:07.897994  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:07.902674  8211 raft_consensus.cc:606] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:07.905377  8211 raft_consensus.cc:614] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:07.906944  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:07.804181 13821 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.881160 13924 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.917021 13924 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:07.808260 13461 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.920710  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:07.811851 13969 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.874727 13949 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.927047  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:07.897697  7957 rpcz_store.cc:276] 0709 13:14:57.641183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.641354 (+   171us) service_pool.cc:224] Handling call
0709 13:15:07.897214 (+10255860us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:03.118742 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:03.246865 (+128123us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:03.253736 (+  6871us) write_op.cc:435] Acquired schema lock
0709 13:15:03.255494 (+  1758us) tablet.cc:662] Decoding operations
0709 13:15:03.423698 (+168204us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:03.423810 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:03.423849 (+    39us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:15:03.425105 (+  1256us) tablet.cc:701] Row locks acquired
0709 13:15:03.425125 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:15:03.425267 (+   142us) write_op.cc:270] Start()
0709 13:15:03.425380 (+   113us) write_op.cc:276] Timestamp: P: 1783602903425224 usec, L: 0
0709 13:15:03.425401 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:15:03.522521 (+ 97120us) log.cc:844] Serialized 6668 byte log entry
0709 13:15:03.864694 (+342173us) op_driver.cc:464] REPLICATION: finished
0709 13:15:03.916726 (+ 52032us) write_op.cc:301] APPLY: starting
0709 13:15:03.916870 (+   144us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:04.240109 (+323239us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b5400172d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb011e28 00007f6dbea4c581 00007f6dbea4bd58 00007f6dbea509f1 00007f6dbea509a9 00007f6dbea50939 00007f6dbea50901 00007f6dbea4fb2d 000000000038f354 000000000038f199
0709 13:15:04.603116 (+363007us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:04.603145 (+    29us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:05.633780 (+1030635us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:05.634092 (+   312us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=0,mrs_lookups=0
0709 13:15:05.643061 (+  8969us) write_op.cc:312] APPLY: finished
0709 13:15:05.661775 (+ 18714us) log.cc:844] Serialized 156 byte log entry
0709 13:15:05.679154 (+ 17379us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b1800073ea0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f29522 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:15:05.781061 (+101907us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:05.793962 (+ 12901us) write_op.cc:454] Released schema lock
0709 13:15:05.794028 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:05.794457 (+   429us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:05.794485 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1234609,"reported_tablets":26,"spinlock_wait_cycles":1323392,"child_traces":[["op",{"apply.queue_time_us":51749,"apply.run_cpu_time_us":47404,"apply.run_wall_time_us":1877686,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75187,"num_ops":17,"prepare.queue_time_us":513107,"prepare.run_cpu_time_us":59131,"prepare.run_wall_time_us":1135032,"raft.queue_time_us":114189,"raft.run_cpu_time_us":13399,"raft.run_wall_time_us":65437,"replication_time_us":439145,"spinlock_wait_cycles":156056704,"thread_start_us":749092,"threads_started":5,"wal-append.queue_time_us":397600,"wal-append.run_cpu_time_us":34479,"wal-append.run_wall_time_us":2158699}]]}
I20260709 13:15:07.928690  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:07.904031 13946 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:07.903488 13921 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:07.911823 13977 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:07.914692 13976 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:07.940316  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:07.945230 13461 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:07.945479 13969 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:07.948493 13976 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.950356  8058 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:07.951216 13976 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:07.920845 13753 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:07.957870 13398 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.958384 13398 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:07.959390 13932 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:07.960415 13398 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:07.962603 13723 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:07.945366  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:15:07.964309  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.967208 13585 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:07.968139  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:07.968371 13779 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:07.968734  8138 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:07.969663 13637 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:07.969954  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:07.970438 13938 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:07.970978 13938 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:07.971320 13938 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:07.971659 13938 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:07.971763  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:07.926649 13933 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:07.984402  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.984494  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:07.985451  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:07.940889  8725 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:07.961946  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:07.989445  8055 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:07.991671  8176 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:07.931654 13703 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.997812  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:08.000577  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:08.001127  8722 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:08.001540  8722 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:08.006453  8207 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:15:07.935906 13977 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:08.010936 13753 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.013126  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:08.013629  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.015832  8722 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.017093  8722 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:08.019109  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:08.021551  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:08.023288  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:08.024535  8029 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:07.999171 13637 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.028565  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:08.029639  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:08.032339  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:07.938992 13921 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:08.035053  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 54116.9us
I20260709 13:15:08.037637  8136 raft_consensus.cc:3055] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:08.036659  8058 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:15:08.004647 13980 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:08.038072  8136 raft_consensus.cc:740] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:08.038710  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.038619  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:08.042125  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:08.008631 13977 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:08.045624  8130 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 13.
I20260709 13:15:08.046396  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:08.005882 13984 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:08.018217 13585 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:08.032371  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 162900us
I20260709 13:15:08.061599 13821 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.005756 13703 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.062474 13585 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.062878 13946 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:08.064529 13585 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.065158  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.065649  8722 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
W20260709 13:15:08.068617  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.996807 13933 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.018020 13723 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:08.044873  8136 consensus_queue.cc:260] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.071202  8722 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:08.068964  8104 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:08.069166  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44697.1us
I20260709 13:15:08.071971  8029 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:08.042873 13461 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:08.076953  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:08.037566 13924 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:08.077811  8722 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:15:08.044248 13980 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:08.051220 13817 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:08.070354 13846 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:08.091717  8055 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:08.092339  8693 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:08.075943 13976 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:08.075280 13966 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.100996  8060 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:15:08.038450 13637 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.084954 13927 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.084841 13398 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:08.079819 13924 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:08.100579 13751 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:15:08.098083 13976 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:08.078714 13723 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.092481 13980 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:08.093725 13846 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.103149  8102 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:08.079315 13988 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:15:08.102315 13746 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.132894  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.134061  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.109097 13927 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.151077  8060 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:08.103430 13560 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:08.108532 13198 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:15:08.100750 13402 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:08.104279 13216 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:08.146301  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.148047  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:08.161669  8061 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:08.098064 13817 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.131795 13976 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:08.149586 13949 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.153574  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:08.129089 13924 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:08.155094  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.156240  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.144977 13846 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:08.147832 13988 consensus_queue.cc:1243] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:15:08.154448 13940 consensus_meta.cc:287] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.000s
I20260709 13:15:08.186655  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:08.150930 13800 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:08.107151 13934 consensus_queue.cc:1059] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:15:08.168804 13637 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.171522  8054 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:08.175308  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:08.175676  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:08.192143  8061 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:08.192384  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:08.160888 13402 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.152083 13746 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.193907 13703 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.196300  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:08.196907 13402 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:08.201426  8728 raft_consensus.cc:2393] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 17: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:08.201143  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.158782 13560 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.201093  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.189949 13800 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.203647  8061 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:08.190016 13949 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:08.164618 13216 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.212213  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.199846 13992 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:08.206720 13560 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:08.200569 13991 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:08.226150 13990 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:15:08.189903 13940 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.208732 13800 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:08.230665  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:08.231951  8728 consensus_queue.cc:237] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.242797  8054 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:15:08.220052 13992 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.250178  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.248112  8176 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:08.253866  8054 raft_consensus.cc:2393] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:08.268718 13746 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:08.270560 13966 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.271667 13817 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:08.276423 13723 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:08.277866 13216 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:08.281270 13938 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.229823 13927 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:08.253459 13992 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:08.287497  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.288199  8209 raft_consensus.cc:2393] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:08.288957  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.289593  8209 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 14: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:08.292397  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.293416  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.299610 13940 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:08.227370 13933 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.232743 13921 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.310829  8062 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:15:08.312646  8696 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:08.313645  8696 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:08.313661  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:08.312752  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.314693  8062 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 13: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:08.313814  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.319079  8208 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:15:08.319581  8696 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:08.290256  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.321475  8209 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:15:08.328678  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.327802  8104 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:08.331527  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.332005  8203 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 12.
I20260709 13:15:08.312408 13984 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:15:08.321151 13993 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:15:08.321653 13946 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:08.321914 13828 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:08.336757 13945 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:08.326838 13817 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:08.350531  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.336990 13812 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 13:15:08.349373  8101 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:08.358609  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.363963  8054 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:08.335206 13637 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:08.321513 13988 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:15:08.356033 13945 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.368247  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.369364 13703 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:08.357164 13828 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.370069 13988 consensus_queue.cc:1243] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:08.326505 13751 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:15:08.356894 13809 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.329341 13977 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.329535 13976 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.377709  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.378332  8054 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:08.380517  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.381681  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:08.384229  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.384500  8101 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:08.384852  8054 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:08.385080  8058 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:08.367311 13637 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.387440  8054 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:15:08.388150  8101 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:08.390415  8101 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:08.388484  8058 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:15:08.391616  8054 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:08.392880  8054 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:08.391387  8101 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:08.374958 13828 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:08.395686  8206 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:08.394369 13839 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:08.399264 13765 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:08.399747 13765 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.399402 13846 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.400161 13765 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:08.400854  8722 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 16: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:15:08.402447 13846 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:08.402845 13832 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:08.389253 13821 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:15:08.405524 13976 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:08.405691  8027 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:08.374961 13779 consensus_queue.cc:1059] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:15:08.407087 13779 raft_consensus.cc:3055] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:08.407439 13779 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:08.408176 13779 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.409245 13779 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:08.409334  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:08.415277 13753 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:08.393719 13637 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:08.393021 13988 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:08.431428 13980 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.431934  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:08.433450 13980 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.434947 13977 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:08.435600  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:08.436300  8207 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:15:08.436141 13924 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.437207  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.437281 13992 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.437799  8207 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:15:08.443295 13938 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:08.383136 13809 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.427452 13945 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:08.447703 13809 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:08.449527 13828 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.450614  8729 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 18: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:15:08.450938 13402 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.455161  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:08.455618 13793 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:08.455899 13992 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:08.456513 13219 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:08.456880 13796 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:08.457321 13796 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.457038 13219 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.457684 13796 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:08.459447 13219 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.438074 13924 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:08.465956  8138 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:15:08.466281  8138 pending_rounds.cc:85] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 7
I20260709 13:15:08.466544  8138 pending_rounds.cc:107] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:15:08.466766  8138 raft_consensus.cc:2889] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:08.466987  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.468519  8138 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:15:08.466295 13219 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:08.469091 13219 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.470402  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.471843  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:08.472918  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:08.474448  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:08.475773  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:08.477064  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:08.483180  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.414395 13703 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:08.492022  8135 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:15:08.494040 13219 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:08.495448  8132 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
W20260709 13:15:08.495544  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 104216us
I20260709 13:15:08.453063 13976 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.496425  8062 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:08.498996 13560 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:08.501094 13999 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:15:08.500293 13402 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:08.503818  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:08.508935  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 44309.5us
W20260709 13:15:08.510601  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58499us
I20260709 13:15:08.514525 13560 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:08.518666 13828 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:08.502260 13938 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:15:08.486704 13703 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.522904 13703 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:08.525009 13765 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.488616 13927 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.526844 13927 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.529285 13927 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.491664 13793 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:08.508416 13839 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:08.510206 13977 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:15:08.530930 13793 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.510916 13988 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:15:08.536679 13703 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.537662  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.544184 13934 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.546100 13934 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.547546  8210 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:15:08.548120 13765 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.549453  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.511101  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:08.548130 13934 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.550876  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:08.551517  8208 pending_rounds.cc:85] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 8
I20260709 13:15:08.551457  8721 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 12.
I20260709 13:15:08.552024  8208 pending_rounds.cc:107] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:15:08.552335  8208 raft_consensus.cc:2889] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:08.553172  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:08.554546  8133 raft_consensus.cc:2368] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:08.551280 13977 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:08.556627 13977 raft_consensus.cc:697] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:08.557446 13977 consensus_queue.cc:237] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.555761 13637 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.559146 13737 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:08.564896  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:08.568383 13660 raft_consensus.cc:3055] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:08.481423 13800 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.568782 13660 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:08.569594 13660 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.573498  8130 raft_consensus.cc:2368] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16.
I20260709 13:15:08.553099 13992 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:08.583012  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:08.584360  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:08.586582 13660 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:08.588261  8059 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:15:08.588908  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 78497.8us
I20260709 13:15:08.591741 13800 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.592319 13800 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:08.593813 13800 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:08.575816  8027 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:08.581362 13637 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:08.511499 13976 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.594640  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.595628  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.595772 13637 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:08.596369 13976 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:08.597597  8209 raft_consensus.cc:1217] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Deduplicated request from leader. Original: 9.6->[10.7-15.9]   Dedup: 12.8->[15.9-15.9]
I20260709 13:15:08.598089  8209 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: [10.7-15.9]
I20260709 13:15:08.598690 13796 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.599242  8209 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:08.600596 13796 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:08.596289  8133 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 19: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:08.602957  8175 leader_election.cc:400] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:08.603267  8175 leader_election.cc:403] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 13:15:08.603509  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:08.603946  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.601967  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:08.606112  8054 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:08.579499  8131 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:08.616235  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.617564  8138 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:08.618392  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.619076  8027 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:08.619428  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:08.618116  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.620258  8133 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:08.621223  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:08.621918  8061 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:08.623807  8129 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:08.632056  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:08.632529  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:08.642534  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:08.647042  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:08.647921  8209 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:15:08.648339  8061 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:08.650812  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.651578  8134 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:08.624866 13398 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:08.660104  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 55046.7us
I20260709 13:15:08.623273 13407 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:08.617339 13751 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.626004 13983 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:08.638568 13402 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:08.673995  8054 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:08.676110  8133 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:08.677011  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:08.677676  8133 raft_consensus.cc:2368] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16.
I20260709 13:15:08.678406  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.596319 13637 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:08.680613  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.647262 13985 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:08.644243 13980 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:08.672797 13407 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.645907 13793 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.687918  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:08.728356  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48152 (request call id 1) took 1033 ms. Trace:
I20260709 13:15:08.687091  8209 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:15:08.659904 13926 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:08.728735  8211 rpcz_store.cc:276] 0709 13:15:07.694806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:07.695064 (+   258us) service_pool.cc:224] Handling call
0709 13:15:08.728168 (+1033104us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b540019ebd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb010853 00007f6dbffde60a 00007f6db95647d4 00007f6db9564762 00007f6db9564691 00007f6db956460c 00007f6db95638c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:08.728267 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415013120}
I20260709 13:15:08.729142 13926 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.729787  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:08.729979  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:08.730737  8209 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:15:08.700191  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:08.652441 13753 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:15:08.733110 13926 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:08.733462  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:08.705132 13829 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:08.702809  8205 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3438) took 1012 ms. Trace:
I20260709 13:15:08.735106  8205 rpcz_store.cc:276] 0709 13:15:07.690259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:07.694878 (+  4619us) service_pool.cc:224] Handling call
0709 13:15:07.695126 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:07.865100 (+169974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:08.702573 (+837473us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5400241dd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:08.702752 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270569344}
I20260709 13:15:08.736915  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:08.726732  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.693910 13637 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:08.700555 13985 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:08.724639 13407 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:08.731712  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:08.808115  8133 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:08.808524  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.809235  8134 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:15:08.810202  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:08.673168 13398 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.734249  8206 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:15:08.699543 13995 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:15:08.681411  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:08.687003 13991 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.120s
I20260709 13:15:08.795964  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.684491 13751 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.682979 13988 consensus_queue.cc:1059] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:15:08.664285 13924 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:08.786973  8209 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:08.679845 13821 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.133s
I20260709 13:15:08.840492  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:08.873687  8205 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:08.805465 13832 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:08.845965  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 164643us
I20260709 13:15:08.884251  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:08.807564 13982 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.859341  8198 tablet_service.cc:1558] Processing DeleteTablet for tablet b186be823643475982e269b73d886a42 with delete_type TABLET_DATA_TOMBSTONED (TS d9cd2999aa314b39997b423ec1ae509e not found in new config with opid_index 8) from {username='slave'} at 127.0.0.1:48166
I20260709 13:15:08.735356 13793 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.838165 13800 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:08.895013 13398 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:08.895165 13793 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:08.845917 14003 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 13:15:08.870500  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 457443us
W20260709 13:15:08.870549  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 452022us
I20260709 13:15:08.890889 13938 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:08.892560 13982 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.894614 13800 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.901479  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:08.868994 13949 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:08.890805 13832 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.896646 13821 consensus_queue.cc:1243] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:08.912387 13938 raft_consensus.cc:697] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:08.895583 13985 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:08.915715 13949 raft_consensus.cc:697] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:08.930306 13983 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:08.965369  8026 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:08.970240  8029 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:08.976609  8054 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:08.882642  8206 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:08.977388  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:08.978888  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 317508us
I20260709 13:15:08.979328  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:08.979585  8059 raft_consensus.cc:1240] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:08.980661  8729 raft_consensus.cc:2368] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18.
W20260709 13:15:08.980782  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:08.981328  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:08.871289  8176 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:08.978097  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:08.999878 13982 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:08.999634 13938 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.000725 13923 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:09.003963 13932 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:09.004094 13800 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:08.980650  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:09.019851  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 148474us
I20260709 13:15:09.022038  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:09.023409  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:09.024353  8058 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:09.025377 13832 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:09.025605 13461 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:09.026189 13461 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:09.026619 13461 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:09.026739 13924 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:09.034951 13991 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.035915 13991 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.035998 13560 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:15:09.038126  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.039604  8131 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:09.046005  8176 leader_election.cc:400] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18.
W20260709 13:15:09.046911  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 45538.7us
I20260709 13:15:09.047652 13821 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:09.048440  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:09.048651 13821 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.050681 13977 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.050891 14011 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:09.050904 13796 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:09.051479 13977 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.051826 14011 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:09.051955 13934 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:09.052865 13934 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.053401 14011 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.012684 13923 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:09.106154  8176 leader_election.cc:403] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 13:15:09.072953  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 92141.5us
I20260709 13:15:09.014387 13932 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:09.068176  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:09.054138 13977 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:09.058185  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.075093  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:09.054629  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:09.072583 13982 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:09.063032 13821 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:09.127336  8208 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 14: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:09.128266  8175 leader_election.cc:400] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16.
I20260709 13:15:09.124562  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:09.131592  8175 leader_election.cc:403] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:09.108146 13923 raft_consensus.cc:3060] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:09.120962 13932 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:09.055814 13934 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:09.124912 13991 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:09.128835  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 78465.7us
I20260709 13:15:09.133278  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.060293 13933 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:09.176818  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:09.177835  8059 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:09.171780  8138 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
W20260709 13:15:09.181368  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 136006us
I20260709 13:15:09.182818 13980 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:09.146273 13982 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:09.183097  8104 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:09.192098  8211 raft_consensus.cc:2468] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
W20260709 13:15:09.192531  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.195236 13983 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.195961 13983 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.197918 13949 consensus_queue.cc:237] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:09.198446  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 233478us
W20260709 13:15:09.198396  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 211709us
I20260709 13:15:09.230365  8058 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:09.235725 13988 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:09.236984  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:09.238262  8730 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 16: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:15:09.238402 13982 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.239322 13750 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16.
W20260709 13:15:09.243075  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 115043us
I20260709 13:15:09.246479  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:09.233389  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.198321 13983 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:09.206568 13780 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:09.199730 13751 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:09.205785 13765 raft_consensus.cc:3055] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:09.275719  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:15:09.201233 13998 log.cc:927] Time spent T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.700s	user 0.003s	sys 0.008s
W20260709 13:15:09.275985  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 93076us
I20260709 13:15:09.199007 13757 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:09.281069  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.250396 13988 raft_consensus.cc:697] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:09.254616 13780 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:09.276376 13765 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:09.285466 14011 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:09.295204  8730 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 14: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:15:09.290858  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:09.299527 14017 tablet_replica.cc:333] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:15:09.276857 13461 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.306322 13865 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:09.308202 13461 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:09.308231 13988 consensus_queue.cc:237] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.310835  8722 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:15:09.311981  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.313009  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 913) took 1278 ms. Trace:
I20260709 13:15:09.279284 13933 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.313496  8055 rpcz_store.cc:276] 0709 13:15:08.033932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.034063 (+   131us) service_pool.cc:224] Handling call
0709 13:15:09.293215 (+1259152us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b5400169254. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:09.312765 (+ 19550us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59761024}
I20260709 13:15:09.315209 14011 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:09.316126  8136 raft_consensus.cc:3060] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:09.316584 14011 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.331676  8056 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:09.334764 13703 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:09.336530  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.338271  8730 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:50008
I20260709 13:15:09.339087  8730 raft_consensus.cc:606] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:09.339277  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.340569  8137 raft_consensus.cc:2410] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:15:09.344352  8056 raft_consensus.cc:2410] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:15:09.345537 13660 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 13:15:09.347237  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:15:09.349378  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.350950  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.352429  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.353401  8026 leader_election.cc:400] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16.
I20260709 13:15:09.353695  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.353875 13793 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:09.354614  8058 raft_consensus.cc:2393] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 16: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:09.354549 13793 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.355546  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.356133 13780 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.357574 13413 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:09.357657 13780 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:09.358815  8058 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:09.353718  8026 leader_election.cc:403] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:15:09.362645  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.331321 13934 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:15:09.363564 13933 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:09.365046  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:09.368116  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.369130  8129 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:09.369822  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.370168  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.371151  8207 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 18: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:09.373718 13461 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:09.373925  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 3288) took 1023 ms. Trace:
I20260709 13:15:09.358330 13413 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:09.356865 13793 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:09.374318  8062 rpcz_store.cc:276] 0709 13:15:08.350161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.350295 (+   134us) service_pool.cc:224] Handling call
0709 13:15:09.373528 (+1023233us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b5400335c54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:09.373729 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315014016}
I20260709 13:15:09.373739  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:09.375653  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.381469  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.384029  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.383517  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.391782  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.399385  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:09.396878  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 1034) took 1630 ms. Trace:
W20260709 13:15:09.381860 13737 consensus_meta.cc:287] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.822s	user 0.004s	sys 0.008s
I20260709 13:15:09.395192  8055 raft_consensus.cc:1240] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:09.386009  8129 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
W20260709 13:15:09.389092 13407 consensus_meta.cc:287] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.568s	user 0.009s	sys 0.009s
W20260709 13:15:09.492061  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 138810us
I20260709 13:15:09.363739 13934 raft_consensus.cc:3055] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:09.398156 13765 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.487954  8059 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:09.390844 13413 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:09.489403  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 119216us
I20260709 13:15:09.489568  8129 pending_rounds.cc:85] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 8
I20260709 13:15:09.495613  8129 pending_rounds.cc:107] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:15:09.496024  8129 raft_consensus.cc:2889] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:09.502827 13933 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16.
I20260709 13:15:09.492256 13407 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.514721 13984 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.514827  8026 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.515435 13984 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.516714 13407 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:09.517710  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:09.518312  8722 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:09.462621 13982 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:09.520824 13982 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.521226 13932 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.525091  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.525527  8026 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.525823  8055 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:15:09.494459 13934 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:09.534302  8056 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:15:09.494524 13750 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.551275  8137 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:15:09.552059  8136 raft_consensus.cc:1275] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:15:09.552400 13402 consensus_meta.cc:287] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.870s	user 0.014s	sys 0.005s
I20260709 13:15:09.552819  8136 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 891) took 1516 ms. Trace:
I20260709 13:15:09.552874 13402 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:09.553552 13970 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:09.554491  8133 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:15:09.557343 13984 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:09.558537 13932 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:09.558462 13750 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.559087 13934 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.560674 13934 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:09.560953 13750 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:09.553247  8136 rpcz_store.cc:276] 0709 13:15:08.036587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.037236 (+   649us) service_pool.cc:224] Handling call
0709 13:15:08.037503 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.315170 (+1277667us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b58009b45d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf720aa 00007f6dbaff4447 00007f6dbb00cfe7 00007f6dbb00e7de 00007f6dbb018b49 00007f6dbb019416 00007f6dbb015d8f 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4
0709 13:15:09.552528 (+237358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.552750 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":874752256}
I20260709 13:15:09.550609  8026 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.564085  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:09.566157  8057 raft_consensus.cc:2410] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:15:09.566561  8722 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
W20260709 13:15:09.567823  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 72049.4us
I20260709 13:15:09.571727 13757 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:09.562727 13660 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:09.558538 13982 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:09.487347  8204 rpcz_store.cc:276] 0709 13:15:07.766045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.000462 (+234417us) service_pool.cc:224] Handling call
0709 13:15:08.000699 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:08.075842 (+ 75143us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:15:08.158293 (+ 82451us) log.cc:844] Serialized 14 byte log entry
0709 13:15:08.247230 (+ 88937us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:08.247281 (+    51us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:15:08.247335 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.396352 (+1149017us) spinlock_profiling.cc:243] Waited 74.9 ms on lock 0x7b54002c9554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:09.396424 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:09.396669 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30479,"prepare.queue_time_us":17702,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":1147,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":31096,"spinlock_wait_cycles":172321664,"thread_start_us":158547,"threads_started":2,"wal-append.queue_time_us":208449}
I20260709 13:15:09.568850 13750 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.558285 13980 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18.
I20260709 13:15:09.579057 13461 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:09.606041 13461 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.576238 13984 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:09.602403 13407 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:09.619903 13751 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:09.619731 13809 consensus_queue.cc:1059] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:15:09.622654 13865 consensus_queue.cc:1048] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:15:09.649204  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 115032us
I20260709 13:15:09.608580 13750 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.595067 13757 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.617238 13923 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.661618  8027 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:09.661111 13461 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:09.670083  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:09.678996  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:09.682179  8054 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 19: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:09.682292  8057 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 13:15:09.684103  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 116175us
I20260709 13:15:09.652170 13984 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.685475  8026 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.687250 13461 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:09.695986  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.694161  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:09.699522  8725 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:15:09.699749 13753 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:09.700119 13753 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:09.700448 13753 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:09.701537  8026 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:09.702998  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.704041 13815 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:09.704459 13793 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:09.708495 13923 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:09.711359 13413 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.693720 13660 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:09.712471 13678 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:09.713089 13678 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.685262 13809 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:09.666843 13751 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.715309 13678 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:09.716326  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.717777 13413 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:09.718607 13809 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.721002 13932 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:09.721580 13932 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:09.723548 14001 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.878s	user 0.002s	sys 0.000s
I20260709 13:15:09.723687 13932 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:09.721576  8055 raft_consensus.cc:1240] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: [17.8-17.8]
I20260709 13:15:09.726646  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3457) took 1629 ms. Trace:
I20260709 13:15:09.726774  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.727934  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.727049  8203 rpcz_store.cc:276] 0709 13:15:08.096918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.361742 (+264824us) service_pool.cc:224] Handling call
0709 13:15:08.362017 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:08.362367 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:08.364359 (+  1992us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:08.367162 (+  2803us) log.cc:844] Serialized 26 byte log entry
0709 13:15:08.576774 (+209612us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:08.576812 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:08.576883 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:09.722739 (+1145856us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:15:09.726164 (+  3425us) raft_consensus.cc:1685] finished
0709 13:15:09.726184 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:09.726455 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":4680,"spinlock_wait_cycles":51955456,"thread_start_us":187777,"threads_started":1,"wal-append.queue_time_us":349836}
I20260709 13:15:09.728475  8206 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:09.726668  8101 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.729658  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.730327  8101 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:09.731952 14000 log.cc:927] Time spent T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 1.231s	user 0.001s	sys 0.009s
I20260709 13:15:09.733280  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.733544  8101 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:09.687848 13938 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:09.734892 13984 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:09.735683 14042 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:09.736405 14042 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:09.736501 13757 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:09.737259 14042 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:09.737661 14042 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:09.739705  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 2341) took 1148 ms. Trace:
I20260709 13:15:09.740178  8131 rpcz_store.cc:276] 0709 13:15:08.591493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.615966 (+ 24473us) service_pool.cc:224] Handling call
0709 13:15:09.739442 (+1123476us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b540035cfd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:09.739603 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837226368}
I20260709 13:15:09.740607  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:09.741672 13461 raft_consensus.cc:697] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:09.725718  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:09.742643  8059 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 16: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:09.742503  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:09.742827 13461 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.743512  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:09.728739  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.745620  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:09.746902  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.749900 13398 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.728546 13750 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:09.750458 14011 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.750919 13703 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:09.750960 14011 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.751765 13703 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.753286 14011 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:09.753892 13703 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:09.725071  8205 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:15:09.737926 13751 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:09.753362 13219 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:09.764285  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 73574.1us
W20260709 13:15:09.725965 14027 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.424s	user 0.001s	sys 0.000s
I20260709 13:15:09.753803 13398 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.761435  8059 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:15:09.738005 14042 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:09.775107 13398 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:09.766880 13219 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.776569  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 3472) took 1170 ms. Trace:
I20260709 13:15:09.777022 13219 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:09.771549  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 55188.5us
I20260709 13:15:09.778522 13765 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:09.765177  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:09.780175  8207 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:09.780218 13985 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:09.780820 13985 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:09.781168 13985 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:09.781332  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.775287  8138 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:15:09.766083 13924 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:09.786242  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.782104  8207 raft_consensus.cc:2393] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 17: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:09.782719  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:09.793385  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:09.776903  8211 rpcz_store.cc:276] 0709 13:15:08.606562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.729583 (+123021us) service_pool.cc:224] Handling call
0709 13:15:09.770846 (+1041263us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b540025d5d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:09.776494 (+  5648us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582536448}
I20260709 13:15:09.785115  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:09.782867  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:09.785876  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.790499 14017 raft_consensus.cc:2243] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:09.814122  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:09.784629  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.816502  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.817617  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:15:09.798442  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.824554  8101 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:09.825707 13757 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:09.826365  8101 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:09.826311 13757 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.828392 13988 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:09.828609 13757 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:09.829006 13988 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:09.830036 13988 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.814575  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.832592  8203 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 19: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:09.832008  8205 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
W20260709 13:15:09.816069 14018 log.cc:927] Time spent T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.502s	user 0.001s	sys 0.001s
W20260709 13:15:09.839207  8103 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.842439  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.843446 13832 consensus_queue.cc:1059] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:09.844357 14017 raft_consensus.cc:2272] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:09.845240  8104 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.845522 13753 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.846330  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:09.846644 13821 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:09.847239 13821 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.782032  7960 catalog_manager.cc:5697] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:09.857195  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.858268  8060 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:15:09.859120  8059 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:15:09.862797  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:09.861997  7960 catalog_manager.cc:5697] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 6 to 7, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 7 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:09.866971  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.866024  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.871587  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:09.862331 13678 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.872394 13832 raft_consensus.cc:3055] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:09.872835 13839 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:09.875047 14042 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.872826 13832 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:09.877087  8026 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.877161 13832 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.878240  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1773) took 1266 ms. Trace:
I20260709 13:15:09.878546 13832 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:09.878643  8130 rpcz_store.cc:276] 0709 13:15:08.593419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.721042 (+127623us) service_pool.cc:224] Handling call
0709 13:15:08.721396 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:08.982395 (+260999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:08.982463 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:08.983486 (+  1023us) log.cc:844] Serialized 26 byte log entry
0709 13:15:09.135208 (+151722us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:09.135257 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.730667 (+595410us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b5400032ad4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:09.730713 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:09.815322 (+ 84609us) spinlock_profiling.cc:243] Waited 6.72 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:15:09.848167 (+ 32845us) raft_consensus.cc:1685] finished
0709 13:15:09.848186 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:09.859659 (+ 11473us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1309,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":345,"spinlock_wait_cycles":613102336,"thread_start_us":148233,"threads_started":1,"wal-append.queue_time_us":310459}
I20260709 13:15:09.880039 13923 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:09.880434 13923 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:09.880590  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:09.880849 13923 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:09.883010  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.872867 13678 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:09.886699 14022 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.567s	user 0.003s	sys 0.010s
I20260709 13:15:09.886835 13809 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:09.852126 13398 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:09.889309 13398 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:09.889680 13398 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:09.890544  8209 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 19: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:09.891346  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3283) took 2395 ms. Trace:
I20260709 13:15:09.891726  8063 rpcz_store.cc:276] 0709 13:15:07.495525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:07.514770 (+ 19245us) service_pool.cc:224] Handling call
0709 13:15:07.515056 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:07.520511 (+  5455us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:07.520538 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:07.520937 (+   399us) log.cc:844] Serialized 26 byte log entry
0709 13:15:09.070544 (+1549607us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b1800428ce0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb60d1 00007f6dbafd1a0f
0709 13:15:09.187759 (+117215us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:09.198503 (+ 10744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.198636 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:09.885960 (+687324us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:15:09.891037 (+  5077us) raft_consensus.cc:1685] finished
0709 13:15:09.891057 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:09.891168 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54666,"raft.queue_time_us":5208,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"spinlock_wait_cycles":36420224,"thread_start_us":61449,"threads_started":1,"wal-append.queue_time_us":1797721}
I20260709 13:15:09.893787  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.894160 13753 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:09.894497  8059 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:09.895211  8693 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.895293  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.896340 13865 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:09.897187 13865 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 18 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:09.793260 13985 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.899077  8060 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:09.900564 13985 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:09.901508  8208 raft_consensus.cc:2368] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15.
I20260709 13:15:09.903790 13991 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:09.903949  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:09.904482 13991 consensus_queue.cc:1059] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:09.905952 13984 raft_consensus.cc:3055] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:09.906402 13984 raft_consensus.cc:740] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:09.907444 13984 consensus_queue.cc:260] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.908846 13757 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:09.878847  8026 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.909538 13757 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.914647  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.915172 13747 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:09.915334  8133 raft_consensus.cc:2368] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 18. Current term is 19.
I20260709 13:15:09.915627 13747 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:09.916064 13747 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:09.917479  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.919121  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:09.919636  8138 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:09.895514  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.925985  8055 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:09.894068  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.897181  8059 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:09.893181  7960 catalog_manager.cc:5697] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:15:09.938843  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 60100.3us
I20260709 13:15:09.939431  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.942621  8029 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.944717  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:09.945187  8131 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:15:09.944717  8104 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.949734  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:09.949868  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.950600  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.921053 13765 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:09.953194  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.877228 14042 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:09.951210  8056 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:09.951792  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:09.955576 13815 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:15:09.956135 14039 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.100s	user 0.002s	sys 0.000s
I20260709 13:15:09.954304  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.944190  7960 catalog_manager.cc:5697] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 9 to 11, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 11 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:09.960482  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:15:09.961151  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 57810.1us
I20260709 13:15:09.964406  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:09.958873 13751 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:09.973788  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.974509  8104 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:09.952739 13821 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:09.977465  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:09.898131 13865 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:09.982738  8133 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:15:09.951017 13923 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.978333  8203 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 20.
I20260709 13:15:09.953399 13765 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:09.962532 13932 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:09.984622  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 1986) took 1015 ms. Trace:
I20260709 13:15:09.984735  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:09.989809  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:09.989279 13560 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:09.992329 13839 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:09.992908 13940 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:15:09.993911  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:09.994277  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 107660us
I20260709 13:15:09.994593  8129 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:09.995117  8102 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:09.996377 13809 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:09.997231  8102 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:09.997069 13398 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:09.998126 13678 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:10.009960  8130 raft_consensus.cc:2393] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 19: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:15:10.012709  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 73222.2us
I20260709 13:15:10.016170 13839 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:10.016772 13923 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:09.989144 13765 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:10.018877 13398 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:10.019501 13398 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:15:10.024878  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.998497 13413 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.031746  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:10.013387  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:10.033962  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:10.036417  8207 raft_consensus.cc:2410] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:15:10.037230  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:10.037915  8130 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 19: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:10.037956  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:10.038800  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:10.039076 13828 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:10.040153 13828 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:10.040037  8130 raft_consensus.cc:3055] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:10.040690  8130 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:10.040934  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:10.041448  8054 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:15:10.041366  8130 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.042377  8130 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:10.042444  8134 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:10.043895  8134 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:10.043262  7960 catalog_manager.cc:5697] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 12 to 13, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 13 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:10.045779  8060 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:10.046909  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:10.020771 13198 raft_consensus.cc:2955] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 3 to 11, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:10.051882 14017 ts_tablet_manager.cc:1916] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:10.054689  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:10.055236  8061 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:10.057411  8055 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:10.045414  7960 catalog_manager.cc:5697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 6 to 12, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 12 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:10.062564  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 96201.3us
I20260709 13:15:10.067713  8054 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:10.075944  8133 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:15:10.077489  8104 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:10.080842  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:10.080925  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:10.074589 13413 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.077298  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:10.091081  8130 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:15:10.091460  8130 pending_rounds.cc:85] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 8
I20260709 13:15:10.091697  8130 pending_rounds.cc:107] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:15:10.091949  8130 raft_consensus.cc:2889] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:10.102602 13980 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:10.103165 13980 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:10.103605 13980 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:10.107796  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:10.110003  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:10.108609  8132 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:10.100477 13832 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:10.113176  7960 catalog_manager.cc:5697] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:10.098593 13988 raft_consensus.cc:2804] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:15:10.139544  8102 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.122159  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:10.111974 13940 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:10.173259 14052 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 13:15:10.174717  7960 catalog_manager.cc:5697] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 10 to 12, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 12 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:10.176487 14046 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:15:10.176826  7960 catalog_manager.cc:5697] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 7 to 10, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 10 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:10.163902 13988 raft_consensus.cc:697] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 19 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:10.181557 13988 consensus_queue.cc:237] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.163264 13703 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.203879 13703 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.207549 13980 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.209604 13980 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:09.988776  8062 rpcz_store.cc:276] 0709 13:15:08.969673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:09.523355 (+553682us) service_pool.cc:224] Handling call
0709 13:15:09.523647 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.523970 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:09.685257 (+161287us) log.cc:844] Serialized 14 byte log entry
0709 13:15:09.883952 (+198695us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b18004dca40. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb623a 00007f6dbb020a2a
0709 13:15:09.976191 (+ 92239us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:09.976236 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:09.984354 (+  8118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.984415 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:09.984555 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1517,"prepare.queue_time_us":170536,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":20708,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":10704,"spinlock_wait_cycles":2860800,"thread_start_us":241797,"threads_started":2}
I20260709 13:15:10.211771  8138 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:10.238581 13747 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.249817  8131 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:10.251927  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:10.262648 13413 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:10.265203  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 227300us
I20260709 13:15:10.266201  8693 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:10.267733  8061 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:10.272006 13984 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:10.274595  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:10.280398  8135 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:10.281672 13198 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:38710 (request call id 6) took 2054 ms. Trace:
I20260709 13:15:10.282109 13198 rpcz_store.cc:276] 0709 13:15:08.227700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.227822 (+   122us) service_pool.cc:224] Handling call
0709 13:15:08.245003 (+ 17181us) log.cc:844] Serialized 494 byte log entry
0709 13:15:08.834651 (+589648us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b180047cf20. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7ea3c0a 00007f6dc15e3219 00007f6db7f2ba40 00007f6db7f264e4 00007f6db7f26455 00007f6dbaf6fb0b 00007f6dbaff4b63 00007f6dbb01c752 00007f6dbb0149fa
0709 13:15:09.730958 (+896307us) spinlock_profiling.cc:243] Waited 7.97 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:15:10.281560 (+550602us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":475326,"raft.queue_time_us":472203,"raft.run_cpu_time_us":1406,"raft.run_wall_time_us":56068,"spinlock_wait_cycles":25600256,"thread_start_us":120184,"threads_started":1,"wal-append.queue_time_us":209667}
I20260709 13:15:10.284294 13757 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:10.284606 13747 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:10.286772  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:10.286828  7946 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b578d88c1c1f4ca58b7291fed4909f09 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 13)
I20260709 13:15:10.289556  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:10.290381  8203 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 19: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:10.292402  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:10.296373  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:10.296952  8055 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:10.296993  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:10.297791  8062 raft_consensus.cc:2393] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 19: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:10.300554  8054 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:15:10.303908  8056 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:10.307668  8055 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
W20260709 13:15:10.315923 14048 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.475s	user 0.003s	sys 0.000s
I20260709 13:15:10.250236 13765 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.317095  8137 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:10.317232  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:10.317822  8210 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:10.318699  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:10.319269  8204 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:10.321183  8211 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:15:10.322580  8203 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:15:10.326745  8204 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:10.330119  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:15:10.334208  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 68075.6us
I20260709 13:15:10.330292  8728 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:10.344027  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:10.349892  8029 leader_election.cc:400] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15.
I20260709 13:15:10.350369  8029 leader_election.cc:403] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:15:10.352813  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:10.353646  8055 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:15:10.376979  8132 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:15:10.319237 13765 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:10.349330 13198 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:10.378845 13413 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:15:10.373273 13660 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:10.373389 13757 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 14. Current term is 15.
I20260709 13:15:10.408773  8101 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:10.409775  8101 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:10.410120 14042 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:10.410495 13765 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:10.410547 14042 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:10.411176 14042 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:10.413806  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 78816us
I20260709 13:15:10.416078  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:10.418520  8101 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:10.383895 13660 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:10.428364  8102 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:10.446034  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:10.440726  8101 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:10.419314 13198 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:15:10.420337 13809 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:10.450695  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:10.423183 13821 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:10.439438 13660 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:10.449268 13839 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:10.456557  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.441547 13765 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:15:10.448210 14011 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:15:10.480114 13703 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:10.505265  8026 leader_election.cc:400] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 18. Current term is 19.
W20260709 13:15:10.510805  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 388703us
I20260709 13:15:10.456359 13988 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:10.516337 13988 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:10.517392 13988 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.524727  8203 raft_consensus.cc:2955] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 3 to 11, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:10.527052 14042 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.529089 14042 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:10.529547  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:10.529913 14017 ts_tablet_manager.cc:1929] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 9.7
I20260709 13:15:10.530421 14017 log.cc:1199] T b186be823643475982e269b73d886a42 P d9cd2999aa314b39997b423ec1ae509e: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/wals/b186be823643475982e269b73d886a42
I20260709 13:15:10.531204  8206 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:10.531735 14017 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:48166 (request call id 0) took 1677 ms. Trace:
I20260709 13:15:10.532646 13750 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:10.532092 14017 rpcz_store.cc:276] 0709 13:15:08.854032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.859042 (+  5010us) service_pool.cc:224] Handling call
0709 13:15:09.162910 (+303868us) ts_tablet_manager.cc:1151] Deleting tablet b186be823643475982e269b73d886a42
0709 13:15:09.189446 (+ 26536us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:15:10.330841 (+1141395us) tablet_metadata.cc:665] Metadata flushed
0709 13:15:10.529780 (+198939us) tablet_metadata.cc:665] Metadata flushed
0709 13:15:10.531621 (+  1841us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":522413,"tablet-delete.queue_time_us":147937,"thread_start_us":26365,"threads_started":1}
I20260709 13:15:10.533051 13750 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:10.533427 13750 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:10.536703  7948 catalog_manager.cc:5028] TS d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): tablet b186be823643475982e269b73d886a42 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
W20260709 13:15:10.518222  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 211350us
I20260709 13:15:10.510857  8026 leader_election.cc:403] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:10.543035 13753 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:10.545730  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:10.545845  8026 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:10.547802  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 102315us
I20260709 13:15:10.548779  8102 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:10.548761  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:10.550679  8102 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:10.555778  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 102275us
I20260709 13:15:10.557288  8026 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:10.557765  8101 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:10.558627  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:10.559082  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:10.559432  8208 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:10.561403 13982 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:10.561441 13938 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:10.556203 13751 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:10.562816 13938 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:10.565135  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:10.567262  8054 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:15:10.569830  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:10.576802  8132 raft_consensus.cc:2410] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:15:10.578069  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:10.579644  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:10.580268  8061 raft_consensus.cc:2393] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 16: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:10.582135 13407 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:10.586745  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:10.588953  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:10.586741  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:10.553243 13815 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 18. Current term is 19.
I20260709 13:15:10.573038 13938 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:10.602010 13938 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.605629 13938 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:10.615525 13198 raft_consensus.cc:993] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: : Instructing follower 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:10.616101 13198 raft_consensus.cc:1081] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Signalling peer 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:10.618269  8130 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:46944
I20260709 13:15:10.618919  8130 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:10.619347  8130 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:10.622596  8026 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:10.626901  8026 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:10.627882  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:10.576026 13970 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:10.627009  8208 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:10.578639 13932 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:10.634878  8209 raft_consensus.cc:2368] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16.
I20260709 13:15:10.547278 14042 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:15:10.578845 13809 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:10.656880  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:10.553545 13398 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:15:10.660137  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:10.586426 13982 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:10.563068 13751 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:15:10.619555 13991 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:10.631140 13923 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:10.646286 13933 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:10.675855  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:10.667613  8136 raft_consensus.cc:2955] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 3 to 11, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:10.648576 13932 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:10.593565 13980 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:15:10.656718 13809 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:10.668682 13678 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:15:10.695472  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 55640.4us
I20260709 13:15:10.692593  8104 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:10.647089 13703 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:15:10.676875 13924 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:10.645090 13660 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:10.742177  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:10.671171 13982 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.695710 13988 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.686385 13991 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:10.692960 13932 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:10.691362 13940 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:10.697211 13809 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:10.751689 14046 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:15:10.756697  8029 leader_election.cc:400] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16.
I20260709 13:15:10.764761  8029 leader_election.cc:403] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:15:10.758636  8056 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 20. Current term is 21. Ops: []
W20260709 13:15:10.755244  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 54672.8us
I20260709 13:15:10.698920 14052 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:15:10.770632  8130 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:10.747587 13678 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:10.708293 13753 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:15:10.756906 13461 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:10.774650  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:10.757745 13747 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.782053  8130 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:10.709084 13977 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:15:10.759697 13924 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:10.794268  8061 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:10.771417 13933 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16.
I20260709 13:15:10.819092  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:10.819669  8055 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:10.821354  8060 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:15:10.823079  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:10.823778  8728 raft_consensus.cc:3055] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:10.824184  8728 raft_consensus.cc:740] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:10.760386 13988 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.786418 13398 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:10.780436 13753 consensus_queue.cc:1243] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:10.834510 13398 raft_consensus.cc:697] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:10.851955  8055 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:10.816577 13747 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.833452 13780 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:10.857002  8729 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:15:10.855088  8060 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:15:10.860970 13977 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.861543 13977 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:10.862442 14076 log.cc:927] Time spent T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:15:10.863914 13977 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:10.864621 13924 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:10.866549 14078 log.cc:927] Time spent T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:15:10.932663  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 267803us
I20260709 13:15:10.834010 13991 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.7, Last appended by leader: 7, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.952179  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:10.868698 13780 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.949558  8026 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:10.936666 13461 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.973483  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:10.976542  8135 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:10.978921  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:10.983214  8137 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:15:10.985330  8026 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:10.985677  8057 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:15:10.987385  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:10.988080  8060 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:15:10.969460 13983 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.991487  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:10.910655 13678 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.984302 13800 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:10.986975 13933 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:10.976346 14071 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:10.988817 13982 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:10.992139 13660 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:10.999943  8132 raft_consensus.cc:1217] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Deduplicated request from leader. Original: 9.6->[10.7-16.10]   Dedup: 12.8->[13.9-16.10]
I20260709 13:15:11.102536  8132 pending_rounds.cc:85] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 8
I20260709 13:15:11.102783  8132 pending_rounds.cc:107] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:15:11.102989  8132 raft_consensus.cc:2889] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:10.982517 13461 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:10.992058 13757 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:10.956591 13398 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:11.067931 14074 log.cc:927] Time spent T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
I20260709 13:15:10.993880 13923 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 13:15:11.006096 13983 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:11.115437 13750 consensus_meta.cc:287] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.582s	user 0.011s	sys 0.010s
W20260709 13:15:11.116163 14075 log.cc:927] Time spent T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.214s	user 0.001s	sys 0.001s
W20260709 13:15:11.162778  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.176213  8208 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:15:11.167725 13988 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:11.138576 14071 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:11.157518 14092 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:11.182193 14092 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:11.152694 13747 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:11.163399 14095 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:15:11.245069 14101 log.cc:927] Time spent T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:15:11.231148 13780 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:11.241966 14099 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:15:11.167768 13750 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:11.247859 13750 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:11.248584  8212 raft_consensus.cc:2368] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17.
I20260709 13:15:11.243988 13703 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.249944 13988 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.250109 13703 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:11.250469 13988 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:11.252301 13703 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:11.252942 13988 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:11.271155  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:11.271878  8205 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:11.273342  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:11.284272  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:11.384554  8204 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:11.389366  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:11.398254  8207 raft_consensus.cc:2368] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16.
I20260709 13:15:11.409221  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:11.413439  8211 raft_consensus.cc:1275] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:15:11.428599  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 437264us
I20260709 13:15:11.272145 13703 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.145s
I20260709 13:15:11.281682 13198 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.137s
I20260709 13:15:11.281523 13865 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.140s
I20260709 13:15:11.541371  8026 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:11.408879 13750 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:15:11.588032  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 46767.3us
W20260709 13:15:11.604806  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 58772.8us
I20260709 13:15:11.540953 13923 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:15:11.596787 13750 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:11.634416 13413 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:11.634505 14094 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:11.635187 14094 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:11.637833 14094 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:11.542019 14011 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:15:11.551046 13983 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:11.640760 14071 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:11.641425 13198 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:11.641848 13198 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:11.642943 14089 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:11.643644 14089 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:11.650504 13932 consensus_meta.cc:287] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.881s	user 0.009s	sys 0.004s
I20260709 13:15:11.650960 13932 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:11.651950  8063 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:11.654713  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 253748us
W20260709 13:15:11.664417  8137 consensus_meta.cc:287] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.000s
I20260709 13:15:11.664837  8137 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:11.631440  8104 leader_election.cc:400] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16.
I20260709 13:15:11.679803  8104 leader_election.cc:403] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:11.696043  8205 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
W20260709 13:15:11.733584  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 105541us
W20260709 13:15:11.734890  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:11.746786  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69843us
I20260709 13:15:11.553246 14092 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:11.736752  8104 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:11.753988  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:11.754523  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:11.754680  8205 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:11.756762  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:11.758734 14103 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:11.759223 14103 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:11.760025 14103 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:11.676074 13934 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.762185  8063 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:11.762426 13934 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:11.762905 14072 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.107s
I20260709 13:15:11.763463  8063 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 3344) took 1144 ms. Trace:
I20260709 13:15:11.763827  8063 rpcz_store.cc:276] 0709 13:15:10.618619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.619316 (+   697us) service_pool.cc:224] Handling call
0709 13:15:11.763316 (+1144000us) spinlock_profiling.cc:243] Waited 994 ms on lock 0x7b54001e8cd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:11.763411 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2286625536}
I20260709 13:15:11.764619 13934 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:11.772987 13932 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:11.774428 13970 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:11.774628 13461 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:11.774927 13970 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:11.775918 13815 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:11.783044  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:11.786036  8205 raft_consensus.cc:2368] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:11.806138  8029 leader_election.cc:400] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:11.806599  8029 leader_election.cc:403] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:15:11.816936  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:11.748701 13413 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:15:11.821966  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 419283us
W20260709 13:15:11.828140  8132 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.006s	sys 0.006s
I20260709 13:15:11.776417 13815 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:11.828469  8132 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.006s	sys 0.006s
I20260709 13:15:11.827145  8104 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:11.804687 13932 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:11.834740  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:11.749835 13398 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:15:11.825593 13703 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:11.825415 13780 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:11.826087 14080 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:11.824049 13793 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:11.851995  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:11.836499 13751 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.857106  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59868 (request call id 3506) took 1308 ms. Trace:
I20260709 13:15:11.860214  8212 rpcz_store.cc:276] 0709 13:15:10.541691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.542439 (+   748us) service_pool.cc:224] Handling call
0709 13:15:11.850068 (+1307629us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b5400189554. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:11.850226 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1583505792}
I20260709 13:15:11.840901 14071 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:11.855523  8206 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:11.910774  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:11.864185 14102 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16.
I20260709 13:15:11.865144 14110 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:11.876376  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:11.878854 13980 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:11.879971 13924 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:11.890861 14089 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:11.891402 13970 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:11.896718 13750 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:11.774281 13198 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:11.898772 13815 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:11.899638 14114 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:11.900167 14080 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:11.829655 13413 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:11.858256  8728 consensus_queue.cc:260] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:11.916298  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:11.916868 14092 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.842247 13932 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:11.927906  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:11.855477 13780 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:11.859408 13703 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:11.930688 14084 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:11.906162 14071 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:11.926656  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:11.939579  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:11.951853  8061 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:11.941812  8055 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:11.929365  8063 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:11.945264  8059 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:11.908541 13751 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:11.918203 14110 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:11.965687  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:11.964869  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:11.944571 13980 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:11.972168  8059 raft_consensus.cc:2368] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16.
I20260709 13:15:11.951123 14102 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:11.953945 14071 raft_consensus.cc:3060] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:11.958983 13198 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:11.979732  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:11.981817  8055 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:11.943605 14092 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:11.986496  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:11.986728  8061 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:11.989933  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:11.974273 13980 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:11.990659  8061 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 18: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:15:12.001964  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.943732 14114 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:11.941640 14115 consensus_queue.cc:1175] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d9cd2999aa314b39997b423ec1ae509e" responder_term: 20 status { last_received { term: 20 index: 16 } last_committed_idx: 16 last_received_current_leader { term: 20 index: 16 } }
I20260709 13:15:12.010949  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:12.011936 14085 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:12.013595 13703 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:12.015810 13932 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.020913  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.049652  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:12.050156  8062 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:12.007653 13980 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:12.030269 13413 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:12.039827  8728 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:11.991261 14073 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:11.990502 13924 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:12.052492  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 168665us
W20260709 13:15:12.056277  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 98781.4us
I20260709 13:15:12.066874  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:12.073925  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:11.980664 14102 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:12.058837 13809 consensus_meta.cc:287] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 1.286s	user 0.018s	sys 0.009s
I20260709 13:15:12.073945  8721 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:12.114044 14110 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:12.115796  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:12.056111 14070 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:15:12.130741  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 76470.1us
I20260709 13:15:12.117054 14125 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:12.049175 14085 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.136072  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:12.123936 13747 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:12.079834 13809 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.123034 14082 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:15:12.155869  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 77119.6us
I20260709 13:15:12.126995 14110 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.158937  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:12.081939 13924 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.123659 13678 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:15:12.154560 14117 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:15:12.079303 14073 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:12.134574 14128 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:12.171084 13780 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:12.223079  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 107950us
I20260709 13:15:12.121548 14070 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:12.194437 14110 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:12.231418 14118 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:15:12.210381 13461 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:12.244163  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 47310.6us
I20260709 13:15:12.245266  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:12.174625 13747 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.236580 13983 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:12.197912 13678 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:12.237099 14092 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.253909  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:15:12.258517 14116 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:15:12.258848 14126 consensus_queue.cc:1048] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:15:12.259492 14109 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:15:12.260759 14119 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 13:15:12.261528 14123 log.cc:927] Time spent T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:15:12.263841 13678 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:12.173065 14125 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:12.193869 14082 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:12.298861  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.299554  8130 raft_consensus.cc:2368] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:15:12.268203 14140 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.246096 13461 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.333520  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 1802) took 1346 ms. Trace:
I20260709 13:15:12.339921  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.338804  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:12.343859  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.344092  8134 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:12.306704 14092 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:12.335958  8132 rpcz_store.cc:276] 0709 13:15:10.987436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.987602 (+   166us) service_pool.cc:224] Handling call
0709 13:15:10.987915 (+   313us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:15:11.127481 (+139566us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:11.127546 (+    65us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:11.140211 (+ 12665us) log.cc:844] Serialized 52 byte log entry
0709 13:15:11.838140 (+697929us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:11.838181 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:11.838246 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:12.333261 (+495015us) raft_consensus.cc:1685] finished
0709 13:15:12.333287 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:12.333455 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71742,"raft.queue_time_us":40307,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":17680,"thread_start_us":581982,"threads_started":1,"wal-append.queue_time_us":759675}
W20260709 13:15:12.297708 14120 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:15:12.294328 13983 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.344564  8129 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
W20260709 13:15:12.347903  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 174353us
I20260709 13:15:12.335059 14114 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.309216 13678 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:15:12.356840  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 111729us
I20260709 13:15:12.360093  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.360395  8101 leader_election.cc:400] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16.
I20260709 13:15:12.361501  8101 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:12.298156 14109 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.364531 13924 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:12.365104 13924 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:12.365569 13924 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:12.366096 14109 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.366633 14127 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:15:12.367858 14127 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.368263 14127 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.368479 14069 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:12.374378  8728 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:12.374382  8102 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:12.384352  8203 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3512) took 1114 ms. Trace:
I20260709 13:15:12.384742  8203 rpcz_store.cc:276] 0709 13:15:11.270460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.270568 (+   108us) service_pool.cc:224] Handling call
0709 13:15:11.270821 (+   253us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:11.282534 (+ 11713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:11.282560 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:11.401241 (+118681us) log.cc:844] Serialized 26 byte log entry
0709 13:15:11.977771 (+576530us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:11.977816 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:11.977885 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:12.384141 (+406256us) raft_consensus.cc:1685] finished
0709 13:15:12.384167 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:12.384294 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7511,"raft.queue_time_us":104036,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":12508,"thread_start_us":426995,"threads_started":1,"wal-append.queue_time_us":465310}
I20260709 13:15:12.387090  8728 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 1049) took 1552 ms. Trace:
I20260709 13:15:12.387476  8728 rpcz_store.cc:276] 0709 13:15:10.822659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.822800 (+   141us) service_pool.cc:224] Handling call
0709 13:15:11.847305 (+1024505us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b4c000c3c80. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db92e2a20 00007f6db92f3a29 00007f6db92b9644 00007f6db92e0f2d 00007f6db92e0b93 00007f6dbb01118b 00007f6dbb011cb5 00007f6dbb00cf30 00007f6dbb00e7de 00007f6dbb01a27c 00007f6dbffdbb17
0709 13:15:12.039426 (+192121us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b58008aced0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf720aa 00007f6dbaff4447 00007f6dbb00cfe7 00007f6dbb00e7de 00007f6dbb01a27c 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c
0709 13:15:12.375084 (+335658us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242200064}
I20260709 13:15:12.388314  8101 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:12.354372 14124 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:12.390669  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:12.390704  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:12.390918 14145 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:12.358922 13703 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:12.391229  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:12.391714  8209 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:48152
I20260709 13:15:12.424577  8209 raft_consensus.cc:606] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:12.425153  8209 raft_consensus.cc:614] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:12.398218 13747 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.398842 13809 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.401886  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.402060  8102 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:12.438040  8175 leader_election.cc:400] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:15:12.363211 14114 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:12.435258 13703 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:12.362251 14102 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:12.400815 13983 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:12.483176  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45341.9us
I20260709 13:15:12.336627 14140 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:12.495314  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 93325.9us
I20260709 13:15:12.370882 14092 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.369630 13678 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.505486  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.387231 14069 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:12.481712 14085 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.491918 13703 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:12.507966  8129 raft_consensus.cc:1240] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:12.501147 13809 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:12.500782 14102 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:12.523931 13809 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:12.464978 14145 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:12.465324 14124 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.466367 13924 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.529642  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 2008) took 1727 ms. Trace:
I20260709 13:15:12.523989 14151 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:12.540179 14158 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:15:12.532760  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3511) took 1262 ms. Trace:
I20260709 13:15:12.541232 14135 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:12.504774 14073 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:12.542763  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 154527us
I20260709 13:15:12.577971  8101 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:12.499552 14110 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.557610  8132 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:12.575337  8208 rpcz_store.cc:276] 0709 13:15:11.270057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.270188 (+   131us) service_pool.cc:224] Handling call
0709 13:15:12.004650 (+734462us) spinlock_profiling.cc:243] Waited 4.31 ms on lock 0x7b5800168398. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:15:12.004834 (+   184us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:12.008928 (+  4094us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:12.008958 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:12.010872 (+  1914us) log.cc:844] Serialized 26 byte log entry
0709 13:15:12.133293 (+122421us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:12.133339 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:12.133400 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:12.532423 (+399023us) raft_consensus.cc:1685] finished
0709 13:15:12.532451 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:12.532693 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8,"raft.queue_time_us":105981,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":6225,"spinlock_wait_cycles":9924480,"thread_start_us":122079,"threads_started":1,"wal-append.queue_time_us":345600}
I20260709 13:15:12.494422 14114 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:12.587157  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.587807 14114 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.588439  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.589332 14110 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:12.589496  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.589834 14110 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.590713  8726 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:12.591953 14124 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.556557 13809 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:12.592391 14092 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.560555 13934 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:12.593091 13934 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.593422 14092 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:12.595028 13934 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.595247 14125 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.596544 14158 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:12.597065 14158 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.598156  8132 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:12.519042 14070 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:12.562182  8058 rpcz_store.cc:276] 0709 13:15:10.797854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.798011 (+   157us) service_pool.cc:224] Handling call
0709 13:15:10.819943 (+ 21932us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:10.898500 (+ 78557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:10.898530 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:10.899069 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:15:10.984648 (+ 85579us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:10.984684 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:12.519537 (+1534853us) spinlock_profiling.cc:243] Waited 53.6 ms on lock 0x7b54001e91d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:12.519606 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:12.519671 (+    65us) raft_consensus.cc:1685] finished
0709 13:15:12.519691 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:12.524768 (+  5077us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":796,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":27165,"spinlock_wait_cycles":123278592,"thread_start_us":24779,"threads_started":1,"wal-append.queue_time_us":98245,"wal-append.run_cpu_time_us":2333,"wal-append.run_wall_time_us":1177128}
I20260709 13:15:12.599809  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:12.600313  8058 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:12.600644 13198 consensus_meta.cc:287] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.001s
I20260709 13:15:12.599830 14069 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:12.579515 14135 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.579962 13839 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:12.603363 13839 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:12.605165  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.606499 14124 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:12.605435 13839 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:12.607074 14124 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.561602 14126 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:12.582265 14145 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:12.556377 14089 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.582705 14151 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:12.584537  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:12.585553 14153 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:15:12.586011 14082 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:12.591980 14110 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.560127  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.639716  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:12.639930 13809 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.640332  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.640496 14151 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:12.641451  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.641932 13809 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.609141  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:12.609385  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.611048  8058 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:12.663034  8138 raft_consensus.cc:2393] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 16: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
W20260709 13:15:12.626482  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 368557us
I20260709 13:15:12.631125 14155 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:12.632216  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.633643 14140 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:12.634781 13678 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.635526  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.668659  8063 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:12.635877 13737 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:12.637589 14089 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:12.608498  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.642580  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.642719 14158 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:12.645164 14071 consensus_meta.cc:287] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.656s	user 0.009s	sys 0.000s
I20260709 13:15:12.651548  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.681288  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.681902  8058 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:12.682083  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:12.574265 14102 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.683538 14102 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.685611 14102 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.687395 14089 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:12.651037  8176 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:12.656378  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.688469  8055 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:12.657629  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.613507 14126 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:12.650650 13461 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.632216 14127 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:12.656489 13980 consensus_meta.cc:287] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.575s	user 0.006s	sys 0.003s
I20260709 13:15:12.666031  8133 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:12.667369  8730 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 21.
I20260709 13:15:12.673336  8129 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:12.675839 13934 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.685796  8134 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:12.704676  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:12.690436  8101 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:12.696336 14071 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:12.709041  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 90053.3us
I20260709 13:15:12.709059  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.711138  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:12.712541  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:12.701706 14126 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:12.701769 14089 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:12.720805  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:12.725021  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.727187 14162 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:12.710948 13934 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.725258 14177 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:12.729234 14082 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.725623  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.731622  8056 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:12.732684 14162 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.736935 14080 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:12.737051 14162 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.708439 13980 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.732815 14177 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:15:12.751315  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 60966.2us
I20260709 13:15:12.751603 14169 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:12.752139 14169 raft_consensus.cc:697] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:12.753000 14169 consensus_queue.cc:237] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.755792  8026 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:12.733134 14135 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.748654 14177 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:12.740567 14082 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:12.775345  8104 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.67:35745 timed out after 1.850s (SENT) [suppressed 4 similar messages]
I20260709 13:15:12.788152  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.767890 14071 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.779896 13980 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:12.784507 14167 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:12.789690  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.791574  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.792454  8101 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:12.795953  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.800585  8133 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:12.809475  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.810809  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.813059 13934 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.823297  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:12.827097  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:12.833250 14133 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.478s	user 0.000s	sys 0.003s
I20260709 13:15:12.853240 13703 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.854373  8131 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:12.824846 14071 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:12.855261 14071 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:12.855645 14071 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:12.859814  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:12.860915  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:12.861617  8054 raft_consensus.cc:2393] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 18: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:12.861541  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:12.862232  8212 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:12.863196  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.864674  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.865797  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.798666 14167 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:12.867094 14167 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:12.874955  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.876154  8058 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
I20260709 13:15:12.794603 14135 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:12.875487  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:12.878075  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.882973  8136 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:12.875550  8054 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:12.884397  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.880461  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:12.892094  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:12.892349  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:12.892690  8212 raft_consensus.cc:1813] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Handling vote request from an unknown peer 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:12.824191 14124 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:12.893420  8212 raft_consensus.cc:2410] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:15:12.895870  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.887313 13413 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:12.911847  8101 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:12.896458 14164 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:12.897322 13934 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:12.909936  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:12.916766  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:12.909838  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:12.921540  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:12.915581 13413 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.893386 14135 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:12.920930 13934 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:12.913012 14127 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:12.934155  8056 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:12.917883 14164 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:12.919821 14124 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:12.939247  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:12.964108 14124 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:12.964306  8210 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:12.964507 14124 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:12.942222 14092 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:12.959205 14128 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:12.946230  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:12.959621  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 184299us
I20260709 13:15:12.954329  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:12.968600 13703 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:13.029718  8211 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 13:15:13.030481  8211 pending_rounds.cc:85] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 7
I20260709 13:15:13.030762  8211 pending_rounds.cc:107] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 13:15:12.966696 14164 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:12.959715 14127 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:12.916980  8101 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:15:13.033648  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 116735us
I20260709 13:15:12.959021  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:12.963802  8138 raft_consensus.cc:1813] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Handling vote request from an unknown peer 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:12.963347 14135 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.034700  8138 raft_consensus.cc:2410] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:15:12.950812 14167 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.031049  8211 raft_consensus.cc:2889] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:13.034767  8101 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:13.042037  8060 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:15:13.055356  8728 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:50008
I20260709 13:15:13.055899  8728 raft_consensus.cc:606] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:13.056233  8728 raft_consensus.cc:614] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:13.056922 14042 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:13.057406 14042 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.057824 13945 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:13.058379 13945 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.059902 14042 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.061821 14126 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.063987  8054 raft_consensus.cc:2368] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:15:13.064144 13413 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:13.055051 14093 raft_consensus.cc:2804] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:13.047911 13703 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:13.065325 13703 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:13.065686 13703 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:13.065984 14127 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:13.066219 14169 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:13.066758 14169 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.068823 14169 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.069518 14093 raft_consensus.cc:697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:13.070401 14093 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.070976  8130 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:13.075785 13750 raft_consensus.cc:3055] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:13.076165 13750 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:13.076905 13750 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.078032 13750 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:13.065593  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:13.082031  8060 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:13.063954 14126 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.099408  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:13.086325 14144 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.003s	sys 0.034s
I20260709 13:15:13.109580  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.109709  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:13.110114  8211 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:13.119031  8728 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:50008
I20260709 13:15:13.119771  8728 raft_consensus.cc:606] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 18 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:13.089036 14135 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:15:13.107426 14084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.137s	user 0.008s	sys 0.029s
W20260709 13:15:13.108172 14144 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.004s	sys 0.034s
W20260709 13:15:13.127502 14084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.155s	user 0.009s	sys 0.029s
I20260709 13:15:13.138723  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:13.141752  8137 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1797) took 2171 ms (client timeout 2716 ms). Trace:
I20260709 13:15:13.142561 14177 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.146629  8728 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a003b4584d3747a381522d67299aaed2"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:50008
I20260709 13:15:13.147364  8728 raft_consensus.cc:606] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:13.147759  8728 raft_consensus.cc:614] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:13.149809  8208 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:15:13.142280  8137 rpcz_store.cc:269] 0709 13:15:10.970450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.978691 (+  8241us) service_pool.cc:224] Handling call
0709 13:15:13.141523 (+2162832us) spinlock_profiling.cc:243] Waited 3.69 ms on lock 0x7b5400409cd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:13.141653 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8489216}
I20260709 13:15:13.127197 14135 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.138114 14084 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:13.161301 14177 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:13.166913  8728 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:50008
I20260709 13:15:13.169050  8728 raft_consensus.cc:606] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:13.169360  8728 raft_consensus.cc:614] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:13.164951 13983 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:13.163597 13991 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:13.164558 13934 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.183276  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:13.173146 14167 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:13.191604 14135 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.173555 14071 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.195238 14071 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:13.199710 14188 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:15:13.208524  8728 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:50008
I20260709 13:15:13.173777 13983 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.180521 13991 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.209121  8728 raft_consensus.cc:606] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:13.216933 13751 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:13.218626  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:13.220489  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.221547  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.226758  8062 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:13.228317 14124 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.232766 13703 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.237043 13983 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.239574 14082 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:13.240319 13945 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:13.243754 13991 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.262840  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:15:13.263113  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.263660  8724 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:13.190393 14084 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.278674  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.278133  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.280915  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.298745  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:13.301424  8132 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:13.304159  8176 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:13.306126  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.307315  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:13.308703  8175 leader_election.cc:400] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:15:13.309149  8175 leader_election.cc:403] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:15:13.313907  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.304013 14091 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:13.306149 13991 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:13.329432  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.330045  8133 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:13.304512 14124 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:13.324141 14195 log.cc:927] Time spent T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:15:13.321194 14091 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:13.336977 14091 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:13.319871 13750 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:13.348452 13940 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:13.348801 14177 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:13.349016 13940 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.349320 14177 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.356194  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.357719  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.361512 14127 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:13.362192  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.362717  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.362963  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.363265  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.363511  8211 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:13.357790 13737 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:13.366518  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.370451  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.372782  8130 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:13.371727  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.342257 13703 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:13.351251 13940 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.351249 14177 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.335320 14164 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.379565  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.387501  8102 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:15:13.390079  8693 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.353302 13991 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:13.353323 13753 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:15:13.388621 14192 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 13:15:13.362082 14127 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.397799  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55848 (request call id 1047) took 1149 ms. Trace:
I20260709 13:15:13.395147  8130 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:13.401726  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.402473  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.400446 14163 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:13.412904  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.408499  8726 rpcz_store.cc:276] 0709 13:15:12.248850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:12.249027 (+   177us) service_pool.cc:224] Handling call
0709 13:15:13.397634 (+1148607us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54007611d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:13.397747 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383006464}
I20260709 13:15:13.418722  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.420677  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.420837  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.421468  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.421887  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.434279  8204 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:13.378522 13737 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.436637  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:13.438409  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.451234  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.483506  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.484166  8056 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:13.484922  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:13.485394  8056 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:13.493005  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.494498  8726 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:15:13.497597  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.408185 13753 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.502810  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.504004 13940 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:13.504642  8027 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:13.504515 13940 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.505380 14167 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:13.505831 14167 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:13.506414  8027 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:13.506542 13940 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.507042 13940 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:13.507467 13940 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.507869 14190 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:13.508410 14190 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.509423 13940 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.510135 13934 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:13.510666 13934 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:13.511147 13934 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:13.512142 13793 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.512586 13793 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.513270 13985 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:13.513772 13985 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.514712  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.515616 13985 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:13.512821  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.515857  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.516538  8721 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 21: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:13.518129  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.518420  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.519271  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.519788  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:13.520504  8060 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:13.523783 13793 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:13.535830 14164 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:13.537458 14127 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:13.535852  8057 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:15:13.546659  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.547719  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:13.548627  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.549391  8138 raft_consensus.cc:2410] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 19 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 7.
I20260709 13:15:13.551460  8131 raft_consensus.cc:2393] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 16: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:13.552158  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.400952 13991 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.554090  8060 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:13.557790  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.554773  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.559720  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:13.543066 14197 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:15:13.564742  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.565320  8133 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:13.568971  8129 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:13.569829  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.570112  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.569319  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:13.551097 14190 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.571945  8026 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:13.553467 14127 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:13.551605 13932 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:13.581243  8026 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:13.583200  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.555039 13198 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:13.587128  8056 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:13.563329 14162 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.650316  8727 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 1040) took 1458 ms. Trace:
W20260709 13:15:13.644186  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 63028.3us
I20260709 13:15:13.648527  8208 raft_consensus.cc:1275] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:15:13.650689  8727 rpcz_store.cc:276] 0709 13:15:12.123898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:12.131253 (+  7355us) service_pool.cc:224] Handling call
0709 13:15:12.157146 (+ 25893us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:12.334780 (+177634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:12.334813 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:12.334852 (+    39us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:12.334893 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:13.582413 (+1247520us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b5400694e54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:13.582457 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:13.582566 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":408083328}
I20260709 13:15:13.652823  8026 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:13.657290 13747 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:13.658015  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.658746 13747 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:13.658989  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:13.659148 13747 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:13.659327  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.687050  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:13.659605  8724 raft_consensus.cc:2376] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18. Re-sending same reply.
I20260709 13:15:13.687630  8208 raft_consensus.cc:3060] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:15:13.660182 13198 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.671376  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:13.669723  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.671119  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.670841  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.577522 13793 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:13.694653  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 125306us
I20260709 13:15:13.694051  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.696408  8175 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:13.644590 14073 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:13.696480  8026 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:13.717984  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.718864  8102 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:13.644910 14082 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:13.725936  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.726627  8208 raft_consensus.cc:2468] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:13.707358  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.721060  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:13.728082  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:13.722325  8060 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:13.724109  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:13.649170 13461 raft_consensus.cc:2764] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:15:13.681003 13737 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:13.659852 14108 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:13.732071  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.732252  8206 raft_consensus.cc:2393] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 17: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:13.730454  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:13.725159 14073 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.734376 13815 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:13.746493 14148 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:13.747774 13828 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:13.748252 13828 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.750085 13828 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:13.731354  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:13.718729 14173 log.cc:927] Time spent T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:15:13.743777  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:13.753990  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:13.754899  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:13.745157 13753 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:13.745936 14073 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:13.750706  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:13.756664  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.761308  8061 raft_consensus.cc:2393] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 16: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:13.649268 14127 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.728091 14082 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:13.751971  8696 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:13.770515  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.770922  8696 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:13.771580  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:13.771960  8696 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:13.752358 13815 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:13.735339 13983 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:13.742691 13780 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:13.655735 13932 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.742888 13793 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:13.723378 14084 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:13.760017  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:13.661185 14162 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:13.762158 14177 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:13.775022 14082 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:13.751726  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.764479  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:13.772571  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:13.773985  8205 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:13.719230 14190 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:15:13.752316  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.775537 13815 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:13.774055 14073 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:13.777309  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.801229  8207 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:13.777807  8104 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:13.787261 13983 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:13.783170  8208 raft_consensus.cc:2468] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:13.784257  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:13.763463 14124 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:15:13.787515 14084 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:15:13.789268  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 282920us
I20260709 13:15:13.726766 13750 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15.
I20260709 13:15:13.790156  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.797742  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.812847  8204 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:15:13.778375 13198 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:13.759531 13924 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:15:13.788060 14143 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:13.801397  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:13.763872 13934 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.803493  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:13.823326  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:13.756183  8102 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:13.803977  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.804319  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 17 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:13.660399 14167 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.806945 14190 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:13.808254 13753 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:13.804875 13780 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:13.796969 14058 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:13.810846 13932 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.791006 14177 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:13.819116  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.819727  8203 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:15:13.817899 13991 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:13.830368  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:13.790820 13793 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.762729 13747 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:13.849269 14149 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:13.811340 13983 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:13.850898  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.851727  8062 raft_consensus.cc:2393] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:13.817109 14084 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:13.856823  8061 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: []
W20260709 13:15:13.825309  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 69197us
I20260709 13:15:13.855659 14163 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:15:13.824864 14093 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:13.833823 14136 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:13.839921  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:13.838709 14190 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:13.851851 14058 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:13.872694  8102 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:13.862905 14177 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:13.879913  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 108009us
W20260709 13:15:13.885320  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 45482.6us
I20260709 13:15:13.890298  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.890990  8102 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:13.891892  8104 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:13.873785 13983 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:13.892973  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:13.894776  8102 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:13.880350 14163 consensus_queue.cc:1243] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer d9cd2999aa314b39997b423ec1ae509e log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:15:13.880573 14091 consensus_meta.cc:287] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.543s	user 0.008s	sys 0.002s
I20260709 13:15:13.895319  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:13.900370  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 139971us
I20260709 13:15:13.895933 14124 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:13.888335 14190 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:13.888973 14102 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:13.909013  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.889873 14058 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:13.921670  8696 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:13.910401  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:13.911499  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.895771 13800 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:13.896499 14149 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:13.897400 14093 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:13.931833  8061 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:13.904579 13747 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:15:14.007073  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 66242.8us
I20260709 13:15:14.010015 13750 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:14.010557 13750 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.012611 13750 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:13.909420 14136 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:13.930441 13780 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:14.032191  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:13.932801 14127 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:13.904958 14091 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.047745 13815 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.054239 13983 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.070761  8211 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:14.037664 14143 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:14.025626 14124 consensus_queue.cc:1048] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 13:15:14.072005  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:14.073499  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:15:14.073580  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 163295us
I20260709 13:15:14.034477 14212 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:14.075317  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:15:14.078290  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 155492us
I20260709 13:15:14.082384 14210 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:14.084138 13983 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:14.084372 13815 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:14.088215  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:14.072729 13934 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:14.091758  8029 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.096383  8029 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.086823  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:14.092394  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:14.097373  8029 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:14.096742  8696 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:14.100840  8029 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.102295  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:14.099916 14145 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:14.101298 13932 raft_consensus.cc:2764] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:15:14.101658 13815 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:14.104707 14205 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:14.110143  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.110981 13934 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:14.112774  8029 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.113592 13747 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:14.115881  8029 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:14.117357 13934 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:14.117791 13934 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:14.118151 13934 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:14.104982 13198 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:14.122746 13198 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:14.123129 13198 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:14.123634 13985 raft_consensus.cc:3055] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:14.124056 13985 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:14.124837 13985 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.127125 14143 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:14.127537 14143 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:14.127897 14143 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:14.126139  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.134503 13793 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:14.135874 13793 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:14.138931  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.128086 13934 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.140797  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:14.141121  8029 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:14.143296  8137 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 19. Current term is 20. Ops: []
W20260709 13:15:14.221187  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 70245.2us
I20260709 13:15:14.222711  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:14.223690  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:14.224532  8210 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:14.222083  8029 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.221851  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:14.225646  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:14.125972 13985 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:14.221284 14209 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.189s	user 0.003s	sys 0.000s
I20260709 13:15:14.225148 13815 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:14.199466 14148 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.198416 14204 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:14.241050  8029 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:14.237944 13815 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:14.242453 13793 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:14.248689 13461 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:15:14.243587 14148 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:14.288409  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 201634us
I20260709 13:15:14.271517 13815 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:15:14.290688  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 49744.5us
W20260709 13:15:14.291159  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 370219us
I20260709 13:15:14.308401  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.274063 13413 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:14.310563  8029 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.311822  8104 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.313194  8104 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.312613 13800 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:14.276437 14091 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:14.318504  8104 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.325995 14084 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.328590 14084 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:14.330744 14217 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:15:14.331661 14148 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:14.331277  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.332126  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:14.307152 14212 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:14.318557 13780 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:14.335107  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:14.317301 14108 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:14.315502 13793 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:14.336460  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:14.333261  8104 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.343118 14145 consensus_queue.cc:1059] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:15:14.354063  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:14.332134 14093 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:14.351941  8104 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:14.364456  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.351279  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:14.351969  8206 raft_consensus.cc:2410] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 19 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 7.
I20260709 13:15:14.332279 13800 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:14.365635  8104 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:14.366129 13800 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:14.366211 13780 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:14.366566 13800 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:14.367036 13800 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:14.358196 14190 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.369621 14190 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:14.370296  8203 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:14.359838 14058 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.372334  8206 pending_rounds.cc:85] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 8
I20260709 13:15:14.372630  8206 pending_rounds.cc:107] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:15:14.372725 14058 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:14.372888  8206 raft_consensus.cc:2889] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:14.374315 14058 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:14.375259 14058 raft_consensus.cc:993] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
W20260709 13:15:14.360574 14082 consensus_meta.cc:287] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.558s	user 0.007s	sys 0.000s
I20260709 13:15:14.376322 14082 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.344381 14163 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:14.367048 13780 raft_consensus.cc:2804] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:14.379909 14145 raft_consensus.cc:3055] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:14.380013 13780 raft_consensus.cc:697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:14.380334 14145 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:14.380949 13780 consensus_queue.cc:237] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.362761 13198 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.381084 14145 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.411540 13198 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:14.375828 14058 raft_consensus.cc:1081] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:14.415124 13815 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.416229  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:14.315052 13413 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:14.416970 13413 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:14.417430  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:14.418467  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:14.419082 14093 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:14.419605 14093 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.420053  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:14.420825  8135 raft_consensus.cc:2410] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:15:14.421522  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:14.421679 14093 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:14.386087 13800 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.422096 13821 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:14.423842 13800 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:14.425040  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:14.425397  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:14.425917  8130 raft_consensus.cc:2376] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16. Re-sending same reply.
I20260709 13:15:14.426172  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:14.426774  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:14.427175  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:14.427660  8135 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:14.389093 14143 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.391309  8208 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:15:14.355738 13703 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:14.404472  8212 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55942 (request call id 1069) took 1046 ms. Trace:
I20260709 13:15:14.404911  8210 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:15:14.408941 14177 consensus_meta.cc:287] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.511s	user 0.006s	sys 0.000s
I20260709 13:15:14.430281  8130 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:14.436941  8135 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:14.480592 14222 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:14.483703 13934 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:14.430965  8138 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:14.431629  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:14.486189  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:14.431308  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:14.470422  8212 rpcz_store.cc:276] 0709 13:15:13.358034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.365125 (+  7091us) service_pool.cc:224] Handling call
0709 13:15:13.365390 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:13.571500 (+206110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:13.571530 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:13.571578 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:13.571611 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:14.404252 (+832641us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b540014a354. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:14.404300 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:14.404420 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473596672}
I20260709 13:15:14.473737  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:14.488464  8210 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:14.475986 14143 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:14.433403  8132 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:14.474165 14093 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:14.478104  8101 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.485411 13991 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:14.501196  8054 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:14.504846 14222 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:14.505542 14222 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:14.506470 14222 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.7, Last appended by leader: 7, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.507321 14145 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:14.497840  8101 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:14.504979  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:14.499248  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:14.514190  8175 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:14.515501  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:14.518676  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.522053  8175 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:14.528157  8104 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:14.519991 14154 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:14.519713 13821 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:14.546463  8210 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
W20260709 13:15:14.529752  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 259611us
I20260709 13:15:14.478585 14177 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.503867 13985 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:14.558284  8104 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:14.525802 14084 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:14.552047 14127 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:14.559851 13991 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:14.576128  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:14.576900  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:14.557040 14093 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:14.557245 13747 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:14.582186 13747 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.583971 14177 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:14.584746 13747 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:14.585075 14136 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:14.585613 14136 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.586820  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:14.587288  8210 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:14.587595 14136 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:14.588244 14136 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:14.588676 14136 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:14.564601 14143 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.589212 14227 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:14.590281  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:14.623618  8210 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:14.592293 14231 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.624869  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:14.597802  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:14.625789  8208 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:14.600251  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 949) took 1205 ms. Trace:
I20260709 13:15:14.626410  8062 rpcz_store.cc:276] 0709 13:15:13.394997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.502359 (+107362us) service_pool.cc:224] Handling call
0709 13:15:14.600055 (+1097696us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b540009c254. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:14.600194 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":668858624}
I20260709 13:15:14.601234  8055 raft_consensus.cc:1240] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:14.558851 13703 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:14.607127 14162 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:14.606107  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:14.614885  8727 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:14.616271 14011 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:15:14.620473  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:14.607141 13413 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.622232  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:14.623756  8059 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:14.595186  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:14.630054  8063 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:14.638279  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:14.644047  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:15:14.647003  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 314941us
I20260709 13:15:14.643360  8029 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.655884  8210 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:14.649097  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:14.650027  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:14.657711  8209 raft_consensus.cc:2410] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:15:14.659509  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:14.661728  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:14.655164  8026 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.664878  8029 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.670404  8208 raft_consensus.cc:2468] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:15:14.663216  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:14.663579  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:14.672574  8062 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:14.674079  8061 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:37156
I20260709 13:15:14.674845  8061 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:14.675163  8061 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:14.677278  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:14.677551  8207 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:14.628724 13703 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:14.678686  8207 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:14.681452  8208 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:15:14.684510  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.684902  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:14.687889  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:14.688405  8207 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:14.687747  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:14.691468  8725 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:14.693375  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:14.648782 14011 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:14.693706  8696 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:14.628805 14143 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.699750  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:14.671854 14162 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:14.692783  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:14.676605 14167 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:14.700354  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:14.686596 13703 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:14.689426 13747 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:14.636950 14231 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:14.714134  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 192234us
I20260709 13:15:14.689265 13198 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:14.701220 14212 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:14.721815  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:14.722404  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:14.697137 14086 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:15:14.713114 13780 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:14.728749  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:14.734686  8063 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:14.709679 14141 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:14.717316 14167 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:15:14.739162  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 180955us
I20260709 13:15:14.740422  8061 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.705590 14011 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.754273  8208 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:14.755401  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:14.759141  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:14.759830  8061 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:14.761783  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:14.729389 13750 consensus_queue.cc:1059] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:15:14.727490 14234 raft_consensus.cc:2804] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:14.766776 13940 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:14.767190 13940 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:14.768046 13940 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.769223 13940 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:14.770869 13413 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:14.720736 13198 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.775797 13198 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:14.759824 14148 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:14.771636 14011 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:14.763218 14177 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:14.732205 14212 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:14.737509 13780 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:14.785593 13821 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:14.785854 13780 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:14.786726 13821 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.763484 13985 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:15:14.790145 14231 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:14.792263 14143 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:14.793990  8054 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:14.712049 14136 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:14.785054  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 96772.5us
I20260709 13:15:14.799160 14242 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:14.783054 14234 raft_consensus.cc:697] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:14.799191 13991 consensus_queue.cc:1059] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:14.799839 14127 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:14.783936  8175 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:14.805469 13703 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.808930  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 2034) took 1086 ms. Trace:
I20260709 13:15:14.809273  8055 rpcz_store.cc:276] 0709 13:15:13.722326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.722474 (+   148us) service_pool.cc:224] Handling call
0709 13:15:14.601044 (+878570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:14.631155 (+ 30111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:14.808755 (+177600us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b540009c250. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:14.808879 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2019957120}
W20260709 13:15:14.813899  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 111627us
I20260709 13:15:14.815135 14227 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:14.800345 14127 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.817612 14127 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:14.817879  8027 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.817811  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:14.819355  8725 raft_consensus.cc:3055] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:14.825562  8721 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:15:14.819685  8725 raft_consensus.cc:740] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:14.827389  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:14.828617  8027 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:14.830761  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 46914us
I20260709 13:15:14.831681  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:14.816707 14227 raft_consensus.cc:3055] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:14.826908  8725 consensus_queue.cc:260] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.833207  8027 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:14.860570  8207 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:15:14.829834 13932 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:14.839788 13983 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:14.802407 14011 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:15:14.807685 13703 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:14.839015 14242 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:14.841053 13413 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:14.838096  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:14.841341 14231 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:14.840595 14227 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:14.866400  8027 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:14.869163  8204 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3565) took 1120 ms. Trace:
I20260709 13:15:14.875056  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:14.850593 13780 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.866914 14205 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:14.887245  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:14.888337  8135 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:15:14.880496  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 196052us
I20260709 13:15:14.880911  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:14.863831 14084 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:14.881989  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:14.851392 14250 consensus_queue.cc:1059] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:15:14.884665  8204 rpcz_store.cc:276] 0709 13:15:13.749232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.813444 (+ 64212us) service_pool.cc:224] Handling call
0709 13:15:13.813673 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:13.813843 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:15:13.910294 (+ 96451us) log.cc:844] Serialized 14 byte log entry
0709 13:15:14.770733 (+860439us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b18004093c0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6dbabbe4bc 00007f6db7f0c48b 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b 00007f6dbabb5d23 00007f6dbabb623a
0709 13:15:14.798528 (+ 27795us) log.cc:844] Serialized 14 byte log entry
0709 13:15:14.868849 (+ 70321us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:14.868912 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:14.868949 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:14.869008 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:14.869105 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27389,"prepare.queue_time_us":221992,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":44607,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":14095,"spinlock_wait_cycles":8132864,"thread_start_us":311176,"threads_started":3,"wal-append.queue_time_us":170626}
W20260709 13:15:14.898639  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 144341us
I20260709 13:15:14.904426  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:14.906292  8175 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:14.909644 13940 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:14.893353 13932 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:14.884233 13413 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.870666 14149 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:15:14.912375  8175 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:14.889904 13991 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:14.913627 14258 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:14.914881 13413 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:14.916064 14227 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.889174 14127 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:15:14.853227 14167 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.841107 14234 consensus_queue.cc:237] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.912503 14177 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:14.897852 13983 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.922173  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:14.927954  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:14.886698 14212 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.944653 14145 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:14.945403  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:14.951031 13983 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.953138 13780 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:14.920799 14109 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 13:15:14.917326 14231 consensus_queue.cc:1059] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:15:14.924592 14244 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:15:14.959435  8135 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 17: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:14.922688 13940 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:14.938428 14204 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.960050  8211 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3553) took 1214 ms. Trace:
I20260709 13:15:14.975484  8211 rpcz_store.cc:276] 0709 13:15:13.745965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.751036 (+  5071us) service_pool.cc:224] Handling call
0709 13:15:13.751345 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:14.075020 (+323675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:14.959747 (+884727us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b540025cbd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:14.959980 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":732554112}
I20260709 13:15:14.920279 14205 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:15:14.979238 13983 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:14.959964 14227 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:14.984752 14236 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.264s	user 0.001s	sys 0.000s
I20260709 13:15:14.986037 14190 raft_consensus.cc:3055] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:14.986445 14190 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:14.987262 14086 consensus_queue.cc:1059] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:14.989723 14204 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:14.991946 14204 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:14.992591 14205 consensus_queue.cc:1243] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 8a968cada1e4446ebb2a5e167176e649 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:15:14.993003 13413 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:14.993654  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:14.994311  8726 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:14.987284 14190 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.976075 13750 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:14.995697 14190 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:14.996917 13934 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:14.997268 13747 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:14.997656 13747 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:14.997470 13934 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.998610  8027 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:14.998807 13753 raft_consensus.cc:3055] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:14.998500 13747 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:14.999214 13753 raft_consensus.cc:740] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:14.999586 14204 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:15.000018 14204 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.999967 13753 consensus_queue.cc:260] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.001222 13753 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:15.002115 14082 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:15.004132 14232 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:15.004545 14232 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:14.996917  8693 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:14.969666  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 47557.7us
I20260709 13:15:15.000524 14204 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:15.005442 14232 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:15.008998 14232 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:15.011632 13934 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:15.012077  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:15.012720  8138 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:15.013126  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:15.015731 14250 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:15.017205 14246 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:15.017638 14246 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:15.018471 14246 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.019807 14246 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:15.026962  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:15.030463  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:15.031478  8135 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 19: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:15.033316  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:15.039043  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:15.041747  8693 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:15.006934  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:14.962859  8029 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:14.975062 14210 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.005s	sys 0.013s
W20260709 13:15:15.044889  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 82150us
I20260709 13:15:15.053051  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:15.053797  8211 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:15.060235  8135 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:15.063180  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:14.999680 13747 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:15.063783  8131 raft_consensus.cc:1240] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:15.064431  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:15.065615  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:15.066398  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:15.066972  8204 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:15.070641  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:15.049606 14210 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.005s	sys 0.013s
I20260709 13:15:15.070588  8102 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:15.043444 14248 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:15.045847 14225 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:15.049976 14230 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:15.078438  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:15.078652  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:15.075932  8134 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:15.076349  8726 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:15.049073 14220 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:15.083160  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:15.049228 14259 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:15.082482  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:15.089781  8132 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:15:15.079051  8206 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:15.085402  8134 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:15.064090 13678 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 13:15:15.046046 14204 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:15.110157 14204 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:15.081274 14136 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:15.087956 14210 raft_consensus.cc:3055] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:15.111114 14210 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:15.111975 14210 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:15.112906  8724 raft_consensus.cc:1240] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:15.112972 14210 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:15.022372 13800 consensus_queue.cc:1059] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:15:15.114235 13815 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:15.114904 13800 raft_consensus.cc:3055] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:15.115263 13800 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:15.115829 13780 raft_consensus.cc:3055] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:15.082544 13940 consensus_queue.cc:1059] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:15:15.053051 13750 consensus_queue.cc:1059] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:15:15.116225 13780 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:15.116647  8059 raft_consensus.cc:2368] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 19. Current term is 20.
I20260709 13:15:15.117009 13780 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:15.082647 14248 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.120992 14248 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:15.121788 13940 raft_consensus.cc:3055] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:15.122157 13940 raft_consensus.cc:740] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:15.123237 14145 raft_consensus.cc:3055] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:15.122931 13940 consensus_queue.cc:260] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:15.123646 14145 raft_consensus.cc:740] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:15.124404 13940 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:15.124444 14145 consensus_queue.cc:260] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:15.125718 14145 raft_consensus.cc:3060] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:15.126206 13750 raft_consensus.cc:3055] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:15.126602 13750 raft_consensus.cc:740] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:15.127427 13750 consensus_queue.cc:260] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.128474 13750 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:15.129422 14238 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.351s	user 0.003s	sys 0.000s
I20260709 13:15:15.141065  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:15.146103  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:15.146714  8130 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:15.147658  8208 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:15.148047  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:15.148607  8721 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:15:15.106870  8211 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:15:15.091261 14225 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:15.152063  8208 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:15:15.152695  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:15.158747  8209 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:15.153405  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:15.116070 13800 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:15.155634  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:15.118212 13780 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:15.163028 14042 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:15:15.171147 14042 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:15.171505 14042 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:15.172318 14042 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:15.173715 14042 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:15.183271  8130 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 20.
I20260709 13:15:15.162618 13800 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:15.187697  8210 raft_consensus.cc:3055] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:15.237902  8210 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:15:15.188462 14237 log.cc:927] Time spent T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.415s	user 0.001s	sys 0.001s
I20260709 13:15:15.196228 14250 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:15.205160  8137 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:15.182229  8721 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 20.
I20260709 13:15:15.241185  8027 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:15.235881  8211 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:15.230932  8104 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:15.238687  8210 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 19.8, Last appended by leader: 8, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.242545  8210 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:15.248443 14246 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:15.249029 14232 consensus_queue.cc:1059] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:15.249409 14167 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:15:15.250672  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 93375.7us
I20260709 13:15:15.251277  8029 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:15.252507  8029 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:15.256465  8029 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:15.257817 13678 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:15.258663  8210 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 19 index: 8. Preceding OpId from leader: term: 20 index: 8. (term mismatch)
I20260709 13:15:15.259029  8210 pending_rounds.cc:85] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 7
I20260709 13:15:15.259235  8210 pending_rounds.cc:107] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 19.8
I20260709 13:15:15.259444  8210 raft_consensus.cc:2889] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:15.260128  8059 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:15.262064  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:15.266414  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:15.267175  8057 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:15:15.249899 14167 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:15.255263 14232 raft_consensus.cc:3055] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:15.264111 13413 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:15.276443  8104 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:15.276983  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:15.263881 13934 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:15.271188 13747 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:15:15.274518 14154 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:15:15.270409 13983 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:15:15.301162 13703 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:15.274600 14232 raft_consensus.cc:740] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:15.278007 13413 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:15.322580  8029 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:15.317401 14167 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:15.326732  8057 raft_consensus.cc:1275] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:15:15.318329 14093 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:15.328088 14093 raft_consensus.cc:3055] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:15.328455 14093 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:15.329281 14093 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.325163 13815 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:15.330972 13815 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:15.331290  8063 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:15.332326 14093 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:15.319875  8725 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:15.334868 14232 consensus_queue.cc:260] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:15.336019 14232 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:15.321645 13934 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:15.338248 13934 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.323663  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:15.341538  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:15.343621  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:15.324108  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:15.283063 13750 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:15.350541 13750 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:15.351068 13750 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:15.346189 14239 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:15:15.331355 13815 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:15.318779  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:15.385051  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 66338.7us
I20260709 13:15:15.386688  8693 leader_election.cc:400] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 19. Current term is 20.
I20260709 13:15:15.387072  8693 leader_election.cc:403] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:15.348459  8206 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:15.358018 13461 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:15.295113 14204 consensus_queue.cc:1059] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:15:15.297827 13747 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:15.282140 14042 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:15:15.325870 13413 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:15.317401  7961 catalog_manager.cc:5697] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 11 to 16, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 16 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:15.366535 14252 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:15.390733  8207 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:15.391546  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:15.392673  8029 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:15.392758  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:15.404129  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:15.394754 14219 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:15.407718  8061 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 7. Preceding OpId from leader: term: 20 index: 8. (index mismatch)
I20260709 13:15:15.430387  8725 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 9.
I20260709 13:15:15.413473  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:15.397585 14220 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 19. Current term is 20.
I20260709 13:15:15.432960  8029 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:15.435475  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:15.439131  8056 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:15.407226 13753 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:15.444383 13747 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:15.444711 14256 raft_consensus.cc:3055] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:15.445173 14256 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:15.431653 13703 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:15:15.433398 14109 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:15.449364  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:15.450606  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:15.453610 13983 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.454133 13983 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.438849 14154 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:15:15.455843 14154 raft_consensus.cc:3055] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:15.456171 13983 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:15.456326 14205 raft_consensus.cc:3055] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:15.456727 14205 raft_consensus.cc:740] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:15.457500 14205 consensus_queue.cc:260] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.460646 14225 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:15.465533  8206 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 13:15:15.406746 13934 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:15.420894 14136 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:15:15.465907  8206 pending_rounds.cc:85] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 7
I20260709 13:15:15.468519  8206 pending_rounds.cc:107] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 13:15:15.468775  8206 raft_consensus.cc:2889] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:15.469091  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:15.406791 13461 raft_consensus.cc:697] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:15.438890 14162 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:15.470939 14247 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:15:15.470714 13461 consensus_queue.cc:237] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:15.456207 14154 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:15.433351 13991 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:15.469758  8060 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:15.475108  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:15.480965  8060 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:15.482370 13934 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:15.486536  8027 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:15.493026  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:15.493620  8131 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:15.497642  8212 raft_consensus.cc:1240] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:15.499439  8209 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:15:15.501873  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:15.513622 14083 consensus_queue.cc:1048] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:15.515300 14243 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:15.515551  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:15.483350 14190 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:15.488260 14205 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:15.516261 14243 raft_consensus.cc:3055] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:15.472079 14154 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.445994 14256 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.542094  8696 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:15.519084 13747 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:15.579995  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 53601.3us
I20260709 13:15:15.581779  8130 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:15:15.523077 13780 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:15.538371 14243 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:15.586283 14243 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.587529 14243 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:15.587769 14249 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:15:15.592202 14270 log.cc:927] Time spent T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:15:15.524876 14190 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:15.578778 13703 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:15:15.593770 14190 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:15.595544 13750 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.597620 13750 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:15.564950 14073 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:15:15.600409 14154 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:15.606263  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:15.606895  8131 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:15.564607 14136 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:15:15.565027 14083 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:15:15.583884 14212 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:15.614645  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:15.621817  8056 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:15.615875  8204 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:15:15.597575 13413 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.618325  8131 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:15.630393  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:15.606010 13815 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.651918  8061 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:15.659526  8056 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:15.645448 13750 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:15.632632 14073 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:15:15.691955 13815 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:15.693248  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:15.702077  8204 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:15.717424  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:15.718251  8137 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 17: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:15.646804 14042 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:15.725677  8029 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:15.731714  8060 raft_consensus.cc:1275] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:15:15.726892 14238 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:15:15.728420 14251 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:15:15.724174 13750 raft_consensus.cc:697] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:15.732820 14205 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:15.744005 13198 consensus_queue.cc:1048] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:15:15.732539 14042 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.770856 14042 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:15.774461 13413 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:15.775238 13750 consensus_queue.cc:237] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.783187  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:15.785259  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:15.785883  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:15.788137  8060 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 16: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:15.808292  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:15.763687 14205 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:15.825238  8029 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:15.826992 14141 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.827690 14249 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:15:15.860066  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 121451us
I20260709 13:15:15.834756 14141 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.897677 14256 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:15.889479 14205 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.924173 14190 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.935236  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:15.936008  8137 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:15.940934  8725 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:15:15.943248  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:15.943832  8061 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:15.948824  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 55675.7us
I20260709 13:15:15.952138  8131 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:15:15.892678 13750 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:15.926327 14190 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:16.037993  8137 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:16.048404  8061 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
W20260709 13:15:16.057879 14281 log.cc:927] Time spent T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:15:16.059698 14283 log.cc:927] Time spent T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:15:16.050182  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:16.075943  8203 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:15:15.956422 13934 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.104s
I20260709 13:15:15.958398 13703 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.107s
W20260709 13:15:16.097273  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 63880.1us
I20260709 13:15:16.078258 14154 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:16.034088 13750 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:16.091825 14285 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:15:16.113189 14141 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:16.117249  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:16.122655  8203 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:16.123842  8029 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:16.127466 13750 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:16.121954 14154 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:16.129946 14154 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:16.134709  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:16.135483  8136 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:16.090113 13932 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:15:16.137473  8137 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:15:16.134202  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:16.140218  8054 raft_consensus.cc:2393] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 17: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:16.141113 14141 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:16.143834 14141 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.144187 14141 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:16.146500  8208 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
W20260709 13:15:16.164813 14294 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:15:16.121285 14297 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:16.244149  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 79225.5us
I20260709 13:15:16.260006  8725 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:15:16.119256 14167 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
W20260709 13:15:16.304344  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 189697us
I20260709 13:15:16.317731  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:16.296355 14212 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:16.318602  8136 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:16.293018 14297 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:16.327090 14212 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:16.383136  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 45888.8us
I20260709 13:15:16.388485 14232 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:16.390244  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 56849.1us
I20260709 13:15:16.389017 14232 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:16.396956  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:16.397740  8206 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:16.399986  8696 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:16.394155 14232 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:16.401346 14232 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:16.401758 14232 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.402194 14232 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:16.381986  8063 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:15:16.437081 14297 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:16.371066 14311 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
W20260709 13:15:16.442488 14280 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:15:16.494388 14136 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.502784 14136 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:16.504801 14136 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:16.508242 14212 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:16.508989  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 64305.1us
I20260709 13:15:16.511404  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:16.512137  8203 raft_consensus.cc:2393] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 19: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:16.513326  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:16.513885  8204 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:16.515280  8207 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:15:16.516399 14154 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:15:16.519533  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 68541.1us
I20260709 13:15:16.534696  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:16.548032  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:16.548635  8130 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:16.542227 14127 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:16.559798  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:16.558773  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:16.559140  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:16.560920  8727 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:16.543565 14313 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:16.560402  8136 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:16.532980 14232 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:16.555549 14311 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.542800 14312 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:15:16.554879 14042 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.577025 14042 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:16.579310 14042 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:16.553746 14322 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:15:16.569883 14149 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:16.582226 14149 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:16.584317 14149 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:16.585652 14311 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:16.587811 14311 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:16.596534  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:16.599858  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:16.600668  8209 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:16.562616 14127 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:16.603143  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:16.606029  8207 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 18: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:16.571156 14313 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:16.613184 14313 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:16.618357 14167 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.618824 14167 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:16.620710 14167 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:16.574488 14154 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:16.623840  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:16.624107  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 59837.5us
I20260709 13:15:16.626583  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:16.627139  8206 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:16.628691  8029 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:16.629550 14311 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:16.629897 14311 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.630245 14311 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:16.638109 13753 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.638664 13753 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:16.640659 14134 raft_consensus.cc:3055] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:16.640735 13753 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:16.641187 14134 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:16.643687 14154 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:16.643931  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:16.644403 14288 log.cc:927] Time spent T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.351s	user 0.003s	sys 0.000s
I20260709 13:15:16.644833  8207 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 19: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:15:16.644423  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 50862.1us
I20260709 13:15:16.629014  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:16.647575  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:16.648113  8721 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:16.605396 14127 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:16.653015  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:16.653774  8724 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:16.656253 13753 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:16.656786 13753 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:16.659821 13753 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:16.662670  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:16.663241  8136 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:16.670742  8027 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:16.672426  8026 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:16.651551 14190 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:16.711457 14190 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:16.721191 14190 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:16.722258 14232 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:16.743646  8136 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:15:16.744071 14127 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:16.746232 14127 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:16.708542 14243 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:16.755393 14243 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:16.694679 14313 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:16.758865 14313 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.758965  8104 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:16.759523 14313 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:16.760681 13821 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:16.761054 13821 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.761384 13821 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:16.709972 14205 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:16.766391 14205 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:16.766875 14205 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:16.769403 14243 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:16.771004 14091 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.771656 14091 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:16.777287 13821 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:15.648581  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:16.785089 13821 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:16.795265 14127 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.796028 14127 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:16.798075 14127 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:16.801206 14109 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:16.814085 14154 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:16.821949  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:16.822880  8204 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 16: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:16.642027 14134 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:16.845610  7961 catalog_manager.cc:5697] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 13 to 15, now has a pending config: VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66), VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67), VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 15 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:16.857002  7961 catalog_manager.cc:5697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:16.859112  7961 catalog_manager.cc:5697] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 13 to 15, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 15 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:16.861991  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:16.867792 14134 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:16.873216 14230 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.971259 14230 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:16.876055 14315 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:16.877923 14312 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.973840 14315 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:16.975901 14312 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:16.984730 14230 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:16.988852 14315 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:16.990629 14312 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:16.884572  8136 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:15:16.887557 14244 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.991802 14244 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:16.994372 14244 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:16.891155 14091 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:16.895364  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:16.995805  8208 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:16.907850 14311 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:16.912199 14141 consensus_meta.cc:287] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.768s	user 0.010s	sys 0.008s
I20260709 13:15:17.003042  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:17.002974 14141 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:17.003818  8209 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:17.005489  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:17.005730 14141 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:16.920490 14205 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.006212  8209 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:17.007798  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:17.008095 14205 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:17.015631  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:16.935582 13413 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.018715 13413 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:17.021314 13413 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:16.950110 14086 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:17.022087 14086 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:17.024761 14086 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:17.016768  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:17.027110  8209 raft_consensus.cc:2393] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 19: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:17.017643  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:17.012773  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:17.028374  8206 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:17.032821  8208 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
W20260709 13:15:17.033954  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 482057us
I20260709 13:15:17.042299  8696 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:15:17.046002  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 490651us
I20260709 13:15:17.046119  8723 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:17.051196  8027 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:17.055974 14316 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.056905  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:17.057214  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:15:17.055166  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 106162us
I20260709 13:15:17.057786  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:17.058252  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:17.059031 14232 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:17.056479 14316 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.059413 14232 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:17.059830 14232 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:17.060577  8730 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:17.066457 14316 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:17.067672  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:17.073148  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:17.073541  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:17.073865  8724 raft_consensus.cc:2393] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 16: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:15:17.074389  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:17.081022  8102 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:17.086091  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:17.096611  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:17.103467  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:17.104329  8130 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:17.105391  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:17.107388  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:17.108354  8135 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 17: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:17.097338  8730 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:17.054790 14219 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.106204  8131 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:17.111302  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:17.116612  8130 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 17
W20260709 13:15:17.113171  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 54564.5us
I20260709 13:15:17.106722  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:17.071370 14141 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:15:17.134860  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 273004us
I20260709 13:15:17.136377  8104 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:17.126401  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:17.127861  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:17.121515  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:17.074621 13750 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.078208 14250 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:17.138206  8135 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:17.140980  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:17.141788  8138 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:17.143179  8693 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:17.143267  8132 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:15:17.148833  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:17.094674 14312 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:17.149605  8129 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:17.092442 14315 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:17.166733 14073 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:17.171408  7961 catalog_manager.cc:5697] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 17 to 18, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 18 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:15:17.174731  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 93832.2us
I20260709 13:15:17.179380  7961 catalog_manager.cc:5697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.181332  8134 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:17.174132 14073 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.182834 14073 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:17.182500 14315 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.185228 14127 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:17.185629 14127 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.185940 14127 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:17.186710 14042 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:17.187224 14042 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.187628 14042 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:17.191490 14311 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:17.129935 14219 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.195920  8721 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:17.196956 14219 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:17.197228  7960 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34134 (request call id 266) took 39096 ms (client timeout 15000 ms). Trace:
I20260709 13:15:17.231074  8176 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:17.209427  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:17.181591  7961 catalog_manager.cc:5697] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 7 to 12, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 12 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:17.151269 13703 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:17.147009 13934 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:17.265549  7961 catalog_manager.cc:5697] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 12 to 16, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 16 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:17.278326  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:17.279109  8730 raft_consensus.cc:1813] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Handling vote request from an unknown peer 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:17.279871  8730 raft_consensus.cc:2410] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:15:17.238173  8723 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:17.267076 13940 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:17.270645 13703 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:17.129676 14332 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.239818 14254 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:15:17.199769  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 63476.1us
I20260709 13:15:17.229351  8130 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:17.235594 14312 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:17.199082 14250 raft_consensus.cc:697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:17.468721 13703 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:17.278507  7961 catalog_manager.cc:5697] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.198910 14315 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:17.414964  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 212283us
W20260709 13:15:17.255162  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 75796.1us
I20260709 13:15:17.489759  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:17.387089  8203 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.104s	sys 0.107s
W20260709 13:15:17.490497  8203 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.105s	sys 0.109s
W20260709 13:15:17.239519  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 121832us
I20260709 13:15:17.490803  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:17.491333  8210 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:17.424847 13461 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.207468  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:17.151160 14141 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.494210  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:17.494805  8212 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:17.495563  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:17.435319 13940 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:17.197608  7960 rpcz_store.cc:269] 0709 13:14:38.101428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.101715 (+   287us) service_pool.cc:224] Handling call
0709 13:15:17.197141 (+39095426us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:10.384724 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:10.438471 (+ 53747us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:10.445634 (+  7163us) write_op.cc:435] Acquired schema lock
0709 13:15:10.452627 (+  6993us) tablet.cc:662] Decoding operations
0709 13:15:10.524112 (+ 71485us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:10.524201 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:10.524226 (+    25us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:15:10.525110 (+   884us) tablet.cc:701] Row locks acquired
0709 13:15:10.525156 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:15:10.525329 (+   173us) write_op.cc:270] Start()
0709 13:15:10.525477 (+   148us) write_op.cc:276] Timestamp: P: 1783602910525269 usec, L: 0
0709 13:15:10.525509 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:15:10.526094 (+   585us) log.cc:844] Serialized 3604 byte log entry
0709 13:15:11.246689 (+720595us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b18002d05e0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f29522 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:15:11.552586 (+305897us) op_driver.cc:464] REPLICATION: finished
0709 13:15:12.080691 (+528105us) write_op.cc:301] APPLY: starting
0709 13:15:12.120731 (+ 40040us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:12.365909 (+245178us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:12.365950 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:12.605718 (+239768us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:12.605999 (+   281us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 13:15:12.606043 (+    44us) write_op.cc:312] APPLY: finished
0709 13:15:12.652732 (+ 46689us) log.cc:844] Serialized 89 byte log entry
0709 13:15:12.679342 (+ 26610us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:12.679477 (+   135us) write_op.cc:454] Released schema lock
0709 13:15:12.679539 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:12.712393 (+ 32854us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:12.719020 (+  6627us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2334,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":362836,"apply.run_cpu_time_us":20119,"apply.run_wall_time_us":646301,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41747,"num_ops":9,"prepare.queue_time_us":100804,"prepare.run_cpu_time_us":64849,"prepare.run_wall_time_us":300495,"raft.queue_time_us":538668,"raft.run_cpu_time_us":19185,"raft.run_wall_time_us":545250,"replication_time_us":901529,"spinlock_wait_cycles":6382976,"thread_start_us":537891,"threads_started":5,"wal-append.queue_time_us":198727,"wal-append.run_cpu_time_us":14043,"wal-append.run_wall_time_us":2271612}]]}
I20260709 13:15:17.454535 14334 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:17.455139 14091 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:17.450321 14073 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.460790  8723 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:17.472024 14254 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.194085 14314 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:17.288053 14232 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.474795 13413 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:17.527670  8027 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:17.474757  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:17.271374 13934 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.492072 13461 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:17.500294  8102 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:17.536653  8102 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:17.501327  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:17.537359  8208 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:17.504801  7961 catalog_manager.cc:5697] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 13 to 18, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 18 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:17.158617 13750 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.511328  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:17.538944  7961 catalog_manager.cc:5697] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.548882  7961 catalog_manager.cc:5697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 12 to 13, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 13 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:17.512045  8210 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:17.512893  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:17.507145 13940 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:17.550173  7961 catalog_manager.cc:5697] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 13 to 16, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 16 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.527313 14254 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:17.554085  7961 catalog_manager.cc:5697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:17.552106  8206 consensus_meta.cc:287] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.523s	user 0.007s	sys 0.003s
I20260709 13:15:17.558615  8206 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:17.552994 13800 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:17.559576 13800 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:17.559995 13800 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:17.525806 14314 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:17.570561 13461 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:17.571228 13461 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:17.571748 13461 raft_consensus.cc:697] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:17.563387  7961 catalog_manager.cc:5697] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:17.574496  8209 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:15:17.573558  7961 catalog_manager.cc:5697] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 8 to 11, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 11 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:17.575604  7961 catalog_manager.cc:5697] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:17.547130 13934 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:17.579696  7961 catalog_manager.cc:5697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:17.549161 13780 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:17.475309 14332 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.556725 14011 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:17.514158 14334 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:17.527411 14091 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:17.584120 13780 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:15:17.627774  8204 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.098s	sys 0.134s
W20260709 13:15:17.630752  8204 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.098s	sys 0.134s
I20260709 13:15:17.574350 14314 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:17.543212 13413 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:17.601459 13461 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.633430 13780 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:17.659389 14335 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:15:17.663295  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:17.663937  8723 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:17.664989  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:17.577780 14312 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.584381 14250 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.662209 14332 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:17.670576 13753 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:17.692350 13753 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.692728 13753 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:15:17.706199 14313 consensus_meta.cc:287] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.944s	user 0.012s	sys 0.000s
W20260709 13:15:17.722568  8062 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.002s	sys 0.005s
W20260709 13:15:17.723042  8062 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.003s	sys 0.005s
I20260709 13:15:17.711081 14141 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:17.708814 14073 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:17.735929  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:17.739897  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:17.745283  8130 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:17.747025 13461 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.747519 13461 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:17.718616 14232 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:17.668808 14042 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.739015 14134 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.719915 14313 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:17.785048  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:17.800352  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:17.801312  8721 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 19: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:15:17.805168  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:17.806075  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:17.806753  8209 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:17.805346  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:15:17.829023  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 44558.9us
W20260709 13:15:17.831874  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 320610us
I20260709 13:15:17.781880 14091 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:17.856408 13461 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:17.799870 13413 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:17.803316 13750 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:17.781297 14134 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:17.849267 14073 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:17.830632 14307 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:15:17.874194  8175 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.032s	sys 0.064s
W20260709 13:15:17.874549  8175 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.032s	sys 0.064s
W20260709 13:15:17.874796  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 696156us
I20260709 13:15:17.856196 14334 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:17.875590  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:17.881717  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:15:17.885699  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 372872us
I20260709 13:15:17.885187  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:17.899598  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:17.883451 14334 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:17.873560 13413 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:15:17.902940  8130 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:17.882349 14073 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:17.908396  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:17.908954  8130 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:17.893186 13750 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:17.914290  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:17.915233  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:17.878240 14307 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.910792  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:17.900583 14148 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:17.915920  8137 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:17.906453 14154 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:15:17.909307 14127 consensus_meta.cc:287] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.720s	user 0.011s	sys 0.000s
I20260709 13:15:17.895556 14091 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:15:17.906327 14334 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:17.913750 13750 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:17.939509 13750 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:17.927937  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:17.940758  8063 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:17.927552 14073 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.928227 13940 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.931282 14011 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:17.932715 14340 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:15:17.925565 14310 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:17.938539 14313 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:17.929798 14127 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.922972  8062 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:17.944320 14042 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:17.918686 14148 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.983943  8209 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:17.913301 13800 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:17.924104 14344 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:17.967373  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:17.967947  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:17.970721 14337 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:15:17.971573 14307 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:17.969216  8029 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:18.015974  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:17.973909  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:17.924285 14345 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.976830  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:17.966459 13753 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:17.982046 14331 log.cc:927] Time spent T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:15:17.983147 13461 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.027880 13461 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:17.941475 14334 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:17.985061  8696 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:15:18.031065  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 46082.5us
I20260709 13:15:17.941097 14339 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:17.986896  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:17.945976 14154 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:17.989048  8057 raft_consensus.cc:2393] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 16: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:17.991495 13940 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:17.993222 14127 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:17.987094 14148 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:17.994446 13800 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:18.043643 14348 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:15:17.995201  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:17.934103 13703 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.045020  8203 raft_consensus.cc:2368] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 19. Current term is 20.
I20260709 13:15:17.938324 13780 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:18.045969 13753 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:17.995800  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:18.043884  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:18.046500 13703 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:18.039086  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 129572us
I20260709 13:15:18.047592 13780 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:18.001608  8134 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:18.002188  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:18.002631  8207 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 17: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:18.012670  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.050776  8206 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:17.976186  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:18.051565  8723 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:18.005125  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:18.052464  8724 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:18.027295 13413 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:18.019496  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.014935  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:18.054538  8055 raft_consensus.cc:2393] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 17: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:18.022390  8054 raft_consensus.cc:2368] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
W20260709 13:15:18.019816  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 50676.7us
I20260709 13:15:18.031767  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:18.056318  8029 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:18.057155  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:18.058511 14313 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:18.058708 14093 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:18.066854 14093 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.067173  8730 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:18.027951 14345 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.067796  8696 leader_election.cc:400] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 19. Current term is 20.
I20260709 13:15:18.053637 13413 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:18.068789 14093 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.068990 13413 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:18.058871 14230 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:18.069773 14230 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.069499 13934 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.070174 14230 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:18.073413 14314 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.062669 14254 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.075133 14154 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.075978 14254 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.039099 14342 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:18.076779 14342 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:18.077580 14342 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:18.078423 13461 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.078516  8129 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:17.988202 14344 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:18.060969  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:18.079406 14344 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.080626 14344 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:17.974673 14011 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.083159 13413 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.083390  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:18.083756 14011 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.083932 14314 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.084774 14256 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:18.085249  8060 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:18.085310 14256 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.042001 14339 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.087411 14339 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:18.087541 13413 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.062466 14134 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.068817 14345 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.084818  8726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55848 (request call id 1097) took 1554 ms. Trace:
I20260709 13:15:18.087428 14256 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:18.058321  8029 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:18.031905  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:18.064391  8723 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:18.094360  8129 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:18.102398 14316 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:18.103507 14316 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.103880 14316 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:18.105792 14313 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:18.088480  8726 rpcz_store.cc:276] 0709 13:15:16.530469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.530721 (+   252us) service_pool.cc:224] Handling call
0709 13:15:16.530965 (+   244us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:16.531257 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:16.531282 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:16.539532 (+  8250us) log.cc:844] Serialized 26 byte log entry
0709 13:15:16.702619 (+163087us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:16.702656 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:16.702714 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:18.084634 (+1381920us) raft_consensus.cc:1685] finished
0709 13:15:18.084652 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:18.084768 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1222,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":8703,"thread_start_us":147584,"threads_started":1,"wal-append.queue_time_us":924076}
I20260709 13:15:18.107280 14334 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:18.109915  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:15:18.112332  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 80503.8us
I20260709 13:15:18.113772  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:18.114073  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:18.115331  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:18.118570  8210 raft_consensus.cc:2368] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17.
I20260709 13:15:18.117081  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:18.115911  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:18.120497  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:18.121006  8137 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:18.123016  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:18.123512  8130 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:18.126863  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.128258  8211 raft_consensus.cc:2368] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 18. Current term is 19.
I20260709 13:15:18.129427  8724 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:18.127573  8132 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:18.130817  8176 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:18.134320  8130 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:18.135231  8102 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:18.202852  8209 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
W20260709 13:15:18.202032  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 70123us
I20260709 13:15:18.202330  8102 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:18.208701  8175 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:18.209432  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.217404  8137 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:18.220791  8207 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:18.220988 13413 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:18.221652 13413 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.202003 13940 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:18.135452 14314 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:18.229713  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:18.139917 13747 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:18.231395  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:18.208118 14127 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:15:18.242862  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 102329us
I20260709 13:15:18.128100 14256 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:18.244450  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:18.137553 14339 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:18.235179  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:18.251168  8206 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:18.246902 13461 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:18.247478  8203 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:18.243259 13413 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.103874 13934 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.224062 14345 raft_consensus.cc:2764] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:15:18.252192 13461 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.220041 14091 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:18.220883 14141 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:18.246435 14312 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:18.259210 14339 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:18.265820 14243 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:18.263477 14332 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:18.266611 14091 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.263525  8207 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:18.260569 14256 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.253898 13747 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.223732 13753 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:15:18.278623  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.356177  8129 raft_consensus.cc:2368] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:18.279309  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:18.270452  8104 leader_election.cc:400] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17.
I20260709 13:15:18.357407  8132 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:18.357615  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 87337.4us
I20260709 13:15:18.248881 14311 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:18.282088  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.359427  8130 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:18.282770  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.282879  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.360128  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.287745  8206 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:18.245869 13940 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.290161  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:18.364228  8724 raft_consensus.cc:2368] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:15:18.360337  8135 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:18.267271 14134 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:18.259557 14346 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:18.264081 14127 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:18.276171 13750 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.259755 14314 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:18.269874 14334 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:18.254168 14244 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:18.267954 14312 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.271575 14148 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.354043 13461 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.283440  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.382948  8134 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:18.360385  8138 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:18.370173 14356 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:18.365540  8176 leader_election.cc:400] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:15:18.327872 14339 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:18.284727 14141 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:18.384647  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 245646us
I20260709 13:15:18.360870  8129 raft_consensus.cc:1813] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Handling vote request from an unknown peer 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:18.393143  8129 raft_consensus.cc:2368] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 16. Current term is 17.
I20260709 13:15:18.289386 14332 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.388239  8208 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:18.390529  8176 leader_election.cc:403] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:15:18.394013  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.396009  8138 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
W20260709 13:15:18.396178  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 187568us
I20260709 13:15:18.370914 14311 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.408273 14332 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:18.396874  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:15:18.412580  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 47225.8us
I20260709 13:15:18.413239  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:18.376636 14346 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:18.417430 14346 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:18.374979 14134 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:18.419502  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:18.419809 14134 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:18.420064  8138 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:18.420715  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.421059  8123 tablet_service.cc:1558] Processing DeleteTablet for tablet a478a4313e6149378cae4b6f833e3c2a with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 4 (current committed config index is 9)) from {username='slave'} at 127.0.0.1:36736
I20260709 13:15:18.421818  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.422000  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.422684  8131 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:18.422847  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:18.423527  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:18.423764 14356 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:15:18.424353  8131 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:18.408288 14311 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.425280  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:18.425966  8136 raft_consensus.cc:2368] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:18.427258 14091 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:18.428575  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.423395  8130 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:18.427187  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.431886  8129 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:18.433671 14367 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:18.434113 14367 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:18.434484 14367 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:18.434659  8138 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:18.435576  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.436156  8060 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:18.438064  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.439036 14334 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:18.417694  8208 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:18.414227  8175 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:18.442987  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.443715  8061 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
I20260709 13:15:18.439453 14334 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:18.444730  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.446609 14334 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:18.446925  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:18.447177 13940 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:18.447607  8060 raft_consensus.cc:2368] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17.
I20260709 13:15:18.441922  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:18.448119 13747 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.448436  8055 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:18.299969 14230 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.453962  8057 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 18: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:18.442844  8132 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:18.460620 13750 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:18.466816  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 52659.9us
I20260709 13:15:18.467765  8175 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:18.469929  8026 leader_election.cc:400] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:18.472211  8026 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:18.472859  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.474916  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.388671 14244 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:18.462442 14362 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
W20260709 13:15:18.485466  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 85389.9us
I20260709 13:15:18.483572 13750 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:18.489363 13750 raft_consensus.cc:697] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:18.489720 14344 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:18.489421  8026 leader_election.cc:400] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 16. Current term is 17.
I20260709 13:15:18.490392 13750 consensus_queue.cc:237] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:18.492396  8026 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:18.490116 14344 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.493131 13932 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:18.493566 13932 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.494097 13932 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:18.493500 14344 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:15:18.503927  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 145671us
I20260709 13:15:18.504287 14339 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.504858  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.505326  8211 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:18.505599  8138 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:18.487428  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:18.504734  8104 leader_election.cc:400] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 18. Current term is 19.
W20260709 13:15:18.493780 14363 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:15:18.461274 14314 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:18.478881 14364 log.cc:927] Time spent T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 13:15:18.507364  8104 leader_election.cc:403] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:15:18.508379  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.506384  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:18.517997  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:18.518800  8138 raft_consensus.cc:2368] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:18.469066 14091 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:18.514647  8130 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:18.469178 14356 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:18.513901  8101 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:18.533557  8175 leader_election.cc:400] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:18.533864  8175 leader_election.cc:403] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:15:18.508678  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.506647  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:18.529752  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:18.520787  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:18.535595  8138 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 17: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:18.512250  8029 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:18.514245  8055 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:18.485503 14243 raft_consensus.cc:697] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:18.520802  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:18.536047 14356 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:18.547016  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:18.548245  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:18.552114  8204 raft_consensus.cc:2368] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:18.549623  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:18.553174  8206 raft_consensus.cc:2368] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 18. Current term is 19.
I20260709 13:15:18.553417  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:18.554229 13413 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:18.554845  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:18.555362  8207 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:18.555804  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:18.556967  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:18.556525  8204 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:18.515571 14311 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:18.554680 13413 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.561450  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:18.561589  8209 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:15:18.506317 14339 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:18.567710  8207 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:18.456337 14127 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:18.539395 14347 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 18. Current term is 19.
I20260709 13:15:18.538039 13932 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:18.619335  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 61180.4us
I20260709 13:15:18.550251 14154 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:18.621996  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:18.558387 13934 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:18.551616 14091 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:18.568380 13800 raft_consensus.cc:2764] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:15:18.630919 13413 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.631251  8104 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:18.635381 14091 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:18.636152 13932 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.569574 14011 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:18.659798  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 86380.6us
I20260709 13:15:18.660703  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:18.570818 13750 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:15:18.665880 14148 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:18.619903 14318 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:18.673857  8101 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:18.665928  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 106915us
I20260709 13:15:18.675595 14346 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:18.667784 13800 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:18.677634 14244 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.678182 14141 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.679550 14344 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.681429 14356 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:18.630401 13934 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.710695  8726 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:15:18.710947 13934 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:18.712985 14344 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.681820 14334 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.682855 14312 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.685075 14367 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.685557  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:18.659629 14256 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:18.693320 14230 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.694577 14318 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.695351 14145 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:18.699393  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:18.736801  8029 leader_election.cc:400] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:18.737129  8029 leader_election.cc:403] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:15:18.737778  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:18.699959  8104 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:18.703117  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.741110  8060 raft_consensus.cc:2368] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 19. Current term is 20.
I20260709 13:15:18.703492  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.742125  8063 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:18.703809  8723 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:18.701845  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.704391  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.703923  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:18.705951  8129 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:18.706192  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.744376  8061 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:18.708875  8134 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:15:18.713828  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:18.663693 14347 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:15:18.697526 14369 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:15:18.717868  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:18.719537  8101 leader_election.cc:400] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17.
I20260709 13:15:18.761237  8059 raft_consensus.cc:2368] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18.
W20260709 13:15:18.761416  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 51322.1us
I20260709 13:15:18.726094  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:18.727288  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:18.727627  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:18.730695  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:18.732553  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:18.764339  8212 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:18.733373  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:18.765154  8205 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:18.745874  8056 raft_consensus.cc:2368] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:18.746578  8175 leader_election.cc:400] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:18.771356  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.771579  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:18.772069  8129 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:18.747741 14145 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.775440 14145 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.706944 14252 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:18.775658  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:18.776525  8131 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
W20260709 13:15:18.723598 14316 consensus_meta.cc:287] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: Time spent flushing consensus metadata: real 0.619s	user 0.009s	sys 0.000s
I20260709 13:15:18.778575  8210 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:18.704845 14346 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:18.780751 14346 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:18.785867  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:18.791877  8131 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:15:18.804159  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 73061.7us
I20260709 13:15:18.805132  8029 leader_election.cc:400] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 18. Current term is 19.
I20260709 13:15:18.805400  8029 leader_election.cc:403] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:15:18.722985 13800 raft_consensus.cc:697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:18.777674 14316 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:18.808168  8205 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 20.
I20260709 13:15:18.797216 14368 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:15:18.713730 14243 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:18.760613 14190 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:18.825186 14318 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.828177 14312 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:18.851524  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:15:18.857751  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 52772.5us
I20260709 13:15:18.857859 14366 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:15:18.858220  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 72454us
I20260709 13:15:18.891960  8029 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:18.858585 13800 consensus_queue.cc:237] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.860731  8212 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:18.897320  8208 raft_consensus.cc:1240] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 19. Current term is 20. Ops: []
W20260709 13:15:18.864152  8026 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.773s (SENT) [suppressed 3 similar messages]
I20260709 13:15:18.867172 14250 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:18.869305 14383 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:18.749269 14347 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.910678 14316 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:18.910817 14347 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:18.760195 13780 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:18.914021 13780 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.914433 13780 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:18.885156  8211 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:15:18.896088  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 44651.2us
I20260709 13:15:18.880070 14256 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.918015  8129 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:18.907837 13934 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:18.906073 13747 consensus_queue.cc:1059] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:18.889500 14346 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:18.924664  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 176379us
I20260709 13:15:18.926581 14366 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 18. Current term is 19.
I20260709 13:15:18.921358 14154 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:15:18.927544 13747 raft_consensus.cc:3055] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:18.899398 13750 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:15:18.989926  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 227647us
I20260709 13:15:18.990084  8131 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:18.989523 13747 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
W20260709 13:15:19.025915  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69581.9us
W20260709 13:15:19.046559  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 92368.7us
I20260709 13:15:19.045571  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:19.051201  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:18.991660 14381 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:15:19.057740  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 291236us
I20260709 13:15:19.052443  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:18.989749 13750 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:18.921619 14256 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:19.098248 14367 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:19.061201 14381 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:19.125991 14391 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:19.127400 13703 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:19.130949 13780 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:19.132849  8205 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:15:19.134374 13780 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:19.049787 14344 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:15:19.131628 13934 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:19.132232 14381 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:19.130059 14346 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:19.063865  8029 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:19.079353 14318 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:15:19.152945  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 89167.6us
I20260709 13:15:19.079685 13750 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:19.153793 13934 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:19.153115 14318 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:19.154510 13934 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:19.155220  8029 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:19.155040 14318 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:19.155984 13934 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:19.158931 13747 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:19.160410 13747 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:19.180787  8029 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:19.211242  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:19.212945  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:19.213889  8212 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 18: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:15:19.226065 14389 log.cc:927] Time spent T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:15:19.215962 14334 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:19.215173 14344 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:19.247597 14219 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:19.247992 14219 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:19.251098 14334 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:19.251585 14334 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:19.253690 14334 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:19.254902 13750 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:19.256981 13750 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:19.239575 14210 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:19.257984 14210 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:19.259315 14381 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:19.260288 14210 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:19.243695 14347 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:19.261713 14256 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:19.261898 14381 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:19.248823 14219 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:19.270089 14344 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:19.284241 14219 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:19.289640 14347 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:19.294653 14386 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.296s	user 0.004s	sys 0.000s
I20260709 13:15:19.296133 14344 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:19.343245  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 291850us
W20260709 13:15:19.347191  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 166360us
W20260709 13:15:19.343245  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 478918us
I20260709 13:15:19.358937  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:19.360711  8208 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 20: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:19.367172 14086 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:19.367920 14086 raft_consensus.cc:697] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:19.370713  8730 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:15:19.371102  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:19.386108 14244 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.394483  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:19.396373  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:19.422883 14244 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:19.423564  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:19.376022 14086 consensus_queue.cc:237] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:19.429188  8724 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:15:19.430449 14341 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:19.431517 14341 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:19.438562  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:19.439271  8129 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:15:19.439759  8133 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 17: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:19.444227  8175 leader_election.cc:400] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 19. Current term is 20.
I20260709 13:15:19.446012 14232 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:15:19.448340 14381 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:19.449230 14381 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:19.455749  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:19.449051 14232 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:15:19.459093 14381 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:19.484236 14381 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:19.460944 14244 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:19.462363  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:19.462783  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:19.497203  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:19.497421  8208 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:19.471374 14341 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:19.471961  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:19.474767  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:19.480237  8054 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:19.482843  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:19.483274  8129 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:15:19.495173  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:19.497936  8137 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:19.505591  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:19.505833 14365 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:19.506383 14365 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:19.506783 14365 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:19.508502  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:19.509364 14332 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:19.510696  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:15:19.511072  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 55583us
I20260709 13:15:19.511411  8212 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:19.516742  8026 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:19.524811  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:19.508199  7945 catalog_manager.cc:5028] TS 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): tablet a478a4313e6149378cae4b6f833e3c2a (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:19.526314 14344 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:19.511420 14332 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:15:19.529222  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:19.529397  8132 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 1894) took 1026 ms. Trace:
I20260709 13:15:19.512153  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:19.530656 14256 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:19.529870  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:19.531271 13750 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:19.531502 14256 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:19.533334  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:19.529881  8132 rpcz_store.cc:276] 0709 13:15:18.503000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.506423 (+  3423us) service_pool.cc:224] Handling call
0709 13:15:19.529049 (+1022626us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b5400062054. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:19.529214 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2280011136}
W20260709 13:15:19.538179  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:19.541821  8723 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:19.545280 14256 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:19.547192  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:19.485193 14381 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:19.561817  8133 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:19.567619  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:19.550832 14244 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:19.569020  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:19.568287 14256 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:19.580892  8212 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:19.571115 14244 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.589883  8054 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:19.592686  8726 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 13:15:19.580834 14256 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:19.593425 14256 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:19.606455  8696 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:19.607712  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 70246.8us
I20260709 13:15:19.609901 14371 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:19.593122  8726 pending_rounds.cc:85] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Aborting all ops after (but not including) 13
I20260709 13:15:19.611681 14381 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:19.611989  8726 pending_rounds.cc:107] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:15:19.612329  8726 raft_consensus.cc:2889] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:19.613299  7961 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37774 (request call id 143) took 9856 ms. Trace:
I20260709 13:15:19.614675  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:19.615715  8132 raft_consensus.cc:2368] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 19. Current term is 20.
I20260709 13:15:19.622685  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:19.626392  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:19.627128 14365 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:19.629522 14365 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:19.630110 14141 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:19.630894 14141 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:19.622740  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 75725.5us
I20260709 13:15:19.633224  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:19.633677 14141 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:19.590757 14244 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:19.635054  8175 leader_election.cc:400] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18.
I20260709 13:15:19.635491  8175 leader_election.cc:403] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:15:19.613732  7961 rpcz_store.cc:276] 0709 13:15:09.756830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:09.757181 (+   351us) service_pool.cc:224] Handling call
0709 13:15:19.613046 (+9855865us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:17.938436 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:17.939171 (+   735us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:17.939212 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:15:17.939228 (+    16us) tablet.cc:662] Decoding operations
0709 13:15:17.944479 (+  5251us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:17.944539 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:17.944569 (+    30us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:15:17.997917 (+ 53348us) tablet.cc:701] Row locks acquired
0709 13:15:17.997939 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:15:17.998098 (+   159us) write_op.cc:270] Start()
0709 13:15:17.998220 (+   122us) write_op.cc:276] Timestamp: P: 1783602917998049 usec, L: 0
0709 13:15:17.998247 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:15:17.998953 (+   706us) log.cc:844] Serialized 7299 byte log entry
0709 13:15:18.698925 (+699972us) op_driver.cc:464] REPLICATION: finished
0709 13:15:19.050363 (+351438us) write_op.cc:301] APPLY: starting
0709 13:15:19.061935 (+ 11572us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:19.248817 (+186882us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:19.248919 (+   102us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:19.297931 (+ 49012us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:19.298233 (+   302us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 13:15:19.298278 (+    45us) write_op.cc:312] APPLY: finished
0709 13:15:19.301022 (+  2744us) log.cc:844] Serialized 164 byte log entry
0709 13:15:19.302306 (+  1284us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:19.302541 (+   235us) write_op.cc:454] Released schema lock
0709 13:15:19.302619 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:19.303110 (+   491us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:19.303148 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2216071,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":294203,"apply.run_cpu_time_us":23084,"apply.run_wall_time_us":303338,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79367,"num_ops":18,"prepare.queue_time_us":200400,"prepare.run_cpu_time_us":38251,"prepare.run_wall_time_us":290819,"raft.queue_time_us":204524,"raft.run_cpu_time_us":28876,"raft.run_wall_time_us":261102,"replication_time_us":670182,"thread_start_us":689154,"threads_started":4,"wal-append.queue_time_us":282639}]]}
I20260709 13:15:19.638751 13750 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:19.639631  8175 leader_election.cc:400] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:19.639815 14356 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18.
I20260709 13:15:19.643915 14332 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:15:19.660989  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 144489us
I20260709 13:15:19.659369 14252 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:19.662432 14252 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:19.658449  8027 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:19.674708  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:19.676213  8054 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
W20260709 13:15:19.678591  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 97503.4us
I20260709 13:15:19.684348  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:19.684944  8129 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:15:19.686647  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:19.695084  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:19.680132 14381 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:19.696202 14381 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:19.696586 14381 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:19.700210 14252 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:19.705134  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:19.713989 14252 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.714570 14252 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:19.715670  8057 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:15:19.716753 14252 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:19.718410  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:19.718817  8059 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:19.721032  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:19.722908  8104 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:19.723845 14252 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:19.724507  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 85077.6us
I20260709 13:15:19.725586  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:19.732609  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:19.733114  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:19.733799  8134 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:19.734373  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:19.735985  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:19.737141  8132 raft_consensus.cc:2393] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:19.737063  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:19.738108  8129 raft_consensus.cc:2393] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 20: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:19.738375  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:19.741606  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:19.742769  8212 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:19.745594  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:19.745893  8062 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 13:15:19.746349  8062 pending_rounds.cc:85] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 13
I20260709 13:15:19.746716  8062 pending_rounds.cc:107] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:15:19.747038  8062 raft_consensus.cc:2889] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:19.742769  8058 raft_consensus.cc:2368] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:15:19.747035 14073 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:15:19.750023  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:19.751534  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:19.752978  7959 catalog_manager.cc:5697] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 9 to 12, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 12 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:19.754501  8026 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:19.765303 14086 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:19.768327  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:19.771366  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 45905.9us
I20260709 13:15:19.773379  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:19.773679 14356 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:19.774633 14356 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:19.775177 14356 raft_consensus.cc:697] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:19.776161 14356 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:19.783740  8056 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:19.785135 14329 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:19.787022  8029 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:19.788738 14086 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:19.789111  8730 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:15:19.789278 14086 raft_consensus.cc:697] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:19.790306 14086 consensus_queue.cc:237] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:19.795157 14379 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:19.790706 14252 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:19.755792  7959 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 6 to 14, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 14 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:19.814848  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 137272us
I20260709 13:15:19.819264  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:19.832372  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:19.854439  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:19.857241  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:19.857949  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:19.858490  8131 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:19.858736  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:19.859138  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:19.859301  8134 raft_consensus.cc:2376] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18. Re-sending same reply.
I20260709 13:15:19.861596  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:19.869755  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:19.870929  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 96634.1us
I20260709 13:15:19.852666  7959 catalog_manager.cc:5697] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 9 to 12, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 12 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.862932  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:19.817564 14244 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:19.881724  8026 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:19.883651  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:19.899286  8131 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:19.911070  8054 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:15:19.911141  8138 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:15:19.909087  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:19.880694 14332 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:15:19.909552  7959 catalog_manager.cc:5697] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:19.924979 14141 raft_consensus.cc:2764] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:15:19.902570 14256 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:19.920569 14365 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:19.955307  8204 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:19.912949 14381 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:19.966543  8212 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
W20260709 13:15:19.968955  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 68226.7us
I20260709 13:15:19.946769 14365 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:20.032938 14256 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:20.033704 14382 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:15:20.038887  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 189372us
I20260709 13:15:20.040482  8026 leader_election.cc:400] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:20.045555 14379 raft_consensus.cc:3055] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:20.045984 14379 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:20.046886 14379 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:20.048086 14379 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:20.052002  7959 catalog_manager.cc:5697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.054450 14243 consensus_queue.cc:1059] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 13:15:20.015158 14365 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:20.062475 14244 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:20.062721  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:20.063174 14244 raft_consensus.cc:3055] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:20.063609 14244 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:20.064640  8206 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:15:20.064486 14244 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:20.068521 14381 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:20.053574  7959 catalog_manager.cc:5697] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 11 to 15, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 15 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:20.075569  8204 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:15:20.076210  7959 catalog_manager.cc:5697] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 9 to 12, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 12 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:20.091018  7959 catalog_manager.cc:5697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 7 to 10, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 10 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.092883  7959 catalog_manager.cc:5697] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 9 to 12, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 12 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:20.100001  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 48938.8us
I20260709 13:15:20.103982 14365 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:20.104795  8026 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:20.115629 14254 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:20.117925 14134 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:15:20.119786  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 213582us
I20260709 13:15:20.126818 14380 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:20.127713 14314 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:20.121728 14256 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.120258 14365 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:20.140427  8026 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:20.166563  8207 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:20.154174  7959 catalog_manager.cc:5697] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 11 to 12, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 12 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:20.150588 14256 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:20.249471  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 343051us
W20260709 13:15:20.253753  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 343479us
I20260709 13:15:20.253835 14291 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:15:20.253856  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 86859us
I20260709 13:15:20.253043  8101 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:20.261482  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 353513us
I20260709 13:15:20.257532 14256 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:20.265686 14134 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:20.275957  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:20.278514 14291 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:15:20.285552  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 270014us
W20260709 13:15:20.291501  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 321632us
W20260709 13:15:20.291989  8104 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.290740  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:20.306704  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:20.307569  8102 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:20.312101 14311 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:20.334635  8204 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:15:20.335112 14244 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:20.344780  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:20.352522  8204 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:15:20.327961 13750 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:15:20.358173  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45351.1us
W20260709 13:15:20.359076  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 106130us
W20260709 13:15:20.365223  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 236756us
I20260709 13:15:20.367048 14425 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:20.367457  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:20.367776 14423 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:20.368381 14423 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:20.383409  8206 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:20.390956  8026 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:20.392395  8026 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:20.393653  8026 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:20.394654  7959 catalog_manager.cc:5697] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 8 to 11, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 11 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:20.398231 14432 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:20.399143 14432 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:20.386523  8212 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:20.405275 14429 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:20.381310 13750 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:20.405907 14429 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:20.407383 13750 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:20.399518 14432 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:20.407919 14429 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:20.407989 14432 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:20.409801 14434 raft_consensus.cc:2764] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:15:20.411046 14356 raft_consensus.cc:3055] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:20.411422 14356 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:20.412415 14356 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:20.413654 14356 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:20.414881 14423 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:20.417869  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:20.423766 13934 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:20.428411  8133 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:15:20.439062  8175 leader_election.cc:400] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 19. Current term is 20.
I20260709 13:15:20.439467  8175 leader_election.cc:403] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:15:20.438684  7959 catalog_manager.cc:5697] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 12 to 13, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 13 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:20.441004  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:20.441164 13934 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:20.442610 14365 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 19. Current term is 20.
I20260709 13:15:20.442677 14332 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:20.443864 14134 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.444278 14430 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:20.444482 14134 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:20.441012  7959 catalog_manager.cc:5697] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 7 to 9, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 9 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:20.465319 14426 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:20.480038  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:15:20.482213  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 88670.4us
W20260709 13:15:20.491511  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 111673us
I20260709 13:15:20.493672  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:20.496027  8131 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:20.496083  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:20.497180  8132 raft_consensus.cc:2393] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:20.496134  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:20.499317  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:20.500622  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:20.503940  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:20.504778  8056 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 16: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:20.506563  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:20.509096  8101 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:20.509389  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:20.517097  8059 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:15:20.456195 14382 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:15:20.530880  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:20.531595  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:20.538460  8134 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:15:20.544768  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:20.492187 13934 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:15:20.551888 13750 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.553839  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:20.564756 14134 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:20.557057 13750 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:20.500051 14371 consensus_queue.cc:1059] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:15:20.570647 14243 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.591475 14243 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:20.593538 14243 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:20.570211 14423 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:20.586319 14428 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:20.603138 14428 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:20.603514 14428 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:20.604368 14428 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:20.605487 14428 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:20.589224  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 57483.7us
W20260709 13:15:20.591001 14438 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:15:20.614075  8058 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:20.613977  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:15:20.624181  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 123981us
I20260709 13:15:20.625770  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:20.632755 13750 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:20.632140  8101 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:20.645000 14429 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:20.647248 14141 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.665130  8026 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:20.671556 14073 raft_consensus.cc:3055] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:20.672142 14073 raft_consensus.cc:740] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:20.674553 13934 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:20.673173 14073 consensus_queue.cc:260] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:20.675590 14073 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:20.676565  8026 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:20.679162 14344 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:20.691015 14444 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
W20260709 13:15:20.695797  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 46527.5us
I20260709 13:15:20.696599  8175 leader_election.cc:400] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:15:20.698778 14332 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:15:20.700325  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 89322.2us
W20260709 13:15:20.702884  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 78026.8us
I20260709 13:15:20.703742 14141 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:20.704113  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:20.705442  8101 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:20.705875 14243 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:15:20.710753 14398 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:20.678143  8026 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:20.713927  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:20.739225  8135 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:20.743621  8137 raft_consensus.cc:1240] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:15:20.710240 14347 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:20.744088  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:20.755342  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:20.781145  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 67860.7us
I20260709 13:15:20.705885 14141 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:20.781175  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:20.666153 14432 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:20.786624  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:20.743621 14380 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:20.787343  8054 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
W20260709 13:15:20.789469  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 111443us
I20260709 13:15:20.755436 14341 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:15:20.795112  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 54887.1us
I20260709 13:15:20.802802  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:20.803385 14459 log.cc:927] Time spent T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:15:20.794251 14243 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.804348 14243 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:20.755934 14434 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:15:20.806386 14243 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:20.806825 14356 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:20.743448 14252 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:15:20.811993  8062 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:20.817157  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:20.812302 14356 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:20.825156 14446 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:20.825665 14356 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:20.825764 14446 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:20.831888  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:20.810984 14347 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:20.817842  8175 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:20.861804  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:20.862650  8130 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:20.866780 14429 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:15:20.879940  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 62185.8us
I20260709 13:15:20.884286  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:20.888309  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:20.889191 14332 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:20.889770 14365 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:20.889883  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:20.890930 14332 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:20.891502  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.894460  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:20.903918 13934 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:20.905354 14446 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:20.924808  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:20.935665 14347 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:20.938858 14432 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:20.891907 14429 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:20.940682  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 83282.6us
I20260709 13:15:20.941534  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:20.946866  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:20.959721 14430 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:20.988525  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:20.989984  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 119897us
I20260709 13:15:20.990190 14379 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:21.018119 14379 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:21.020265 14379 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:20.990248 14073 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:21.021147  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:21.020931 14073 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:20.990746  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:15:21.023723  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 129361us
I20260709 13:15:21.025210  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:21.039027  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 48383.7us
I20260709 13:15:20.982093 14430 raft_consensus.cc:697] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:21.047717  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:21.088912  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 58313.2us
I20260709 13:15:21.060616 14472 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:21.107040  8056 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:21.107614  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:21.111729  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 51665us
I20260709 13:15:21.023058 14446 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:21.038417 14398 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.131076 14430 consensus_queue.cc:237] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:21.153329 14398 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:21.152385 14472 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:21.134174 14473 log.cc:927] Time spent T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:15:21.136023 14073 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:21.136744  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:21.139349 14380 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:21.054880 14365 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:21.141463 14440 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:15:21.119433 14485 log.cc:927] Time spent T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:15:21.118165  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:21.166180  8129 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:15:21.168440  8134 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:15:21.168460  8138 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:21.156019 14472 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:21.155964 14398 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:21.166059  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:21.174459  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:21.177448  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:21.178071  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:21.162130 14365 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:21.183393  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:21.186280  8101 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:21.200098  8696 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:21.181833 14365 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:21.203737  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:21.203312  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:15:21.215003  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 96956.4us
I20260709 13:15:21.215571  8136 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:21.184962 14475 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:21.193300 14398 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:21.175046  8137 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:21.219054  8137 raft_consensus.cc:606] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:21.170852 14440 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:21.220614  8101 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:21.220350  8696 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:21.210425 14446 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:21.232630 14470 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:21.232774 14479 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:21.219138 14398 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:21.224018 14379 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:21.268749  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 81161.3us
I20260709 13:15:21.272135  8058 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:21.268255 14470 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:21.250128 14398 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:21.274040 14332 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:15:21.280642 14440 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:21.285579  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:21.287303 14254 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:21.292100 14254 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:21.294951 14365 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:21.297453 14365 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:21.291359  8137 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:15:21.299237  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:21.299638  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:21.300591  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:21.301191  8136 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:21.306660  8205 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:21.311231 14481 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:21.312040 13934 raft_consensus.cc:3055] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:21.312386 13934 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:21.312862  8136 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:21.314388 14479 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:21.315492 14472 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:21.315891 14472 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:21.317051 14472 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:21.317654 14443 raft_consensus.cc:3055] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:21.319355 14472 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:21.318107 14443 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:21.327394 14443 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:21.325886  8101 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:21.338871  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:21.343626  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:21.328944 14443 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:21.352010  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:21.347970  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:21.349885 14488 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:21.355494 14474 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:21.356877 14254 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:21.361315  8130 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:21.361970  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:21.276619 14470 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:21.362793  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:21.353790 14344 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:15:21.367090  8026 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:21.369982  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:21.354878  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:21.348414 14347 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:21.377825  8062 raft_consensus.cc:2410] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:15:21.380247  8026 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:21.366231 14311 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:21.347100 14356 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:15:21.369767 14426 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:21.388217 14382 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:15:21.390787 14447 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:21.401778 14447 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:21.393034 14470 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:21.363656 14475 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:15:21.395465 13934 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:21.407164 13934 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:21.369922 14488 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:21.408926 14488 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:21.410233 14244 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:21.388634 14086 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:15:21.386835 14341 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:21.377239 14481 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:21.415311  8136 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:21.450150  8058 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:15:21.451229  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 81334.7us
I20260709 13:15:21.455688  8026 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:21.457418  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:21.462013  8204 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:21.448410 14474 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:21.463757  8062 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:21.469611 14472 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:21.469831 14086 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:21.470109  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:21.461277 14429 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:15:21.476661 14314 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:21.478374 14465 raft_consensus.cc:993] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:21.478826 14465 raft_consensus.cc:1081] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:21.465654 14474 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:21.484661 14474 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:21.476584 14447 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:21.487887 14398 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:21.491858 14398 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:21.497903  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:21.499306  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:21.500416  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:21.500790  8696 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:21.470136 14472 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:21.504802 14379 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:21.506233 14472 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:21.522863  8129 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:21.526703  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:21.528782  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:21.531319 14472 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:21.531878 14472 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:21.533908 14472 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:21.537460  8136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b4358ace86545339036f21610735b47"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:21.538017  8136 raft_consensus.cc:606] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:21.538362  8136 raft_consensus.cc:614] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:21.547633  8104 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:21.549975  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:21.550629  8207 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:21.556318  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:21.562644  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:21.570268  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:21.579015 14470 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:21.579370 14494 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:21.580945  8060 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49222
I20260709 13:15:21.581542  8060 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:21.581868  8060 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:21.559453 14430 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:15:21.584663 14430 raft_consensus.cc:3055] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:21.584738  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:21.585078 14430 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:21.583938 14493 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:21.586740  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:21.593796  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:21.590579  8060 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:21.613807  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:21.636026  8060 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:21.645699  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:21.646216  8211 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:21.585872 14430 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:21.661808  8132 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:21.670059  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:21.630180 14425 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:15:21.675812 14430 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:21.713066 14398 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:21.735976  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:21.742800 14501 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:15:21.773329 14500 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:15:21.841465 14471 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:21.853004 14379 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.853682 14379 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:21.862656 14254 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:21.856335 14379 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:21.931324  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:21.914953 14499 log.cc:927] Time spent T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:15:21.938045  8204 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:21.939467  8061 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:21.951411  8054 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:21.930365 14380 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.957472  8029 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:21.842157 14471 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:21.967619 13750 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:21.962364 14481 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:21.960223 14380 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:22.025282  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 355272us
I20260709 13:15:22.026005  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:22.027195  8211 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:22.034541 14423 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:22.040172 14423 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:22.042384  8209 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:22.015364 14332 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:15:22.017068 14428 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:15:22.047592  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:22.046765  8101 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:22.057855  8135 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 15.
I20260709 13:15:22.045301 14254 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:15:22.035388 14493 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:22.072532 14423 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:22.076885 14380 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:22.061844 14465 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:22.061664  8101 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:22.082101  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:22.098629 14073 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:22.099062 14073 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:22.099387 14073 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:22.100869  8029 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:22.101541 13934 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:22.101945 13934 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:22.102705 13934 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:22.119464  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:22.121354 14471 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:22.121665  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:22.087983 14493 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:22.122395  8057 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 16: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:22.120134  8062 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:22.130275 14493 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:22.131309  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:22.097568 14423 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:22.141109  8137 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:22.141391  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:22.141710  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:22.143424  8061 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:22.144097 14428 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:22.147855 14428 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:22.148198 14428 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:22.154129  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:22.154760  8130 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:22.155244  8130 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:22.154680  8137 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:22.156054  8137 raft_consensus.cc:606] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:22.164678  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:22.171217 13750 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:22.165500  8133 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:22.198107  8130 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:22.205001  8130 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:22.210611  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:22.211316  8203 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 16: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:22.211339  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:22.217243  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:22.215219  8206 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:22.241228 14073 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:22.248929 14428 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:22.266067  8721 raft_consensus.cc:2368] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17.
I20260709 13:15:22.266260  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:15:22.270121  8104 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:22.279140 14073 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:22.286865 14479 raft_consensus.cc:993] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:22.287379 14425 raft_consensus.cc:1081] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:22.287074 14428 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:22.291373  8102 leader_election.cc:400] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17.
I20260709 13:15:22.291692  8102 leader_election.cc:403] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:22.293128 14471 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:15:22.294080  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:22.299672  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:22.300268  8207 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:22.304183 14425 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:22.304292  8104 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:22.305328 14516 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:22.305708 14516 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:22.306046 14516 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:22.308370  8130 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:22.309165  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:22.309748  8138 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:15:22.313411  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:22.324520  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:22.325193  8055 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:22.327039  8206 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59868
I20260709 13:15:22.327227 14471 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17.
I20260709 13:15:22.327610  8206 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:22.327997  8206 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:22.332156 14379 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.332731 14379 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:22.335170 14379 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:22.336526  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:22.337216  8134 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:15:22.338476  8693 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:22.339285 14379 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:22.339658 14379 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:22.339982 14379 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:22.343582  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:22.344321  8211 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:15:22.348579  7961 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 14 to 17, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 17 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:22.350172  7961 catalog_manager.cc:5697] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 8 to 9, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 9 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:22.351691  7961 catalog_manager.cc:5697] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:22.352870  7961 catalog_manager.cc:5697] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 17 to 18, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 18 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:22.354559  8206 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:22.361416  8207 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:22.362432  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:22.363660 14440 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:22.364323 14440 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:22.364737  8055 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:22.365164 14440 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:22.373304  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:22.374055  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:22.375226  8138 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:22.380487  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:22.381053  8134 raft_consensus.cc:3055] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:22.381332  8134 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:22.356729  8206 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:22.387316  7961 catalog_manager.cc:5697] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 9 to 16, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 16 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:22.389569  8055 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:22.389498  7961 catalog_manager.cc:5697] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 18 to 20, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 20 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:22.392900  7961 catalog_manager.cc:5697] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 16 to 20, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 20 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:22.405092  8132 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:22.379729 14380 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:22.393445  8134 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:22.412298  8134 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:22.421742  8134 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:22.424592  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:22.403088  7961 catalog_manager.cc:5697] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 3 to 11, term changed from 15 to 17, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67), VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted, no longer has a pending config: VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66), VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67), VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 17 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:22.422902 14380 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:22.438506  7961 catalog_manager.cc:5697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 20 to 21, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 21 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:22.431031 14428 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:15:22.439105  7959 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34120 (request call id 249) took 30960 ms (client timeout 15000 ms). Trace:
I20260709 13:15:22.441520  7961 catalog_manager.cc:5697] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 11 to 12, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 12 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:22.443679  7961 catalog_manager.cc:5697] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:22.445384  7961 catalog_manager.cc:5697] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 14 to 16, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 16 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:22.446740  7961 catalog_manager.cc:5697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:22.445694 14428 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:22.452512 14428 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:22.456506 14516 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:22.460033 14380 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:22.465245  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:22.465971  8062 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 16: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:22.471446  7961 catalog_manager.cc:5697] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:22.474943 14516 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:22.475845  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:22.476307 14379 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:22.441315  7959 rpcz_store.cc:269] 0709 13:14:51.478751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.654431 (+175680us) service_pool.cc:224] Handling call
0709 13:15:22.439016 (+30784585us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:20.838711 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:20.839343 (+   632us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:20.839388 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:15:20.839405 (+    17us) tablet.cc:662] Decoding operations
0709 13:15:20.840670 (+  1265us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:20.840758 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:20.840795 (+    37us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:15:20.918525 (+ 77730us) tablet.cc:701] Row locks acquired
0709 13:15:20.918580 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:15:20.918777 (+   197us) write_op.cc:270] Start()
0709 13:15:20.918955 (+   178us) write_op.cc:276] Timestamp: P: 1783602920918706 usec, L: 0
0709 13:15:20.918996 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:15:20.919835 (+   839us) log.cc:844] Serialized 5154 byte log entry
0709 13:15:20.947651 (+ 27816us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6db7f0c0ac 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f28c02 00007f6dbabb020b
0709 13:15:21.269806 (+322155us) op_driver.cc:464] REPLICATION: finished
0709 13:15:21.436410 (+166604us) write_op.cc:301] APPLY: starting
0709 13:15:21.436522 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:21.446049 (+  9527us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:21.446085 (+    36us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:21.494133 (+ 48048us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:21.494415 (+   282us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 13:15:21.494482 (+    67us) write_op.cc:312] APPLY: finished
0709 13:15:21.494933 (+   451us) log.cc:844] Serialized 121 byte log entry
0709 13:15:21.496308 (+  1375us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:21.496458 (+   150us) write_op.cc:454] Released schema lock
0709 13:15:21.496525 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:21.496942 (+   417us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:21.496981 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":647,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":160377,"apply.run_cpu_time_us":16656,"apply.run_wall_time_us":64859,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58467,"num_ops":13,"prepare.queue_time_us":154864,"prepare.run_cpu_time_us":34634,"prepare.run_wall_time_us":338357,"raft.queue_time_us":32918,"raft.run_cpu_time_us":20983,"raft.run_wall_time_us":104949,"replication_time_us":350648,"spinlock_wait_cycles":38765440,"thread_start_us":419390,"threads_started":4,"wal-append.queue_time_us":302763}]]}
I20260709 13:15:22.479255  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:22.480014  8207 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:22.484215 14379 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:22.486155  8206 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:22.488950  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:22.494331  8055 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:22.493148  8134 raft_consensus.cc:3060] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:22.493223  8104 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:22.496757  8207 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:22.497841  8696 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:22.507341 14516 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:22.508942 14516 raft_consensus.cc:697] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:22.509755 14516 consensus_queue.cc:237] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:22.506009 14379 raft_consensus.cc:2804] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:22.516604 14341 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.520623 14379 raft_consensus.cc:697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:22.521154 14341 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:22.521733 14379 consensus_queue.cc:237] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:22.523926 14341 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:22.525221  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:22.528860  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:22.532859 14291 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:22.533329 14291 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:22.535722  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:22.536405  8062 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 17: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:22.540746  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:22.541319  8135 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:22.542435  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:22.543362  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:22.546141  8134 raft_consensus.cc:2468] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:22.543612  8029 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:22.550067 14474 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:22.562104  8206 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:22.548264 14291 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:22.548503 14073 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:22.568289 14475 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:15:22.571062 14073 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:22.585840 14073 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:22.600006  8134 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:22.617125 13934 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:22.691651 14073 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:22.693836 14073 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:22.695489  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:22.696815  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:22.697497  8062 raft_consensus.cc:2393] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 17: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:22.708585  8138 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:22.709565  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:22.711584 14073 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:22.712980 14073 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:22.713831 14073 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:22.720130 14380 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.720633 14380 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:22.727394  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:22.731280 14380 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:22.733253  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:22.734033  8134 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:22.735445  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:22.736327 14380 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:22.736919 14380 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:22.737526 14380 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:22.797749 14380 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:22.807668  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:22.808198  8134 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:22.808091  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:22.809671 14380 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:22.821308  8135 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:15:22.827193  8134 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 21.
I20260709 13:15:22.828752  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:22.830369 14073 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:15:22.843297 14380 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:22.847261 14380 raft_consensus.cc:697] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 21 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:22.850455 14380 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:22.908447 14474 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.908975 14474 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:22.911166 14474 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:22.912847  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:23.365295  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:23.731702 14542 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:23.733382 14542 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:23.735471 14542 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:23.773407  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:23.774111  8136 raft_consensus.cc:2368] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 20. Current term is 21.
I20260709 13:15:23.810933  8026 leader_election.cc:400] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 20. Current term is 21.
W20260709 13:15:23.931546  7992 debug-util.cc:398] Leaking SignalData structure 0x7b08001b7e20 after lost signal to thread 6940
I20260709 13:15:24.133363 14440 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.180778 14440 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.166530 14506 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.182971 14440 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.183511 14440 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.183420 14506 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.176865 14474 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.187583 14474 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.191526 14474 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.179010 14542 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.181201 14379 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.411726 14379 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.414021 14379 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.184206 14440 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.416105 14440 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.416646 14440 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.417145 14440 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.234150 14543 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.418641 14543 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.419214 14554 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.419950 14554 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.421057 14543 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.174193 14471 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.421859 14471 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.424134 14471 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.424670 14471 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:24.425216 14471 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.304971 14314 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:15:24.426637  8027 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.166s	user 0.013s	sys 0.016s
W20260709 13:15:24.427012  8027 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.167s	user 0.013s	sys 0.016s
I20260709 13:15:24.426858 14314 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.429056 14314 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.377338  8055 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 1043) took 1493 ms. Trace:
I20260709 13:15:24.145967  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49222 (request call id 3584) took 1262 ms. Trace:
I20260709 13:15:24.404662 14551 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:24.430663 14551 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.433001 14551 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.434417  8055 rpcz_store.cc:276] 0709 13:15:22.883679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.376548 (+1492869us) service_pool.cc:224] Handling call
0709 13:15:24.376879 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.377044 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.377069 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.377104 (+    35us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:24.377131 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.377179 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.377284 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.429806  8062 rpcz_store.cc:276] 0709 13:15:22.884073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.144792 (+1260719us) service_pool.cc:224] Handling call
0709 13:15:24.145182 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.145586 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.145615 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.145663 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:24.145703 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.145760 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.145911 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.436399 14314 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.436826 14545 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.436957 14314 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.437366 14545 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.437819 14440 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.439383 14314 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.439739 14429 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.440529 14429 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.441380 14471 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.442442 14479 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.442999 14516 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.443358 14471 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.443776 14516 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.443924 14471 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.177481 14549 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.408854 14550 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.446012 14516 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.446497 14550 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.447600 13750 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.448171 13750 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.448712 14550 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.450968 14443 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.451514 14443 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.452330 14542 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.453529 14443 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.454049 14443 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.168895 14428 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.439921 14551 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.442914 14479 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.450645 14549 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.430686 14554 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.442461 14440 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.455677 14542 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.180536  8026 leader_election.cc:403] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:24.408910 14506 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.446034 14471 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.456825 14332 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.407590 14552 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.471763 14314 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.472010 14429 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.472160 14552 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.472304 14314 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.473887 14341 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:24.474669 14552 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.474512 14341 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.475124 14545 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.476400 14475 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:15:24.476480 14471 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.462867 14443 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.477182 14471 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.463024 14428 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.479177 14443 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.479441 14471 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.479803 14543 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.480365 14543 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.409734 14548 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:15:24.468317  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 657536us
I20260709 13:15:24.469872 14332 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.450238 13750 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.485088  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.481289 14550 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.465101 14516 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.494215 14548 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.498620 14086 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.499563 14543 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.469101 14474 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.464211 14551 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.500098 14379 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.480381 14428 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.480168 14471 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.483836 14341 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.604130  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.462561 14440 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.605882  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.603693 14086 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.610291 14086 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.495038 14314 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.509548 14543 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.631680 14558 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.632356 14558 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.638897 14558 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.639477 14558 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.639902 14558 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.641405  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.642560  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:24.642871 14558 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.643143  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:24.643356 14558 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.642618  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:24.643975  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.645862  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.505565 14550 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.618932 14545 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.506495 14516 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.617674 14314 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.619158 14428 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:24.511117 14474 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.657064  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.637328 14562 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.664515  8130 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:24.665573 14086 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.666115 14556 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.665717 14562 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.666385 14086 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.666786 14556 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.668286 14562 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.669121 14086 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.669785 14086 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:24.673514 14560 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.674513 14554 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.675058 14554 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.677313 14548 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.677371 14542 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:24.678143 14549 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.613674 14471 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.605911 14379 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.689450 14551 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.628579 14552 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:24.643853 14558 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.696184  8062 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:15:24.687527 14545 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.696794 14440 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.697784  8055 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:15:24.630360 14543 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.699910  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.678211 14542 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.669135 14556 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.674103 14560 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.711983  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.712301 14542 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.712457 14560 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.713004 14560 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.713532 14560 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.701426 14557 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.714735 14557 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.715638  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.716377 14475 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 20. Current term is 21.
I20260709 13:15:24.689463 14314 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.717098 14557 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.719039 14314 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.719235 14560 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.719484  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.719825 14560 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.720232  8204 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:24.720438 14560 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.736766  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.737411  8131 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:24.742522 14556 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:24.743204 14556 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.743759 14560 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.703267  8721 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:15:24.703400 14550 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.703675 14558 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.703819 14553 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.745234 14553 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.702241 14516 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.746817  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.748215 14553 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.748638 14556 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.704182  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.703603  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.699502  8724 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:24.702766 14559 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.706708  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.755385  8207 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
I20260709 13:15:24.705987  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.708178 14479 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.708281 14141 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.708748  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.709978  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.763152  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.720973 14379 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.768731  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.763897  8131 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:24.769385  8136 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:24.721692 14425 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.722611 14471 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.691582 14428 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.782727  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.697679 14552 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.723758  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.725499 14543 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.726171  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.687510 14341 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.727243  8206 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:15:24.728379 14545 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.728523 14554 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.728960  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.729213  8055 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:15:24.720542 14332 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.730849  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:24.705925  8062 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:24.797601  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.758648  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.760299  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.799288  8061 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:24.799562  8054 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:24.800211  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.800438  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.800791  8061 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:24.761363  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.803663 14341 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.804826  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.805637 14341 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.805807  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.805878 14554 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.710263 14474 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.768096  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:24.807652  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 79355.9us
I20260709 13:15:24.770036  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.809289  8132 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:24.770735  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.702734  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.809938  8059 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:24.759833 14443 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.775554  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.775556  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.775906  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.812693  8138 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:24.758239 14559 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.776670 14430 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.777427 14506 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.779127  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2601) took 1009 ms. Trace:
I20260709 13:15:24.772970 14141 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.780553 14425 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.790917 14428 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.698177 14086 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:24.797633  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 151988us
I20260709 13:15:24.807222  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.808328  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.829809  8056 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:24.807957 14332 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.809545  8104 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:24.810846  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.754382 14314 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.824339 14430 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.812867  8063 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:24.795725 14549 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.824338  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.830699  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.823413  8135 rpcz_store.cc:276] 0709 13:15:23.769784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.778094 (+1008310us) service_pool.cc:224] Handling call
0709 13:15:24.778471 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.778696 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.778736 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.778789 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:24.778853 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.778931 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.779059 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.826818  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:24.807446  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.831207 14506 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.834764  8062 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
W20260709 13:15:24.838771  7992 debug-util.cc:398] Leaking SignalData structure 0x7b080034dce0 after lost signal to thread 7995
I20260709 13:15:24.840379  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.833681  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.842319  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:24.854967  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.856859  8723 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:24.839119 14430 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.833921 14314 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.837119  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.839803 14549 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.841827 14567 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:24.843370 14556 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.835868 14141 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.824180 14443 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.807914 14554 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.843083 14545 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:24.886703 14545 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.861377 14073 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.888729 14545 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.888903 14073 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.890233  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.891309  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.892248  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.892407 14073 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.861200 14558 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.864537  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:24.893993 14558 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.894624  8726 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:24.895548  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.896186 14558 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.861641 14508 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.846191 14475 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:24.896608  8133 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:15:24.896420 14567 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.896842 14508 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.871150  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.848492  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:24.862066 14428 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.898532 14567 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.876362  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:24.841603 14552 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.863235  8102 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:24.866732 14086 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.847978 14506 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.880848  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:24.880976 14561 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:15:24.881505  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 72037.1us
I20260709 13:15:24.882162 14546 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:24.883981 14443 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.884258  8130 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:15:24.884830  7992 debug-util.cc:398] Leaking SignalData structure 0x7b080033e820 after lost signal to thread 8087
I20260709 13:15:24.885159  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 1045) took 1980 ms. Trace:
I20260709 13:15:24.886051  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.894299  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.864800  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.897164  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.878413 14556 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.844879 14425 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.934129  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.932116  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:15:24.930866  7992 debug-util.cc:398] Leaking SignalData structure 0x7b0800274b60 after lost signal to thread 8088
I20260709 13:15:24.872709  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:24.932917  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 84512.6us
I20260709 13:15:24.936005  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.832882 14564 raft_consensus.cc:493] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:24.942042  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:24.942643  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.933463  8063 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
I20260709 13:15:24.932060  8728 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 21.
I20260709 13:15:24.936787  8026 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:15:24.939082  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 75916.1us
I20260709 13:15:24.930914  8058 rpcz_store.cc:276] 0709 13:15:22.904450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.884497 (+1980047us) service_pool.cc:224] Handling call
0709 13:15:24.884733 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.884874 (+   141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:24.884898 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.884932 (+    34us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:15:24.884971 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.885018 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.885113 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:24.942049  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.863204 14488 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.873378 14291 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.956947  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:24.951320  8026 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:24.955564  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.962908  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:24.898813 14141 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:24.854135 14332 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.955286 14252 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:24.968997 14141 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.976971 14252 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:24.898607 14428 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.978461  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.977569  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.979146 14516 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:24.979535 14516 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.979569  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.980003 14516 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:24.980667  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.981731  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.982740  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.983439  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.983726  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.984429  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.968482 14382 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:24.985173 14382 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.905675 14314 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.913178 14559 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.907591 14568 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.937135 14561 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.969960  8130 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:15:24.986723  8138 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:15:24.985513  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.970288 14488 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.987823 14561 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:24.988687  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.986135 14568 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:24.945901 14425 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:24.991539  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.991688 14568 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.992096  8212 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:15:24.991712 14425 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.992646 14425 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:24.992910  8131 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:15:24.878774 14565 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:24.993541 14565 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:24.994328 14332 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.901608 14549 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:24.996716  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.966068  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.970503  8026 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:24.969132 14564 raft_consensus.cc:515] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.002434  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.004930  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.006021  8209 raft_consensus.cc:2410] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 20 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:15:24.977041 14141 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:24.984522  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:24.985512 14382 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:24.997782  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:24.934916 14546 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:24.995977  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.977504 14252 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:25.009754  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.011276  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.006834 14559 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:25.014487  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.001503  8026 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:24.980708 14428 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:24.965641 14086 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:24.896843 14475 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:25.021374  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:25.021878  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:24.973616  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.029649  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.023487  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.024964  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.017930  8176 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:25.027433  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:24.968791 14291 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.007478 14479 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:15:25.030174  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:25.243112  7992 debug-util.cc:398] Leaking SignalData structure 0x7b0800038820 after lost signal to thread 8162
I20260709 13:15:25.117767  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.022182 14546 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:25.244349  7992 debug-util.cc:398] Leaking SignalData structure 0x7b0800335780 after lost signal to thread 8236
W20260709 13:15:25.244630  7992 debug-util.cc:398] Leaking SignalData structure 0x7b0800374120 after lost signal to thread 8237
W20260709 13:15:25.245714  7992 debug-util.cc:398] Leaking SignalData structure 0x7b0800265880 after lost signal to thread 8754
W20260709 13:15:25.245994  7992 debug-util.cc:398] Leaking SignalData structure 0x7b0800307fa0 after lost signal to thread 8755
I20260709 13:15:25.051966  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:15:25.246613  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 228768us
I20260709 13:15:25.246919 14428 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:15:25.247153  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 195278us
I20260709 13:15:25.247366 14428 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:25.247470 14398 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:25.024830 14086 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:25.247895 14428 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:25.247987 14398 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:25.248457 14086 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:25.248603 14398 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:25.087368  8134 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.244s	user 0.020s	sys 0.013s
W20260709 13:15:25.250211  8134 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.407s	user 0.020s	sys 0.021s
I20260709 13:15:25.250722 14566 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:25.219560 14568 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:15:25.251384 14566 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.215226  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:25.042315 14488 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:25.032747 14475 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:25.255425 14566 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.191700 14516 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.256603 14398 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.165571 14141 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.208467  8728 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.023685 14559 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:25.259958 14559 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:25.260432  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.027601 14314 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:25.051051  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.261291 14314 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:25.261706 14314 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:25.052384  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.105980  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.079078  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.207628  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.264019  8060 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
I20260709 13:15:25.264067  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.264963  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.265069  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.265439  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.105512 14508 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:25.217849 14564 leader_election.cc:290] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.217330  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:25.051633  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:15:25.267750  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 216170us
I20260709 13:15:25.127513 14425 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.269485  8102 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.269734 14425 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.208923  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.270885 14382 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.079561  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.271965  8062 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:25.069903 14565 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.273639  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.273775 14565 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:25.271435  8102 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:25.274315 14565 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:25.088696  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.274936 14565 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:25.275781 14550 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:25.276038  8055 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:15:25.276180 14550 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:25.024227 14561 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.277199 14561 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.279299 14561 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:25.067461  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.257404 14428 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.280422  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.219986  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:15:25.281862  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 61981.4us
I20260709 13:15:25.282231 14428 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.175175 14252 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.282862 14428 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:25.283191  8132 raft_consensus.cc:1240] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:25.283528  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.283650  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:25.285379 14488 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:25.285753 14488 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.286095 14488 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:25.286640  8134 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 992) took 2450 ms. Trace:
I20260709 13:15:25.287034  8134 rpcz_store.cc:276] 0709 13:15:22.836374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.840984 (+  4610us) service_pool.cc:224] Handling call
0709 13:15:22.841263 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:22.841948 (+   685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.841984 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:22.842579 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 13:15:25.250578 (+2407999us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:15:25.250617 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.286389 (+ 35772us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:25.286440 (+    51us) raft_consensus.cc:1685] finished
0709 13:15:25.286451 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:25.286566 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":2407547,"threads_started":1,"wal-append.queue_time_us":95697,"wal-append.run_cpu_time_us":1871,"wal-append.run_wall_time_us":1002065}
I20260709 13:15:25.219875 14571 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:15:25.288662 14556 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:25.289559  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:25.262230  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.290457 14252 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.291960  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2609) took 1519 ms. Trace:
I20260709 13:15:25.292040  8207 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:15:25.292394  8137 rpcz_store.cc:276] 0709 13:15:23.772136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.736463 (+964327us) service_pool.cc:224] Handling call
0709 13:15:25.105095 (+368632us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b58001c3e98. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1eee 0000000000375939 00007f6dbff651e0 00007f6dc002e0c4 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:15:25.289923 (+184828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:25.290082 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:25.290104 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:25.290138 (+    34us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:25.290172 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.290222 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:25.291806 (+  1584us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b5400130e50. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:25.291907 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428461568}
I20260709 13:15:25.040370  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.247853  8101 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.294836  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.295166  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.103060  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.295938  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.272866 14382 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:25.297256 14508 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:25.297804 14554 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:25.297864 14508 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:25.298611 14508 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:25.307396  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.283219 14428 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:25.297148  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:25.302495  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:25.302992  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:25.311038  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 63261us
I20260709 13:15:25.312045  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.258775 14516 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.305001  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.049558  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.313505  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.313632  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.285980  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.266001  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.276738 14550 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:25.306687  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.315752  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.315925  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.299170  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.308486  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.301712  8027 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.307386  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.317070  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.269557 14291 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:25.317795  8061 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:25.318569  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.319836 14542 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:25.320470 14553 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:25.320801  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:25.320859 14553 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:25.321457 14568 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:25.321561 14553 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:25.322170 14428 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:25.323719  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:25.324819 14568 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:25.325181 14568 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:25.325532 14568 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:25.325568  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.327796 14479 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:25.328356 14479 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.330602 14252 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:25.331143 14252 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.332023 14554 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:25.333160 14252 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.334491  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:25.335238  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.335783  8212 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:25.335960 14479 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.336601  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.337471  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:25.260690 14141 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.338963  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.340117  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.340588  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.341631  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.341840  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:25.345268 14568 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.345762 14508 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.331135 14550 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.345528  8176 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.347430 14568 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:25.349728 14546 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.297693 14559 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.352447 14553 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.356475 14565 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.359062 14398 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.401157 14332 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:25.361945 14425 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:25.402455 14425 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:25.402880 14425 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:25.403375  8062 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:25.403858 14559 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:25.362293  8101 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.362701 14440 consensus_queue.cc:1059] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:15:25.404801 14440 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:25.405131 14440 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:25.405342  8101 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.405804 14561 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:15:25.363914  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 55038.5us
I20260709 13:15:25.406258 14561 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:25.402464 14553 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:25.406806  8104 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.406580 14479 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:25.407881 14565 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.365038  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.366932  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:25.367026  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:25.408936  8104 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.368134  8138 raft_consensus.cc:1275] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:15:25.368402  8212 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:15:25.370069  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.410779 14398 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:25.405952 14440 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.411095  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.411825 14440 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:25.412231  8730 raft_consensus.cc:1240] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:25.371821  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.415648 14546 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:25.416844 14479 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:25.417443  8101 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.372941  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:25.372892 14562 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:25.418329 14566 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:25.418715 14566 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.419054 14566 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:25.375754  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.419752  8101 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.378876  8203 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:25.380910  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.385288  7961 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37774 (request call id 144) took 3972 ms. Trace:
I20260709 13:15:25.420821  7961 rpcz_store.cc:276] 0709 13:15:21.412930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.413074 (+   144us) service_pool.cc:224] Handling call
0709 13:15:25.385244 (+3972170us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:22.655913 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:22.678964 (+ 23051us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:22.678997 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:15:22.679014 (+    17us) tablet.cc:662] Decoding operations
0709 13:15:22.680128 (+  1114us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:22.680181 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:22.680207 (+    26us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:15:22.681306 (+  1099us) tablet.cc:701] Row locks acquired
0709 13:15:22.681320 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:15:22.681451 (+   131us) write_op.cc:270] Start()
0709 13:15:22.681558 (+   107us) write_op.cc:276] Timestamp: P: 1783602922681406 usec, L: 0
0709 13:15:22.681575 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:15:22.685067 (+  3492us) log.cc:844] Serialized 5470 byte log entry
0709 13:15:22.937656 (+252589us) op_driver.cc:464] REPLICATION: finished
0709 13:15:22.939970 (+  2314us) write_op.cc:301] APPLY: starting
0709 13:15:22.940038 (+    68us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:22.947426 (+  7388us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:22.947543 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:22.958884 (+ 11341us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:22.959251 (+   367us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=0,mrs_lookups=0
0709 13:15:22.959326 (+    75us) write_op.cc:312] APPLY: finished
0709 13:15:22.959776 (+   450us) log.cc:844] Serialized 129 byte log entry
0709 13:15:22.960679 (+   903us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:22.960874 (+   195us) write_op.cc:454] Released schema lock
0709 13:15:22.960941 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:23.078722 (+117781us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:23.078770 (+    48us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":66194,"reported_tablets":17,"spinlock_wait_cycles":100736,"child_traces":[["op",{"apply.queue_time_us":2113,"apply.run_cpu_time_us":18359,"apply.run_wall_time_us":138674,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62647,"num_ops":14,"prepare.queue_time_us":102291,"prepare.run_cpu_time_us":10857,"prepare.run_wall_time_us":92459,"raft.queue_time_us":107313,"raft.run_cpu_time_us":1076,"raft.run_wall_time_us":3182,"replication_time_us":255918,"spinlock_wait_cycles":25856,"thread_start_us":188805,"threads_started":4,"wal-append.queue_time_us":65169,"wal-append.run_cpu_time_us":48217,"wal-append.run_wall_time_us":1208060}]]}
I20260709 13:15:25.425133 14425 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.425599 14398 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:25.428639 14566 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.389145  7956 catalog_manager.cc:5697] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 10 to 14, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 14 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:25.390973 14428 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.386106  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:25.429945 14562 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:25.400372  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 54920.2us
I20260709 13:15:25.430963  8696 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.432102  8176 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.432317  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.401542 14550 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.360756  8029 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:25.439687  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 78987.5us
I20260709 13:15:25.440640  8029 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.406653 14561 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:25.409571  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.443740  8207 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:25.444738  8207 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:25.366412  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:25.430661  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 44648.8us
I20260709 13:15:25.394232  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.428915  8101 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.430029 14332 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:25.431448  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:25.448359  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.436931  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.437394  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.438405  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.451467  8696 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.445696  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:25.446914 13934 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:25.456875  8101 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.458950  8696 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.459800  8693 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.462945 14567 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:25.464608 14549 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:25.466464 14073 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:25.467101 14073 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:25.432540 14086 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.460780  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.470305  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.470970 14382 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:25.471303 14382 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:25.471639 14382 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:25.471906 14566 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:25.438091 14568 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:25.472505 14475 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.474373  8104 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.476197  8696 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.445425 14465 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:25.478493  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:25.479079  7956 catalog_manager.cc:5697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 10 to 14, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 14 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:25.479725  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:25.479379  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:25.447906 14559 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:25.429474 14488 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.481369  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:25.485036  8054 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:25.481483  8029 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.482519  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:25.419018 14314 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.489710  8029 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.481492  8129 raft_consensus.cc:2410] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:15:25.454811 14548 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:25.490183  7956 catalog_manager.cc:5697] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 12 to 15, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 15 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:25.486016  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:25.490651 13934 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:25.494175 14488 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:25.494756  8204 raft_consensus.cc:2393] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 17: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:25.495079  8057 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:25.495265 14508 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:25.480228 14252 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:25.476953 14341 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:25.514776  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:15:25.509287  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 52450.5us
I20260709 13:15:25.520961  8101 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.501652  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:25.514323  8029 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:25.518996  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44719.3us
I20260709 13:15:25.518713  7956 catalog_manager.cc:5697] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 13 to 17, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 17 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:25.477586 14379 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:25.527017  8175 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:25.522910 14332 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:25.489616 14440 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:25.532019  8209 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:15:25.514488 14555 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:25.489821 14428 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:25.523414 14544 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:15:25.538115  8101 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.522655 14431 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:25.484740 14086 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:25.543072  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 66949.3us
I20260709 13:15:25.533227 14508 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:25.537376  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:25.538957 14488 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:25.528052 14554 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:25.544529  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:25.488586 14398 consensus_queue.cc:1059] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:15:25.550475 14553 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:25.540417 14465 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:15:25.563261  8026 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.564658  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.565231  8062 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 21.
I20260709 13:15:25.565311  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.566313  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b186be823643475982e269b73d886a42" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.569510  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.572322  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:25.573449  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:25.542815  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.483810 14570 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:15:25.547099 14477 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:25.518175 14566 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:25.583084 14566 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.584077  8137 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:25.585016 14566 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:25.539016 14252 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:25.590494 14252 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.590862 14252 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:25.522970 14565 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:25.536389 14314 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.577025 14488 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:25.577179 14554 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:25.602442 14542 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:25.603531 14475 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:25.581430 14425 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:25.548072 14555 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.608008 14555 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:25.611472 14382 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.539228  7956 catalog_manager.cc:5697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:25.554023  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:25.554778  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 95046.7us
I20260709 13:15:25.613482  7956 catalog_manager.cc:5697] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:25.614187  8693 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.615072  7956 catalog_manager.cc:5697] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 12 to 16, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 16 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:25.615875  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:25.617018  8693 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.617743 14570 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:25.617326  7956 catalog_manager.cc:5697] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 11 to 15, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 15 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:25.618523 14570 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:25.618832 14574 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:25.619017 14551 raft_consensus.cc:3055] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:25.619395 14551 raft_consensus.cc:740] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:25.618101  8693 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.620149 14551 consensus_queue.cc:260] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.620748 14508 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:25.622215 14508 consensus_queue.cc:1059] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:25.623265 14440 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:25.623606 14440 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:25.624372 14440 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.625792 14440 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:15:25.626822  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 72841.1us
I20260709 13:15:25.627539  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.628331 14557 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:25.628953 14557 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:25.629279 14557 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:25.629624 14557 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:25.629554  8029 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.630584 13934 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:25.630774  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 88006.6us
I20260709 13:15:25.631850  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:25.632688  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:25.633472 14479 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:25.633688  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:25.633883 14479 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:25.634289 14479 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:25.635406 14382 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.635540  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:25.636264  8129 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 20: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:25.638789  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.575330  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.556304 14428 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:25.582542  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:25.640097  8059 raft_consensus.cc:2393] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 17: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:25.641000 14546 consensus_queue.cc:1059] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:25.641541  8693 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.642490 14575 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:25.643263 14428 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:25.630143 14252 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.643829  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:15:25.644052  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 68777.8us
I20260709 13:15:25.644384  8136 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:25.645143 14252 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:25.647207  8138 raft_consensus.cc:1240] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:25.641014 14379 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:25.649214  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:25.626832 14551 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:25.652593  8129 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:25.648579  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:25.653998  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.654466  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:25.655664 14565 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:25.655738 14554 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:25.656143 14565 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:25.642669  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:25.657087 14565 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.658710 14379 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:25.658797 14565 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:25.661074 14479 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.663300  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:25.664317  8136 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:25.664381  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:25.667881  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:25.669217  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.669899  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.670693  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:25.671619  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:25.670766  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.674458 13934 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:25.676736  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.678211  8059 raft_consensus.cc:2393] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 18: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:25.678867  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:25.686877  8138 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:25.688273  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:25.689152  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:25.686667 14554 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:15:25.691475  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.691879 14567 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:25.693089  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:25.689539  7956 catalog_manager.cc:5697] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 10 to 13, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 13 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:25.709827  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:25.710017  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:25.714608  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.731007  7956 catalog_manager.cc:5697] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 8 to 13, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 13 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:25.709291  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:25.744016  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.750794  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.752197 13934 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:15:25.754062  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 121953us
I20260709 13:15:25.755167  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.755767  8133 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:25.753432  7956 catalog_manager.cc:5697] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:25.758244  7956 catalog_manager.cc:5697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 9 to 16, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 16 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:25.759454 14582 consensus_queue.cc:1059] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:25.760747 14551 raft_consensus.cc:3055] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:25.760412  7956 catalog_manager.cc:5697] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:25.761193 14551 raft_consensus.cc:740] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:25.762193 14551 consensus_queue.cc:260] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.763598  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:25.764580  8059 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 19: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:25.765869  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:25.767688  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:25.769272  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:25.770730  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:25.774802  8026 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.777918  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:25.778574  8135 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:15:25.776365  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:25.779524  8131 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:15:25.782814 14479 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.786723  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.788811  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:25.762228  7956 catalog_manager.cc:5697] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 12 to 17, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 17 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:25.794085  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:25.797688  8102 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.805228  8062 raft_consensus.cc:1240] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:15:25.788265  8026 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.694301 14379 consensus_queue.cc:1048] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.081s
I20260709 13:15:25.766993 14332 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:25.763572 14551 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:25.809558  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.809356 13934 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:25.815717  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:25.820621  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:25.768628 14582 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:15:25.820816  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:25.817716  8026 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.783887 14568 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:25.825518  8101 leader_election.cc:304] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.805079 14475 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:25.826851  7956 catalog_manager.cc:5697] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 12 to 15, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 15 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:25.828780  8101 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.763700 14073 consensus_queue.cc:1059] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:15:25.750221 14562 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 13:15:25.815275 14516 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:25.830035 13934 consensus_queue.cc:1059] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:25.832706 14073 raft_consensus.cc:3055] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:25.835858 14073 raft_consensus.cc:740] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:25.836719 14073 consensus_queue.cc:260] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.828219 14431 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:25.839192 14561 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.823639 14567 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:25.842621  8134 raft_consensus.cc:1240] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:25.821650 14548 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:25.816798 14332 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:25.843205  7956 catalog_manager.cc:5697] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 8 to 18, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 18 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:25.844214  8101 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.827904 14582 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:25.825752 14568 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:25.852339  7956 catalog_manager.cc:5697] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 14 to 17, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 17 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:25.854614  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 77424.3us
I20260709 13:15:25.856678  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:25.857493  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:25.859778 14073 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:25.866523 14561 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:25.867473  8101 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.871080 14555 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:25.851704 14332 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:25.842660 14557 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.842999 14564 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:25.843245 14398 raft_consensus.cc:2749] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:25.882480  8129 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:25.883978  8101 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.905786  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.880769 14568 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:25.916298  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.915597  7956 catalog_manager.cc:5697] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 9 to 12, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 12 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:25.910274  8203 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 21. Current term is 22. Ops: []
I20260709 13:15:25.910833  8133 raft_consensus.cc:1240] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:25.912024  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.879175 14431 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:25.914232  8101 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.915271  8061 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:25.883716 14559 raft_consensus.cc:3055] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:25.925951 14561 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:25.935771 14440 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:25.896875 14582 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:25.939021  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.944446  8175 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:25.944764  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.952690  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.953063  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:25.953193  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:25.943383  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 100690us
I20260709 13:15:25.948272  7956 catalog_manager.cc:5697] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 13 to 17, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 17 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:25.959632  8104 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.960568  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:25.961519  8060 raft_consensus.cc:1240] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:25.961051 14564 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:25.964592 14564 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:15:25.937984 14252 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:25.967532  8101 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.968317 14471 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:15:25.968917 14557 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:25.960685 14554 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:25.915977 14555 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:15:25.971345 14555 consensus_queue.cc:1059] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:25.972049 14555 raft_consensus.cc:3055] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:25.972342 14555 raft_consensus.cc:740] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:25.973099 14555 consensus_queue.cc:260] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:25.973721  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:25.937829 14559 raft_consensus.cc:740] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:25.974550  8061 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 17: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:25.975113 14559 consensus_queue.cc:260] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.976403 14559 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:25.982470 14561 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:25.982743  8101 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:25.984025 14431 consensus_queue.cc:1059] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:25.984421  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:25.984854  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:25.986479 14516 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:25.987768  7956 catalog_manager.cc:5697] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 13 to 16, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 16 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:25.989300  7956 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 14 to 15, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 15 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:25.992969 14575 consensus_queue.cc:1059] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:25.993894 14575 raft_consensus.cc:3055] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:25.994206 14575 raft_consensus.cc:740] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:25.995036 14575 consensus_queue.cc:260] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:25.996249 14575 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:26.003340 14291 raft_consensus.cc:3055] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:26.003816 14291 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:26.004679 14291 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:26.005828 14291 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:26.007694 14555 raft_consensus.cc:3060] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:26.011827  8062 raft_consensus.cc:2468] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:26.013392 14568 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.014521  8211 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:26.017532 14568 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:26.019285  8104 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.027901 14590 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:26.032375  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 69650us
I20260709 13:15:26.035706  8026 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:26.033833  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.037068 14578 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:15:26.041574  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 78730.3us
I20260709 13:15:26.013538 14332 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.049723  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:26.043521 14577 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.014s
I20260709 13:15:26.057723  8026 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.059321  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:26.043568 14475 raft_consensus.cc:3055] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:26.043964 14578 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:26.062227  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:26.062453  8205 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:26.065429 14332 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:26.062315 14475 raft_consensus.cc:740] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:26.066797 14577 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:15:26.067152 14475 consensus_queue.cc:260] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.069157 14477 consensus_queue.cc:1059] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:26.070212 14477 raft_consensus.cc:3055] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:26.070482 14475 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:26.070587 14477 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:26.071902 14477 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.073472 14477 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:26.060971 14567 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:26.061192 14474 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:26.077163 14474 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:26.067179 14577 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:26.078099 14474 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:26.078311 14577 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:26.079910 14577 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:26.082013 13934 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:26.082577  8026 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.084098  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.085295  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:26.043767 14488 consensus_queue.cc:1059] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:15:26.088799  8026 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:26.091459 13934 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:26.091953  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:26.092109 13934 consensus_queue.cc:1059] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:26.093852  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.086402 14568 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:26.103092 14488 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:26.104115 14382 raft_consensus.cc:2749] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:26.104441 14428 raft_consensus.cc:3055] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:26.104800 14428 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:26.106797 13934 raft_consensus.cc:3055] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:26.107194 13934 raft_consensus.cc:740] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:26.107933 13934 consensus_queue.cc:260] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.109072 13934 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:26.088344 14559 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:26.112996 14589 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:26.113065 14549 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:26.118515  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:26.119248  8026 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.120505 14431 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:26.120857 14431 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:26.105638 14428 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.121184 14431 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:26.121698 14428 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:26.125304 14568 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:26.160431 14549 consensus_queue.cc:1059] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:15:26.158579 14567 consensus_queue.cc:1059] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:15:26.161404 14549 raft_consensus.cc:3055] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:26.161795 14549 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:26.162765  8129 raft_consensus.cc:1240] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:26.160147  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:26.161926 14332 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:26.170866 14332 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:26.171665 14332 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.172745 14332 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:26.172863 14549 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.174023 14549 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:26.175701 14431 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.177728 14431 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:26.179399  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:26.182065 14568 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:26.181524 14428 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:15:26.187338  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:26.181808  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:26.181843 14508 raft_consensus.cc:3055] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:26.190550 14508 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:26.192746 14428 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:26.224768  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:26.192528  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:26.244278 14332 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:26.245332  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 52880.5us
I20260709 13:15:26.191452 14508 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:26.223332 14479 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:26.252975  8175 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:26.318264 14479 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:26.318985 14479 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:26.334849  8134 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:26.355744  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet b578d88c1c1f4ca58b7291fed4909f09 with delete_type TABLET_DATA_TOMBSTONED (TS 84d775d1767e4cc4aa7b9c627d8f0fb8 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:50166
I20260709 13:15:26.361871 14428 raft_consensus.cc:2804] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:26.362506 14428 raft_consensus.cc:697] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:15:26.361776  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 108868us
I20260709 13:15:26.390828  8134 raft_consensus.cc:1240] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:26.391762  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:26.392828  8134 raft_consensus.cc:1240] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:26.394663  8026 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.397732  8134 raft_consensus.cc:1240] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 20. Current term is 21. Ops: []
I20260709 13:15:26.399092  8062 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:26.401772  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:26.402024  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:26.407604  8062 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:26.408329  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:26.410269  8134 raft_consensus.cc:1240] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:26.414443  8062 raft_consensus.cc:1240] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:26.400081 14590 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:26.419718 14590 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.421365 14590 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:26.427664 14508 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:26.430414 14428 consensus_queue.cc:237] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.442229  8175 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:26.438681 14559 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:26.446195 14567 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:15:26.451894 14568 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:26.473117 14567 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:26.444813 14549 consensus_queue.cc:1059] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:15:26.473541 14567 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:15:26.486407  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.467094 14556 consensus_queue.cc:1059] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 13:15:26.475544 14568 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:26.489252  8129 raft_consensus.cc:1240] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:26.489498  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:26.490839  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:26.493076  8137 raft_consensus.cc:2393] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:26.494130  8058 raft_consensus.cc:1240] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:26.494628  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:26.483127 14549 raft_consensus.cc:3055] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:26.474023 14479 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.495252  8060 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:26.500633  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:26.498315 14601 tablet_replica.cc:333] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:26.491575 14559 raft_consensus.cc:697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:26.505030  8131 raft_consensus.cc:1240] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 21. Current term is 22. Ops: []
I20260709 13:15:26.505331 14568 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.487978 14590 consensus_queue.cc:1059] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:15:26.496618 14549 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:15:26.502381 14598 log.cc:927] Time spent T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.111s	user 0.001s	sys 0.001s
I20260709 13:15:26.502089 14554 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:26.515480 14488 consensus_queue.cc:1059] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:15:26.504707 14605 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:26.510056 14567 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.510031 14556 consensus_queue.cc:1059] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:15:26.523836 14559 consensus_queue.cc:237] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.527001 14488 raft_consensus.cc:3055] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:26.532878 14488 raft_consensus.cc:740] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:26.533391  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:26.533689 14488 consensus_queue.cc:260] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:26.524616 14332 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:26.534754 14332 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:26.535061 14488 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:26.535666 14332 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:26.536924 14332 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:15:26.545843  8177 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.551209 14554 raft_consensus.cc:697] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:26.554528  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:26.554834  8724 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:15:26.555327  8724 pending_rounds.cc:85] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Aborting all ops after (but not including) 14
I20260709 13:15:26.555603  8724 pending_rounds.cc:107] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:15:26.555892  8724 raft_consensus.cc:2889] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:26.552119 14554 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.558517  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:26.559862 14556 consensus_queue.cc:1059] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:26.560375 14567 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:26.560791 14556 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:26.561180 14556 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:26.561514 14556 raft_consensus.cc:3060] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:26.564277 14431 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 13:15:26.570390 14549 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.571763 14549 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:26.571990 14605 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:26.572903 14605 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.581315 14428 raft_consensus.cc:3055] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:26.581718 14428 raft_consensus.cc:740] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:26.582520 14428 consensus_queue.cc:260] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.584676 14428 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:26.620088  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:26.623397 14605 consensus_queue.cc:1059] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:26.626614 14477 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.627401 14477 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:26.631850  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:26.633277  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:26.635291  8027 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.636325 14589 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:26.638762  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:26.639485  8137 raft_consensus.cc:1813] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Handling vote request from an unknown peer 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:26.640035  8137 raft_consensus.cc:2368] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:26.643826  8026 leader_election.cc:400] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:26.601431 14479 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:26.654631  8104 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.657780 14582 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:15:26.658932 14601 raft_consensus.cc:2243] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:26.659775 14601 raft_consensus.cc:2272] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:26.667763 14567 raft_consensus.cc:3055] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:15:26.668205 14567 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:26.669029 14567 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.674728 14556 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.678692 14556 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:26.689850  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 127338us
I20260709 13:15:26.692551  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:26.693235  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:26.695138  8137 raft_consensus.cc:2393] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:26.652530 14477 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:26.713433 14567 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:26.716312  8137 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:15:26.717877 14601 ts_tablet_manager.cc:1916] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:26.721146 14086 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:26.732642 14571 raft_consensus.cc:3055] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:26.733057 14571 raft_consensus.cc:740] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:26.733836 14571 consensus_queue.cc:260] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:26.739714 14571 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:26.745805  8057 raft_consensus.cc:2468] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:26.748456  8175 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:26.751972 14608 raft_consensus.cc:2804] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:26.797441 14608 raft_consensus.cc:697] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:26.798367 14608 consensus_queue.cc:237] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:26.916122 14608 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:26.916630 14608 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:26.919652 14608 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:26.920599  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:26.920651  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:26.921257  8724 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:26.922545  8176 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:26.923692 14608 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:26.924088 14608 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:26.924448 14608 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:26.925309  8131 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 20: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:26.928272  8131 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:15:26.928445 14601 ts_tablet_manager.cc:1929] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.10
I20260709 13:15:26.929648 14086 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:26.934090 14608 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:26.936949  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:26.937281  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:26.937983  8137 raft_consensus.cc:2393] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 20: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:26.939630 14601 log.cc:1199] T b578d88c1c1f4ca58b7291fed4909f09 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/wals/b578d88c1c1f4ca58b7291fed4909f09
I20260709 13:15:26.940554  7961 catalog_manager.cc:5697] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:26.941982  7961 catalog_manager.cc:5697] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 9 to 17. New cstate: current_term: 17 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:26.944681  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet b578d88c1c1f4ca58b7291fed4909f09 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:26.956112  8057 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:26.956740  8057 raft_consensus.cc:606] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.959508  8057 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:26.960034  8057 raft_consensus.cc:606] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:15:26.962500  8137 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:26.963027  8137 raft_consensus.cc:606] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.964480  8057 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:15:26.966148  8724 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:26.967340  8176 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:26.966028 14571 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:15:26.968226 14488 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:15:26.968080  7956 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34146 (request call id 261) took 16269 ms (client timeout 15000 ms). Trace:
I20260709 13:15:26.968816 14488 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:26.969786 14488 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:26.968931  7956 rpcz_store.cc:269] 0709 13:15:10.698457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.760269 (+ 61812us) service_pool.cc:224] Handling call
0709 13:15:26.967642 (+16207373us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:26.189871 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:26.190579 (+   708us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:26.190621 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:15:26.190644 (+    23us) tablet.cc:662] Decoding operations
0709 13:15:26.255463 (+ 64819us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:26.255539 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:26.255568 (+    29us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:15:26.257195 (+  1627us) tablet.cc:701] Row locks acquired
0709 13:15:26.257217 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:15:26.257361 (+   144us) write_op.cc:270] Start()
0709 13:15:26.257483 (+   122us) write_op.cc:276] Timestamp: P: 1783602926257312 usec, L: 0
0709 13:15:26.257508 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:15:26.316042 (+ 58534us) log.cc:844] Serialized 8239 byte log entry
0709 13:15:26.685110 (+369068us) op_driver.cc:464] REPLICATION: finished
0709 13:15:26.696665 (+ 11555us) write_op.cc:301] APPLY: starting
0709 13:15:26.696743 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:26.816502 (+119759us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:26.816595 (+    93us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:26.908428 (+ 91833us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:26.908783 (+   355us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=0,mrs_lookups=0
0709 13:15:26.908837 (+    54us) write_op.cc:312] APPLY: finished
0709 13:15:26.915441 (+  6604us) log.cc:844] Serialized 188 byte log entry
0709 13:15:26.916214 (+   773us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:26.916418 (+   204us) write_op.cc:454] Released schema lock
0709 13:15:26.916471 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:26.924041 (+  7570us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:26.924071 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4695,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":11338,"apply.run_cpu_time_us":24877,"apply.run_wall_time_us":227325,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91907,"num_ops":21,"prepare.queue_time_us":98850,"prepare.run_cpu_time_us":23699,"prepare.run_wall_time_us":269898,"raft.queue_time_us":171034,"raft.run_cpu_time_us":8531,"raft.run_wall_time_us":11578,"replication_time_us":427464,"spinlock_wait_cycles":4480,"thread_start_us":298038,"threads_started":4,"wal-append.queue_time_us":165483}]]}
I20260709 13:15:26.975729  8057 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a003b4584d3747a381522d67299aaed2"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:26.976795 14608 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:26.977101  8057 raft_consensus.cc:606] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.979681  8057 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ade1b48685164d64a62772d186ac959f"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:26.980379  8057 raft_consensus.cc:606] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.980683  8057 raft_consensus.cc:614] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:26.983108  8058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:26.983603  8058 raft_consensus.cc:606] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:26.984862  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:15:26.986552  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:26.986786  8206 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:15:26.990152 14477 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:26.995198  7961 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37774 (request call id 145) took 1232 ms. Trace:
I20260709 13:15:26.995631  7961 rpcz_store.cc:276] 0709 13:15:25.762822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:25.764300 (+  1478us) service_pool.cc:224] Handling call
0709 13:15:26.995142 (+1230842us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:26.944663 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:26.945230 (+   567us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:26.945266 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:15:26.945289 (+    23us) tablet.cc:662] Decoding operations
0709 13:15:26.945699 (+   410us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:26.945761 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:26.945797 (+    36us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:15:26.946115 (+   318us) tablet.cc:701] Row locks acquired
0709 13:15:26.946135 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:15:26.946272 (+   137us) write_op.cc:270] Start()
0709 13:15:26.946406 (+   134us) write_op.cc:276] Timestamp: P: 1783602926946229 usec, L: 0
0709 13:15:26.946430 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:15:26.980659 (+ 34229us) log.cc:844] Serialized 913 byte log entry
0709 13:15:26.983867 (+  3208us) op_driver.cc:464] REPLICATION: finished
0709 13:15:26.984264 (+   397us) write_op.cc:301] APPLY: starting
0709 13:15:26.984349 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:26.985613 (+  1264us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:26.985693 (+    80us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:26.986863 (+  1170us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:26.987006 (+   143us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:15:26.987045 (+    39us) write_op.cc:312] APPLY: finished
0709 13:15:26.987250 (+   205us) log.cc:844] Serialized 33 byte log entry
0709 13:15:26.987450 (+   200us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:26.987559 (+   109us) write_op.cc:454] Released schema lock
0709 13:15:26.987610 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:26.989190 (+  1580us) spinlock_profiling.cc:243] Waited 33.7 ms on lock 0x7b5400172d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb011e28 00007f6dbea4c581 00007f6dbea4bd58 00007f6dbea509f1 00007f6dbea509a9 00007f6dbea50939 00007f6dbea50901 00007f6dbea4fb2d 000000000038f354 000000000038f199
0709 13:15:26.989380 (+   190us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:26.989412 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12512,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":215,"apply.run_cpu_time_us":3635,"apply.run_wall_time_us":3635,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8360,"num_ops":2,"prepare.queue_time_us":95,"prepare.run_cpu_time_us":5271,"prepare.run_wall_time_us":44682,"replication_time_us":37311,"spinlock_wait_cycles":77433216}]]}
I20260709 13:15:27.027459  8131 raft_consensus.cc:1275] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:15:27.027781  8131 pending_rounds.cc:85] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649: Aborting all ops after (but not including) 11
I20260709 13:15:27.027999  8131 pending_rounds.cc:107] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:15:27.028213  8131 raft_consensus.cc:2889] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:27.030565  8057 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:27.030833 14571 consensus_queue.cc:1048] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:15:27.041177  8057 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:15:27.044679 14508 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:27.045989 14571 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:15:27.076169  8057 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:15:27.080130 14571 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:27.081043  8137 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:27.056921 14562 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.086651 14562 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.059468 14508 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.091272  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:27.091859  8061 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:27.092449 14508 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:27.092568  7955 catalog_manager.cc:5697] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 11 to 16, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 16 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:27.095489  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:27.096107  8061 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 22.
I20260709 13:15:27.093405 14479 raft_consensus.cc:993] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:27.090224 14562 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:27.098827 14479 raft_consensus.cc:1081] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:27.093323  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:27.100371  8134 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:27.094528  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:27.094786  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:27.101899  8211 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 22: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:27.102442  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:27.104136  8211 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59868
I20260709 13:15:27.104702 14508 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:27.104768  8211 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:27.105225 14508 raft_consensus.cc:493] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:27.105585  8211 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:27.109843  8060 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:27.110981 14562 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:27.111405 14562 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:27.111807 14562 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:27.115991  8129 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:27.118991  8134 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:15:27.121234  8137 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:15:27.123034  8211 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:27.124501  8129 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:27.124886 14488 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:27.128998 14488 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.105604 14508 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:27.138259  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:27.138931  8725 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:15:27.140018  8176 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:27.140065 14571 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:27.140877 14567 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:27.141252 14567 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:27.141608 14567 raft_consensus.cc:3060] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:27.144616  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:27.145681  8058 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:27.150501  8211 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:27.154507 14554 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:27.154790 14571 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:27.157864  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:27.158439  8131 raft_consensus.cc:3055] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:27.158735  8131 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:27.159457  8131 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:27.160703  8131 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:27.162508 14608 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:27.153264 14488 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:27.164201  8058 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:27.192498 14508 raft_consensus.cc:515] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.179883  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:27.197187  8131 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:27.198585  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:27.202678 14488 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:27.203403 14488 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:27.215931  8060 raft_consensus.cc:2393] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:27.225556 14440 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.226197 14440 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.230230 14440 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:27.230444  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:27.162184 14567 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.233170 14508 leader_election.cc:290] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:27.234198 14567 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:27.235144 14398 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.235641 14398 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.236905 14562 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.237730 14398 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:27.240427  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:27.240823  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:27.241163  8134 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:27.242419  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:27.204479 14488 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:27.252401  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:27.253373  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:27.253916  8062 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:27.253927  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ca98b6f039a45fb80c6e78ff2a0ae32" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:27.254784  8059 raft_consensus.cc:3060] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:27.255072  8134 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:15:27.255915  8693 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:27.257493  8060 raft_consensus.cc:2393] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 14: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:27.257716  8693 leader_election.cc:304] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:27.249146 14431 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:27.269161  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:27.261894 14508 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:27.266417 14440 raft_consensus.cc:2804] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:27.281836  8725 raft_consensus.cc:2468] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:15:27.282974  8176 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:27.279047 14508 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:27.309139  8060 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 22.
I20260709 13:15:27.273195 14431 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.319509  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:27.321174  8211 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
W20260709 13:15:27.329674  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 46698us
I20260709 13:15:27.320027 14508 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:27.336889  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:27.338508  8204 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 22: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:27.293625 14554 raft_consensus.cc:2804] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:27.285310 14440 raft_consensus.cc:697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 18 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:27.346627  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:27.320488 14562 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:27.360411 14546 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:27.360919 14546 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:27.361328 14546 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:27.390594 14575 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:27.391355 14575 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.395000  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:27.396045 14575 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:27.396198  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:27.396888  8205 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:27.398144  8696 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:27.403486 14575 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:27.403956 14575 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:27.404423 14575 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:27.412628 14575 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.348975 14554 raft_consensus.cc:697] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:27.429525 14554 consensus_queue.cc:237] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.430939  8131 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:27.437673 14440 consensus_queue.cc:237] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.439990 14634 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.440531 14634 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.441804 14431 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:27.443367 14634 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:27.444446 14633 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:15:27.445839  8059 raft_consensus.cc:2468] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:15:27.447698  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:27.448258  8725 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:27.450199  8102 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:27.451898  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:27.452548  8725 raft_consensus.cc:2393] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 17: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:15:27.457317 14634 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:27.460497 14634 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:27.526923  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 59000us
I20260709 13:15:27.465567  8060 raft_consensus.cc:1275] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:27.465384 14431 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.435321 14568 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 13:15:27.631282  7961 catalog_manager.cc:5697] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 16 to 18, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 18 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:27.636102  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:27.535727 14590 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.534431 14634 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:27.465123 14508 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.644176 14431 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:27.538053 14611 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:27.535688 14398 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.642233 14440 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.530577 14605 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
W20260709 13:15:27.783941  8133 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.028s	sys 0.036s
W20260709 13:15:27.841439  8133 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.030s	sys 0.036s
I20260709 13:15:27.537694 14608 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:27.820829  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 182098us
I20260709 13:15:27.806360 14635 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.978287  8725 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:15:27.988766  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 446693us
I20260709 13:15:28.003716  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:27.854152 14398 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.833639 14590 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:27.836830 14611 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.870805 14440 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:27.877203 14605 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.070760  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:28.038789 14567 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.075241  8131 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:15:27.987874 14608 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.004117 14635 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.071307 14575 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.102051 14657 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:28.111244 14568 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.079907 14567 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.125068 14644 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.133446 14559 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.130829 14562 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.135265 14479 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:28.158903 14546 consensus_meta.cc:287] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.797s	user 0.000s	sys 0.018s
I20260709 13:15:28.152833 14657 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 22 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:28.179522 14431 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:28.203358  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 96836.7us
I20260709 13:15:28.206847  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.153496 14568 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.229913  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:28.231967  8206 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:15:28.193519 14660 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:28.240358  8206 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:28.242166  8724 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:15:28.177546 14636 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:28.162005 14644 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:28.226565 14651 log.cc:927] Time spent T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.003s
I20260709 13:15:28.248935  8696 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:28.254451  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.020200  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:28.203142 14508 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.262578  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.263756  8208 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:28.184511 14479 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.180557 14562 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:28.253535 14640 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.363s	user 0.000s	sys 0.001s
I20260709 13:15:28.178710 14559 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.193374 14546 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.279162  8138 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:15:28.292003 14673 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:28.294749 14568 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:28.298609 14440 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.300379 14644 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:28.305132 14565 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.247431 14663 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 13:15:28.253417 14659 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:28.234546 14660 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.324936  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.353078  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.353231  8725 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:28.353662  8135 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:28.327875  8130 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:15:28.343227  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.345446 14639 raft_consensus.cc:993] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:28.352743 14565 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.345645 14488 consensus_queue.cc:1048] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:15:28.352682 14642 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.358902  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49210 (request call id 2205) took 1191 ms. Trace:
I20260709 13:15:28.359047 14642 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.359551 14488 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:28.360358 14565 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.360859 14565 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.352850 14667 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.361292 14565 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.361559 14667 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.362434 14659 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.362876 14590 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:28.287734 14571 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.363263 14678 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.363605 14571 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.363821 14678 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.325879 14647 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.364602 14659 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.364816 14647 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.345781  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.346917  8136 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:28.365768 14678 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.365784  8132 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:28.366808 14647 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.366971 14567 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:28.369493 14660 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:28.370579 14687 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.370884 14666 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.371089 14608 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:28.364398 14668 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.371492 14546 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.371356 14666 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.371868 14668 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.373371 14398 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.347800  8134 raft_consensus.cc:2368] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 19. Current term is 20.
I20260709 13:15:28.261608 14636 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.314456 14672 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:15:28.386703  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.375626  8176 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:28.377462  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.373767 14479 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.330582 14653 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:28.372227 14546 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.393045  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.359283  8057 rpcz_store.cc:276] 0709 13:15:27.167131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.169377 (+  2246us) service_pool.cc:224] Handling call
0709 13:15:27.169709 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:27.170098 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:27.170135 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:27.199928 (+ 29793us) log.cc:844] Serialized 26 byte log entry
0709 13:15:27.527580 (+327652us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:27.527636 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:27.527725 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:28.358699 (+830974us) raft_consensus.cc:1685] finished
0709 13:15:28.358719 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:28.358829 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25375,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":40603,"thread_start_us":232348,"threads_started":1,"wal-append.queue_time_us":462857}
W20260709 13:15:28.393801  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 78352.1us
I20260709 13:15:28.351588  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 2037) took 1153 ms. Trace:
I20260709 13:15:28.365716 14571 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.394258 14647 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:28.394621  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 63191us
I20260709 13:15:28.326258 14611 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.396304  8725 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 9.
I20260709 13:15:28.402700  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.423295  8176 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:15:28.402880  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 154026us
I20260709 13:15:28.403766  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.424449  8726 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:28.373744 14668 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.423132  8721 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 13.
I20260709 13:15:28.406363  8135 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:28.406370  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.373893 14398 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.371526 14578 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:28.397708 14634 consensus_meta.cc:287] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.000s
I20260709 13:15:28.415203  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:28.363564 14667 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.434881  8729 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:28.394431 14567 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.434017  8102 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:28.436551  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.331043  8205 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:28.410950  8029 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:28.436985  8102 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:28.411334  8133 rpcz_store.cc:276] 0709 13:15:27.180428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.180536 (+   108us) service_pool.cc:224] Handling call
0709 13:15:27.180812 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:27.196142 (+ 15330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:27.196174 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:27.216091 (+ 19917us) log.cc:844] Serialized 26 byte log entry
0709 13:15:27.873135 (+657044us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:28.315690 (+442555us) log.cc:844] Serialized 14 byte log entry
0709 13:15:28.333367 (+ 17677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.333468 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:28.333520 (+    52us) raft_consensus.cc:1685] finished
0709 13:15:28.333538 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:28.333692 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14734,"prepare.queue_time_us":532781,"raft.queue_time_us":111062,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":39427,"thread_start_us":917848,"threads_started":2,"wal-append.queue_time_us":885453}
I20260709 13:15:28.425992  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.373749 14666 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.346621 14440 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.066s
I20260709 13:15:28.426430  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:28.339563 14638 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 13:15:28.427491  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.443336  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.321750 14673 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.427834  8723 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:28.405887 14590 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:28.405741 14568 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.430305  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.408797 14565 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.432318 14682 raft_consensus.cc:1081] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:28.458694  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.440079  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.414772 14479 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.459695 14479 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.459971  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.460707  8207 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:28.461704  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:28.462275  8209 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:28.414613 14653 raft_consensus.cc:697] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:28.444056  8135 raft_consensus.cc:2393] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 21: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:28.463655 14653 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.430306 14644 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:28.448153  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.443722  8730 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 13.
I20260709 13:15:28.466569 14644 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.467211 14644 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:28.462039 14479 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.471402  8209 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:28.434129 14611 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.458950  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.472121 14611 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.473299  8696 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:28.474318 14611 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:28.436741 14578 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.475602  8133 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:28.476531 14578 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.477437  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.478333  8205 raft_consensus.cc:2410] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
W20260709 13:15:28.433293  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 78701.7us
I20260709 13:15:28.371134 14687 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.436852 14654 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.414788 14546 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.492069  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.493719  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:28.494277  8060 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:28.496738  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.497026  8057 raft_consensus.cc:1275] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:15:28.497975  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.498579  8058 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 14.
I20260709 13:15:28.494829  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:28.505239  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.505815  8057 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:28.507473 14679 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:28.507696 14666 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.508014 14679 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.508240 14666 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.452049 14590 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:28.442641 14567 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.511608 14590 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:28.513289 14567 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:28.514180 14675 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:28.514365 14670 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.451948 14568 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.514911 14670 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.515887  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 2035) took 1339 ms. Trace:
I20260709 13:15:28.516304 14568 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.516472  8208 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:15:28.514658 14675 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.517869 14670 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.518234 14675 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:28.354183 14657 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.516243  8137 rpcz_store.cc:276] 0709 13:15:27.176509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.176613 (+   104us) service_pool.cc:224] Handling call
0709 13:15:27.176873 (+   260us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:27.195744 (+ 18871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:27.195773 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:27.196183 (+   410us) log.cc:844] Serialized 26 byte log entry
0709 13:15:27.333090 (+136907us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b4800002dd8. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbff3faa1 00007f6dbff36b37 00007f6dbff369ca 00007f6dc002e26f 00007f6dc002df37 00007f6dc004255c 00007f6dc0042508 00007f6dc0042488 00007f6dc0042443 00007f6dc00416ec 00007f6dbef6ba96
0709 13:15:27.536629 (+203539us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:27.536673 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:27.536745 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:28.515626 (+978881us) raft_consensus.cc:1685] finished
0709 13:15:28.515645 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:28.515822 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23451,"raft.queue_time_us":18652,"raft.run_cpu_time_us":2537,"raft.run_wall_time_us":139614,"spinlock_wait_cycles":53234944,"thread_start_us":289491,"threads_started":1,"wal-append.queue_time_us":725138}
I20260709 13:15:28.520834  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.521867  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.522953  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.523566  8137 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:28.524384  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.524993  8137 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:28.525871  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:28.526535  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 92972.6us
W20260709 13:15:28.527256  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 54068.8us
I20260709 13:15:28.510095 14679 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.523926  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:28.529260  8729 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:28.532444  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.533000  8723 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:28.449615 14671 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.426728 14647 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.437242 14398 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.540516  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.541105  8212 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:28.539587  8176 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:28.535820  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.536906  8060 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:28.538093  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.539891  8104 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:28.439402 14634 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.538516  8137 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:28.501562 14639 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:28.550123  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:28.600762  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 72546.2us
I20260709 13:15:28.600692  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.602061  8059 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:28.602986  8211 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 21.
I20260709 13:15:28.601814  8057 raft_consensus.cc:2410] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:15:28.604614  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.525457 14608 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:28.616815  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.503456 14654 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.554479 14562 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:28.628394  8058 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 14.
I20260709 13:15:28.631321  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.603047 14679 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.599687 14567 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.629266 14661 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:28.654270  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.602526 14646 raft_consensus.cc:2764] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 13:15:28.671625  8060 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 19: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:28.606817 14639 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.661494  8129 raft_consensus.cc:2368] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:28.672458  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:28.630174 14608 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.602730 14667 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:28.629683 14559 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.538522 14666 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.676390 14639 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:28.684794  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.676654 14562 consensus_queue.cc:1048] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 13:15:28.676571 14608 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:28.541253 14671 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.681571 14508 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:28.683772 14578 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:15:28.734895  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 53700.7us
I20260709 13:15:28.735404 14687 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.662292 14679 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.729676 14382 raft_consensus.cc:993] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
W20260709 13:15:28.737746  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 300812us
I20260709 13:15:28.685616 14565 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:28.674917 14567 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.674616 14661 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.742520  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.729414 14660 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.727787 14694 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.725081 14667 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.737829 14578 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.736209 14508 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.746279 14691 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:28.762715  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:28.749462 14696 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.763176 14658 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:28.756798 14694 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.764791 14696 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.765029 14658 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.766394 14694 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.766831 14696 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.767341 14696 raft_consensus.cc:1081] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:28.767690 14654 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.769824 14673 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.761212 14508 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:28.748462 14661 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:28.773245 14676 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:28.773697 14676 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.774031 14676 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:28.774596 14546 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.775158 14634 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:28.784412 14686 log.cc:927] Time spent T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:15:28.785476 14605 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.784798 14671 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.785888  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.786643  8131 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 22.
I20260709 13:15:28.787509  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.761677 14565 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:28.790832  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.791109 14565 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.791884  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.792408  8129 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 20.
I20260709 13:15:28.793140  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.792847 14676 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.795404 14676 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.797343 14644 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.798563  8204 raft_consensus.cc:2393] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:28.799008 14641 raft_consensus.cc:993] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:15:28.758975 14578 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:28.800515  8131 raft_consensus.cc:2368] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 14. Current term is 15.
I20260709 13:15:28.808326  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.807173 14641 raft_consensus.cc:1081] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:15:28.815495 14674 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.816042 14674 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.817713 14647 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.819054 14642 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.823921 14567 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:28.824504 14398 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.825052 14701 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.825601 14701 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:28.827033  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 64369.9us
I20260709 13:15:28.827687 14701 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.749230 14660 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:28.829442  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 92414.2us
I20260709 13:15:28.830415  8027 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:28.830613  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.830772  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:15:28.832388  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 51086.2us
I20260709 13:15:28.832387  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:28.832968  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.833271  8730 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:28.833871  8101 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:28.831537  8725 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:28.828190 14701 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:28.800808 14658 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.839875  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:28.841619  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:28.842650  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.843340  8059 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
I20260709 13:15:28.843611  8026 leader_election.cc:400] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:28.843974  8026 leader_election.cc:403] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:28.842034  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.844770  8058 raft_consensus.cc:2410] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:15:28.845557  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:28.845692  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.846235  8059 raft_consensus.cc:3060] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:15:28.846633  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.847322  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.847292  8026 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:28.848778  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:28.851056  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.851271  8101 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:28.764101 14691 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.852878  8054 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 21.
I20260709 13:15:28.854943  8723 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:28.856340  8727 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55848
I20260709 13:15:28.856815  8727 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:28.857112  8727 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:28.827069 14641 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:28.857937  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.863623 14700 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:28.864048 14700 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.864375 14700 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:28.857693 14663 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.868455  8101 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:28.846724 14701 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.876386  8059 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:28.851686 14684 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:28.853609 14647 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:28.895210 14647 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.895745 14647 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:28.852798 14666 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:28.878036  8101 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:28.881104 14693 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:28.881767 14710 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:15:28.881981 14699 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:15:28.858477 14673 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:28.884104 14674 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.886081 14660 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:28.857913 14398 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:28.877543 14701 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:28.873672 14582 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:28.894970 14684 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.880470 14691 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:28.870997  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 429833us
I20260709 13:15:28.848636 14675 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.903450  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.882287 14559 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18.
I20260709 13:15:28.860474 14611 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:28.909796  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.930600 14611 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.931212  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.931394  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.931823  8138 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:28.931982  8725 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:28.932654  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.933230 14611 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.933722  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.939842  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:28.940935 14688 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:15:28.941339 14688 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.941897 14688 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:28.941857  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.891076 14508 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.912531  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.850050 14590 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.867111 14663 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.953246 14554 raft_consensus.cc:493] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:28.953755 14554 raft_consensus.cc:515] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.955842 14554 leader_election.cc:290] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.955986 14508 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:28.956640  8729 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:28.957913 14590 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.958508  8206 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:15:28.961069  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.961278  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.961637  8209 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:28.962148  8212 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59856
I20260709 13:15:28.961655  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.962659  8212 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:28.963044  8212 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:28.907282 14666 raft_consensus.cc:2804] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:28.964534  8205 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a003b4584d3747a381522d67299aaed2"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59856
I20260709 13:15:28.964885 14666 raft_consensus.cc:697] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:28.965056  8205 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:28.965415  8205 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:28.965742 14666 consensus_queue.cc:237] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.967033  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.967761  8130 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:28.907135 14673 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.902576  8730 raft_consensus.cc:2468] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:28.961800  8211 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:28.969626  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:28.975674  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.964572  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7f54a64a14435691eeef2ece07e5dc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.968791  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.978165  8130 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 20.
I20260709 13:15:28.979337  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:28.908236 14710 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.925277 14684 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.985544  8205 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.927309 14639 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.916961 14398 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.871661 14641 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.929314 14635 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:28.941288  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:28.933990  8102 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:15:28.996786  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 62894.9us
I20260709 13:15:28.997951  8102 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:28.905769  8727 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:28.893483 14578 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.951188  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.904958 14693 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.963090  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:29.003618  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.963513  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:28.921051  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 381211us
I20260709 13:15:28.980726  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:29.005014  8209 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 20.
I20260709 13:15:29.005183  8211 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 22.
I20260709 13:15:29.005244  8104 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:28.969702  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:28.893796 14661 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.004979  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:29.007357  8059 raft_consensus.cc:3055] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:29.007660  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:29.007937  8059 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:29.003649  8724 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 12.
I20260709 13:15:28.985355  8729 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:28.986835  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:28.978500  8026 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:28.925570 14608 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:28.954218 14663 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.116667 14641 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:28.993988 14650 log.cc:927] Time spent T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.688s	user 0.007s	sys 0.004s
I20260709 13:15:29.011823  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c4e1687a1db419c8863789bb37c260a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:29.119889 14710 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:29.120189  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:29.120986  8136 raft_consensus.cc:2368] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:29.121877  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:29.020190  8134 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46944 (request call id 1022) took 1603 ms (client timeout 1724 ms). Trace:
W20260709 13:15:29.140688  8134 rpcz_store.cc:269] 0709 13:15:27.394635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.394769 (+   134us) service_pool.cc:224] Handling call
0709 13:15:28.998132 (+1603363us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b54000d20d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:28.998281 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2189507840}
I20260709 13:15:29.142132  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:29.110433 14635 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:29.111774 14398 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.990778 14691 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:29.259243  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:29.112428 14634 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:29.112974 14693 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:29.113265 14639 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:28.972530 14611 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:29.115069  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:28.990866 14648 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:28.946095  8137 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:29.122582 14708 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:28.987516 14673 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:29.011591 14684 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:29.123974 14644 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:29.124583 14578 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.126097 14608 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.127451 14685 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:15:29.129040  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 45446us
I20260709 13:15:29.129841  8129 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:29.131524  8207 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 20: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:15:29.145860  8693 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.6.198.66:45807 timed out after 1.724s (SENT)
I20260709 13:15:28.969751 14667 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.134202  8212 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.200385  8057 raft_consensus.cc:2393] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 19: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:29.214758  8723 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:29.257493  8205 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:29.197896  8059 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.312584  8130 raft_consensus.cc:2393] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 20: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:29.343335  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:29.201099 14635 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.121381 14701 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.369400  8131 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:29.333700 14559 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.312698 14641 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:29.300294 14479 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:29.376492 14709 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.376752 14641 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.377450 14559 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.312120 14691 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.387992 14691 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.379231  8212 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.379468 14664 raft_consensus.cc:2749] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:29.344087 14648 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.384754 14641 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:29.348281 14684 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:29.394294  8138 raft_consensus.cc:2393] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 22: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:29.371696 14635 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:29.335433 14634 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.345227 14611 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:29.470374  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 491922us
I20260709 13:15:29.472167  8062 raft_consensus.cc:1275] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:15:29.351991 14708 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.402567 14684 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:29.473284  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.475613  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.479336  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.479904  8062 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
W20260709 13:15:29.480072  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 334236us
I20260709 13:15:29.463168 14709 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.476298  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.480711  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.483124  8058 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:29.484071  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.484565  8058 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 21.
I20260709 13:15:29.485764  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.481662  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:29.487198  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.489769  8058 raft_consensus.cc:2368] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:15:29.490918  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.389528 14479 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.496203 14709 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.496647 14719 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:29.498567 14508 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:29.493321  8693 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:29.516932  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:29.485770  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:29.499154 14680 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:29.522182  8693 leader_election.cc:400] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 14. Current term is 15.
W20260709 13:15:29.487972 14700 consensus_meta.cc:287] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.623s	user 0.003s	sys 0.007s
I20260709 13:15:29.482575 14559 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.524664  8131 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:29.526571  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:29.524444  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:29.494228 14675 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:29.388844 14685 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:29.517689 14567 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.017s	sys 0.013s
I20260709 13:15:29.534261  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:29.536962  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:29.537663  8131 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:29.519552 14660 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:29.526975 14663 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:29.493652 14488 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:15:29.525220 14657 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:29.525990 14676 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:29.541077 14567 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.017s	sys 0.013s
I20260709 13:15:29.559396  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:29.546429 14663 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.580899 14663 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:15:29.581072 14688 consensus_meta.cc:287] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.639s	user 0.006s	sys 0.003s
I20260709 13:15:29.519819 14719 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.532989 14700 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.575914 14701 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:29.584342 14719 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.585050  8055 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:29.585862 14688 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:29.586753 14714 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:15:29.533618 14680 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.590615  8131 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:29.591998 14680 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:29.556051 14657 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:29.593530 14657 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:29.594648  8136 raft_consensus.cc:2368] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 22. Current term is 23.
I20260709 13:15:29.573136  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:29.576285  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:15:29.577739 14647 consensus_meta.cc:287] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.681s	user 0.007s	sys 0.000s
I20260709 13:15:29.579807 14644 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:29.580345 14732 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.598321 14644 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.598801 14644 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:29.598565 14732 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.599089 14611 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.601127 14663 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.568403 14661 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:29.582036  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 1030) took 1127 ms. Trace:
I20260709 13:15:29.603224 14663 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.602375  8137 rpcz_store.cc:276] 0709 13:15:28.454564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.380430 (+925866us) service_pool.cc:224] Handling call
0709 13:15:29.383605 (+  3175us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:29.383992 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.384031 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:29.384609 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 13:15:29.467069 (+ 82460us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:15:29.467146 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.467267 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:29.581838 (+114571us) raft_consensus.cc:1685] finished
0709 13:15:29.581858 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.581983 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95516,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":12705,"thread_start_us":81919,"threads_started":1,"wal-append.queue_time_us":154921}
I20260709 13:15:29.547808 14660 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.604133 14660 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:29.604691  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2712) took 1077 ms. Trace:
I20260709 13:15:29.605069  8137 rpcz_store.cc:276] 0709 13:15:28.527499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.603966 (+1076467us) service_pool.cc:224] Handling call
0709 13:15:29.604247 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.604412 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.604438 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:29.604477 (+    39us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:29.604506 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.604555 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.604646 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.607270  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2713) took 1078 ms. Trace:
I20260709 13:15:29.609508  8137 rpcz_store.cc:276] 0709 13:15:28.529074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.606228 (+1077154us) service_pool.cc:224] Handling call
0709 13:15:29.606818 (+   590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.606970 (+   152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.606993 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:29.607029 (+    36us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:29.607065 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.607116 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.607221 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.610554 14644 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.597018  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:29.612068  8060 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:29.612548 14660 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.613796  8723 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 19: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:15:29.612746  8137 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46944 (request call id 1033) took 1002 ms. Trace:
I20260709 13:15:29.594632  8026 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:29.614553  8137 rpcz_store.cc:276] 0709 13:15:28.610307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.611655 (+1001348us) service_pool.cc:224] Handling call
0709 13:15:29.612273 (+   618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.612459 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.612485 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:29.612521 (+    36us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:15:29.612551 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.612603 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.612701 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:29.615767 14643 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:29.614890 14660 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.597688 14717 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.618918 14717 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.611315  8029 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:29.597931 14605 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:29.596303  8062 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:29.623558  8062 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:29.584359 14700 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:29.637907 14697 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:29.637954  8062 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.561436 14676 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.639055  8062 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:29.597558  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:15:29.642398  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 44911.6us
I20260709 13:15:29.626271  8060 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:29.596097 14719 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:29.606015 14684 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.643898  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:29.645737 14719 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.648996 14719 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:29.625000 14605 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.650800 14605 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:29.651664 14717 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.655308  8130 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:29.655385 14663 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:29.655859 14663 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.656219 14663 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:29.614295 14693 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.658692  8134 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 20: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:29.663759 14663 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.665693 14663 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.668404 14635 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.661898 14673 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.671579 14635 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:29.673681 14673 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.677428 14641 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.679244 14605 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.675465 14657 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.680207 14641 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.685787 14657 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.675777  8175 leader_election.cc:400] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:29.634533 14647 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.686729  8175 leader_election.cc:403] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:15:29.687474 14663 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:29.689116 14647 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:29.691145 14643 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:29.693328 14644 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.695330 14643 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.695990 14605 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.696497 14663 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:29.690980 14661 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:15:29.698550 14661 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.699039 14661 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:29.639278  8029 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:15:29.744108  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 104744us
I20260709 13:15:29.747413  8063 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 21: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:29.748881  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:29.752170  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.745850  8029 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:29.756217  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.757308  8056 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 22.
I20260709 13:15:29.757579  8129 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36064 (request call id 2714) took 1223 ms. Trace:
I20260709 13:15:29.758093  8129 rpcz_store.cc:276] 0709 13:15:28.534483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.792932 (+258449us) service_pool.cc:224] Handling call
0709 13:15:29.757019 (+964087us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54001f18d4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:29.757332 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":772772736}
I20260709 13:15:29.759157  8058 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 20: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:29.760843  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.753587  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.761994  8063 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:29.753623  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:29.766758  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 3679) took 1150 ms. Trace:
I20260709 13:15:29.767148  8055 raft_consensus.cc:3055] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:29.767634  8055 raft_consensus.cc:740] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:29.767310  8057 rpcz_store.cc:276] 0709 13:15:28.616399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.616540 (+   141us) service_pool.cc:224] Handling call
0709 13:15:29.766254 (+1149714us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b540027a954. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:29.766562 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147149568}
I20260709 13:15:29.769731  8133 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2706) took 1353 ms. Trace:
I20260709 13:15:29.770139  8133 rpcz_store.cc:276] 0709 13:15:28.416019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.257443 (+841424us) service_pool.cc:224] Handling call
0709 13:15:29.333800 (+ 76357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.333967 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.333994 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:29.334038 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:29.334080 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.334148 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.769344 (+435196us) spinlock_profiling.cc:243] Waited 75.9 ms on lock 0x7b5400043650. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:29.769517 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174641920}
I20260709 13:15:29.773759  8138 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36048 (request call id 2056) took 1012 ms. Trace:
I20260709 13:15:29.774224  8138 rpcz_store.cc:276] 0709 13:15:28.761090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.975407 (+214317us) service_pool.cc:224] Handling call
0709 13:15:29.773240 (+797833us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b540023cdd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:29.773557 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":955345664}
I20260709 13:15:29.777877  8130 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2710) took 1351 ms. Trace:
I20260709 13:15:29.782176  8130 rpcz_store.cc:276] 0709 13:15:28.426034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.371596 (+945562us) service_pool.cc:224] Handling call
0709 13:15:29.526166 (+154570us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.655819 (+129653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.762813 (+106994us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54003c8cd4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:29.777248 (+ 14435us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54003c8cd0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb0158ad 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:29.777588 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":636201600}
I20260709 13:15:29.780910 14571 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:29.783944 14643 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:29.783910 14571 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:29.779402  8204 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55942 (request call id 1231) took 1349 ms (client timeout 1511 ms). Trace:
I20260709 13:15:29.784588 14643 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.786633  8722 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:15:29.787232 14643 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:29.787402 14440 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:29.788582 14644 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:29.788372 14440 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.789378 14644 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.790901 14575 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:29.793032  8056 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:29.786875 14605 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:29.793371 14575 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.794194 14647 consensus_queue.cc:1048] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:15:29.796175 14639 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:29.797868 14575 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.794262 14605 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.799998 14648 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.800482 14605 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.801395  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:29.803697  8727 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.803814  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:29.805599 14605 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:29.791556 14571 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.806422  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:29.807034  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:29.807338  8730 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:15:29.808008  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 62497.1us
I20260709 13:15:29.808475  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:29.809140  8029 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:29.808032  8138 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:29.805096  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:29.791712 14567 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:29.811335  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:29.811452  8130 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 20.
I20260709 13:15:29.812192  8136 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:29.812152 14567 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.813318  8135 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36064 (request call id 2705) took 1397 ms. Trace:
I20260709 13:15:29.813692  8135 rpcz_store.cc:276] 0709 13:15:28.415691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.464439 (+ 48748us) service_pool.cc:224] Handling call
0709 13:15:28.464690 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:28.674127 (+209437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:28.674174 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:28.674230 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:28.674263 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.812843 (+1138580us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b54001a9d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb016e01 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c 00007f6db9550cc2 00007f6db935148c 00007f6db93508c6
0709 13:15:29.812904 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.813139 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481246592}
I20260709 13:15:29.792026 14644 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.816265 14565 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:29.816911 14565 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.769393  8055 consensus_queue.cc:260] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.818881 14440 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.819775  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:29.820700  8134 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:29.820989 14565 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.824769  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:29.829510 14654 raft_consensus.cc:2764] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:15:29.802167  8725 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:29.831475  8209 raft_consensus.cc:1240] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:29.832087  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:29.833315  8212 raft_consensus.cc:2368] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 20. Current term is 21.
W20260709 13:15:29.786576  8204 rpcz_store.cc:269] 0709 13:15:28.429753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.430019 (+   266us) service_pool.cc:224] Handling call
0709 13:15:29.778849 (+1348830us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b540014a354. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:29.779159 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":800531712}
I20260709 13:15:29.840375  8029 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:29.841593 14641 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:29.842029 14641 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:29.842559 14641 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:29.843137  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55942 (request call id 1229) took 1417 ms. Trace:
I20260709 13:15:29.843568  8206 rpcz_store.cc:276] 0709 13:15:28.425623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.425775 (+   152us) service_pool.cc:224] Handling call
0709 13:15:29.842723 (+1416948us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b54002c8b54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:29.842963 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1193037696}
I20260709 13:15:29.848052 14567 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.848723 14648 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:29.833966  8696 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:29.829054  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:29.858979  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:29.859671  8212 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:29.861227 14643 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.863183  8062 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:29.864020 14643 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.866979 14662 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:29.867631  8696 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:29.867916 14662 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.869056 14660 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:29.869513 14660 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:29.869917 14660 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:29.873099  8136 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 21.
I20260709 13:15:29.876591  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:29.880221  8212 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
W20260709 13:15:29.881418  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 52594.4us
I20260709 13:15:29.881521 14605 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:29.882634  8175 leader_election.cc:400] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:15:29.882843 14546 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:29.883692 14546 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.885612  8722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:29.888294  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:29.889034  8729 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:29.889744 14546 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:29.889667  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:29.890430  8727 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:29.880323 14636 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.891965  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:29.892179  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:29.893230  8130 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:29.873797 14568 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:29.894455 14568 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.882540  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:29.896415  8135 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 21: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:29.898871  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:29.893139  8132 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 19: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:29.899962  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:29.883086  8175 leader_election.cc:403] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:29.903096  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:29.871449 14644 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:29.907059  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:29.908695  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:29.909286  8134 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:15:29.910820  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:29.920079 14644 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.920676 14688 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.923084  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:29.903735 14662 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.923355 14665 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919) with higher term. Message: Invalid argument: T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17.
I20260709 13:15:29.923446  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:29.923830  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 48775.3us
I20260709 13:15:29.925927  8730 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 22.
I20260709 13:15:29.926945  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:29.928135  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:29.929490 14653 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:29.930065 14653 raft_consensus.cc:697] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:29.931419 14653 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.933833  8131 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36048 (request call id 2049) took 1453 ms. Trace:
W20260709 13:15:29.934269  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 55788.3us
I20260709 13:15:29.934566  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:29.935420  8726 raft_consensus.cc:2393] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 15: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:29.936826  8029 leader_election.cc:400] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 20. Current term is 21.
I20260709 13:15:29.937162  8029 leader_election.cc:403] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:29.938086 14648 consensus_queue.cc:1059] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:29.939533 14648 raft_consensus.cc:3055] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:29.939920 14648 raft_consensus.cc:740] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:29.941098 14648 consensus_queue.cc:260] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:29.938649 14682 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:15:29.934269  8131 rpcz_store.cc:276] 0709 13:15:28.480268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.592087 (+1111819us) service_pool.cc:224] Handling call
0709 13:15:29.592574 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:29.592868 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.592894 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:29.593535 (+   641us) log.cc:844] Serialized 26 byte log entry
0709 13:15:29.921916 (+328381us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:15:29.921987 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.922115 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:29.933438 (+ 11323us) raft_consensus.cc:1685] finished
0709 13:15:29.933456 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.933594 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1280,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":1990,"thread_start_us":311772,"threads_started":1,"wal-append.queue_time_us":321478}
I20260709 13:15:29.927598 14440 raft_consensus.cc:2764] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:15:29.950691  8136 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 20: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:29.952665  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.954221  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:29.965167  8061 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 21: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:29.966223  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.954279 14634 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:29.969688  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:29.971608 14648 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:29.972035  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:29.972061 14687 raft_consensus.cc:493] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:29.973023 14687 raft_consensus.cc:515] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.973862 14719 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:29.974517 14719 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.975329  8726 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:15:29.975576 14688 raft_consensus.cc:493] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:29.976373 14655 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:29.976200 14688 raft_consensus.cc:515] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.976780 14719 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.976941 14655 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:29.977416 14719 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:29.978776  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:29.979449  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:29.980325  8136 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:29.984390 14568 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:29.986815  8135 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:15:29.973009  8058 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:29.990067 14676 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:29.978153 14719 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:29.993494  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 111170us
I20260709 13:15:29.990777  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:29.996846 14641 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:29.999059 14707 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 1.042s	user 0.013s	sys 0.010s
I20260709 13:15:30.001385  8175 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:30.001726  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:30.002261 14568 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:30.002725  8135 raft_consensus.cc:2368] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 22. Current term is 23.
I20260709 13:15:30.003072 14568 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:30.004124 14682 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 20. Current term is 21.
I20260709 13:15:30.004375 14568 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:30.005013  8134 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 20.
I20260709 13:15:30.005926 14708 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.006423 14691 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:30.007113 14691 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:30.008385 14721 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:30.008736 14648 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:30.010705 14691 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.011853 14704 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:30.011313 14648 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:30.014137  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.017064  8055 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:30.010732  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.022455  8175 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:15:30.025173  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.026803 14648 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:30.027516  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.012575 14704 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:30.028965 14590 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:30.029798  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:30.030385 14704 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.031931 14695 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:15:30.032588 14695 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.032948 14695 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:30.034858 14641 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.035216  8062 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:15:30.038012  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.038908 14721 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:30.003124  8059 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:30.031854  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:29.979388 14688 leader_election.cc:290] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.046878  8729 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:15:30.054768  8136 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:15:30.041875 14719 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:30.054652  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:30.011333 14691 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:30.062125  8131 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:30.065761  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.141911  8054 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:15:30.146660  8176 leader_election.cc:304] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:30.146822 14732 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:30.147605  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.154637  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:30.063787 14644 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:30.157251 14691 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:30.162600  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:15:30.163810  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 95541.5us
I20260709 13:15:30.165392  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:30.166290  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:30.166515  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 103588us
I20260709 13:15:30.166875  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.167373  8029 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:30.168161  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:30.168901  8129 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:15:30.169145  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 106029us
I20260709 13:15:30.169524 14671 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:30.170218  8102 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:30.171466 14671 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:30.171592  8102 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:30.171949 14431 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:30.172464 14546 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:30.172744 14732 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:30.173035 14546 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:30.173669 14546 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:30.173079 14660 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:30.172014 14671 raft_consensus.cc:697] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:30.178747 14660 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.179327  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:30.179924  8721 raft_consensus.cc:3060] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:30.180130  8137 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 21: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:15:30.181881  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 126857us
I20260709 13:15:30.181221 14671 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:30.184202 14719 raft_consensus.cc:2749] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:30.184790  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:15:30.188257  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 127292us
I20260709 13:15:30.179421 14660 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:30.172514  8102 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:30.203001  8056 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 21.
I20260709 13:15:30.204898  8055 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:15:30.190301 14431 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 13:15:30.245321  8131 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:30.254603  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:15:30.246254  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 73823.7us
I20260709 13:15:30.246037  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.253566  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:30.271718  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 86392.5us
I20260709 13:15:30.269948  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:30.284492  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:30.285763  8176 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:30.295197  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:30.275646 14568 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:15:30.303133 14695 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:30.305235 14695 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 23 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:30.307648  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:30.310909  8132 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 19: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:30.311961  8129 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 21.
I20260709 13:15:30.312285  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:30.312974  8063 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 18: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:30.311489  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:30.289397 14688 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:30.319036  8059 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:30.320837  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:30.322430 14644 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.327239 14644 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:30.329430 14644 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:30.331931  8059 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37156 (request call id 1092) took 1320 ms. Trace:
I20260709 13:15:30.332314  8059 rpcz_store.cc:276] 0709 13:15:29.003261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.004765 (+  1504us) service_pool.cc:224] Handling call
0709 13:15:30.002167 (+997402us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b58006ea5d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf720aa 00007f6dbaff4447 00007f6dbb00cfe7 00007f6dbb00e7de 00007f6dbb01a27c 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c
0709 13:15:30.324042 (+321875us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1456098176}
W20260709 13:15:30.334537  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 48820.6us
I20260709 13:15:30.336689  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:30.337762  8055 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:30.333359 14688 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 22 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:15:30.342473 14666 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.279s	user 0.029s	sys 0.021s
I20260709 13:15:30.350054  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:30.343048 14695 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:30.353258  8055 raft_consensus.cc:2393] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 17: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:15:30.346705 14666 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.378s	user 0.029s	sys 0.021s
I20260709 13:15:30.346249 14663 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:15:30.357674  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:30.355702 14661 consensus_meta.cc:287] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.589s	user 0.013s	sys 0.002s
I20260709 13:15:30.362396  8058 raft_consensus.cc:2393] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 13: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:30.358223 14695 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:30.367396  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:30.367946  8055 raft_consensus.cc:3060] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:30.376190 14688 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:30.366465 14661 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:30.384012  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:30.381414 14546 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:30.385696 14661 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 23 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:30.386255 14546 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:30.403086  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:30.413309  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:30.416435  8101 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:30.418090 14546 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:30.418503 14546 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:30.418871 14546 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:30.425045 14546 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:30.426527 14546 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.427407  8029 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:30.428485 14668 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:30.406486 14660 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:30.446621  8055 raft_consensus.cc:2468] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 23.
I20260709 13:15:30.500378  8721 raft_consensus.cc:2468] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:30.408413 14653 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:15:30.569739  8101 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:30.543478  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.570245  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:30.545485  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:30.441992 14665 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:30.600394  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.601735  8206 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 23: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:30.508224 14681 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:15:30.606745  8209 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 20: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:30.567265 14691 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.609944  8211 raft_consensus.cc:2393] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 21: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
W20260709 13:15:30.618533  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 206995us
I20260709 13:15:30.619513  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:15:30.622046  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.606925 14692 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.569898 14670 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.215s
I20260709 13:15:30.622562 14665 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:30.623229 14660 raft_consensus.cc:2749] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:30.700310  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:15:30.693595  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 59862us
W20260709 13:15:30.678195  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 111283us
I20260709 13:15:30.624810 14695 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:30.665545  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.667079 14687 leader_election.cc:290] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.647876 14691 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:30.710502  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:15:30.667364 14750 log.cc:927] Time spent T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:15:30.724647  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:30.611786 14671 raft_consensus.cc:2804] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:15:30.666538  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:30.724936  8207 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:30.541131 14663 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:30.689450  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.725863  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.732870 14671 raft_consensus.cc:697] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 23 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:30.733203 14691 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:30.733948 14671 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:30.735663 14691 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.736290 14479 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.738595  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.738777  8208 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 18: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:30.739269  8212 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:30.739784 14663 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:30.740936  8129 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:15:30.741173  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:30.742400  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:30.742554 14663 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:30.742893  8056 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:30.744995  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.745761  8203 raft_consensus.cc:2393] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 21: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:30.746500  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.746634  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.746913  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.747689  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.748490  8204 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
I20260709 13:15:30.748427  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.749362  8210 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 23: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:30.750221  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:30.751149 14665 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:30.751606  8207 raft_consensus.cc:1240] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 12. Current term is 13. Ops: []
W20260709 13:15:30.752058  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 389584us
I20260709 13:15:30.753479 14693 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.654780 14696 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.754215 14696 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:30.753909  8175 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:30.757396 14663 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:30.758271 14663 raft_consensus.cc:697] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:30.758407  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:30.759215 14655 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.759632 14721 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.759893  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:30.731395  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:30.747633  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.759547  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:30.763690  8063 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:30.766414  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:30.644608 14679 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.771401  8723 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:55826 (request call id 1183) took 1247 ms. Trace:
I20260709 13:15:30.661895 14684 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:30.774055  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:30.775975  8129 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:30.642946 14685 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:30.772959  8723 rpcz_store.cc:276] 0709 13:15:29.524087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.524197 (+   110us) service_pool.cc:224] Handling call
0709 13:15:30.760061 (+1235864us) spinlock_profiling.cc:243] Waited 86.3 ms on lock 0x7b54006aead4. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:30.771224 (+ 11163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":198496640}
I20260709 13:15:30.779134  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:30.779732  8721 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:30.781638  8137 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:15:30.775171  8057 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:30.685665 14692 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:30.785102  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:15:30.772536 14748 log.cc:927] Time spent T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:15:30.784705  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:30.761991 14665 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:30.786113  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.791576  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.759305 14663 consensus_queue.cc:237] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:30.772357 14721 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:30.784090  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.789675  8060 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:30.788578  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:30.790306  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:30.793699  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:30.797359  8055 raft_consensus.cc:2393] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 20: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:30.792263  8206 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:30.801012  8209 raft_consensus.cc:2368] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 19. Current term is 20.
I20260709 13:15:30.790503  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:30.805481  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:30.810123  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:30.811538 14721 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:30.813086 14693 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:30.813465 14693 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.813733 14721 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.813835 14693 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:30.819896 14692 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:30.822726 14693 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:30.825088 14693 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:30.793377 14575 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:30.826679 14575 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:30.828588 14575 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:30.830940  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.791069 14665 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:30.832362 14665 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:30.833860  8208 raft_consensus.cc:1275] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:15:30.834909  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.835635  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.835996  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:30.836728  8208 raft_consensus.cc:2393] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 19: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:30.836719  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:30.837831  8206 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 23: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:30.842396  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:30.842768  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:30.842981  8206 raft_consensus.cc:2368] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 22. Current term is 23.
I20260709 13:15:30.844412  8057 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 20: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:30.847689  8721 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:30.843523  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.852510  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:30.853288  8725 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:30.855614  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.856791  8056 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:30.856626  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:30.859447  8726 raft_consensus.cc:2393] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 21: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:15:30.860235  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:30.862408  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:15:30.861982  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 211140us
W20260709 13:15:30.864471  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:30.864784  8026 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.865455  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.865563  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.866294  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:30.866811  8212 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 18: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:30.866993  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:30.868788  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.870302  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:30.871070  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:30.872377  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.875334  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.876330 14732 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:30.876713 14732 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:30.877879  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.878484  8208 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 20.
I20260709 13:15:30.870034  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.879853  8130 raft_consensus.cc:2393] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 20: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:30.873328  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.881016  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.877677 14732 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:30.885108  8203 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:30.888291  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.889293  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.891842  8129 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:30.894165  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.894949  8729 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:30.895375  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.897411  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:30.897743  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 43750us
I20260709 13:15:30.898511 14086 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:30.899029  8102 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:30.900254  8102 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:30.903919 14693 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:30.894176  8210 raft_consensus.cc:2393] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 22: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:30.907660  8102 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:30.913177  8212 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:30.914222  8209 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:15:30.916735  8726 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:15:30.919123  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.920403  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:30.930786  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.937510 14732 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:30.906760 14575 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:15:30.922741 14671 raft_consensus.cc:2749] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:15:30.948047  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.957054  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.958101  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.948681  8029 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:30.960991  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:30.962064  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.965190  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.966007  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.966200  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:30.966750  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b4358ace86545339036f21610735b47" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.974411 14635 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:30.982852  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18de311592f1492b8b460305c56e6749" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.962064  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:30.989106  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.989048 14693 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:15:30.989930  8211 raft_consensus.cc:2368] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 19. Current term is 20.
I20260709 13:15:30.990796  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.992364  8205 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
I20260709 13:15:30.995628  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.995724  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:15:30.999279  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 50667.5us
I20260709 13:15:31.000492  8026 leader_election.cc:400] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 22. Current term is 23.
I20260709 13:15:31.002713  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:31.003293  8210 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:15:31.003490  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:31.004238  8207 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 19: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:31.004535  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b578d88c1c1f4ca58b7291fed4909f09" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:31.005321  8210 raft_consensus.cc:2368] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 18. Current term is 19.
I20260709 13:15:31.004261  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:31.006552  8206 raft_consensus.cc:2393] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 10: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:30.943400 14685 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:15:31.006033  8176 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:31.019696  8029 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:31.022272  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:31.025060  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:31.026889  8211 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 22. Current term is 23. Ops: []
I20260709 13:15:31.029839  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:30.998167 14635 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:31.034857  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:31.023913 14643 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:31.048441  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:15:31.050279  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 52558.6us
I20260709 13:15:31.024209 14696 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:31.029521 14086 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:30.999607 14665 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:31.052017  8101 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:31.034161 14661 raft_consensus.cc:2804] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:15:31.054903  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 117035us
I20260709 13:15:31.060859  8134 raft_consensus.cc:1240] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:15:31.062285  8029 leader_election.cc:400] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 22. Current term is 23.
I20260709 13:15:31.060652  8104 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.063252  8029 leader_election.cc:403] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:15:31.067457 14382 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:31.068004 14382 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:31.068424 14382 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:31.069151  8104 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:31.071939  8204 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:31.071748  8026 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.072713 14314 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 22. Current term is 23.
I20260709 13:15:31.073477 14314 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:31.076568  8026 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.077037 14635 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:31.081634 14665 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:31.059775 14718 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:31.082432 14718 raft_consensus.cc:493] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:31.082602  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:31.082774 14696 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:31.050057 14643 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:31.083675 14643 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:31.082834 14718 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:31.084087  8026 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:31.088333  8054 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:15:31.100670  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 73945.8us
I20260709 13:15:31.055389 14722 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:31.064061  8029 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:31.102533  8058 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 13:15:31.057616 14723 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:31.103654  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.105907  8101 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:15:31.108186  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44195.9us
I20260709 13:15:31.109040  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:31.111080  8029 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:31.114584 14692 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:31.114631 14754 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:15:31.105119 14723 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:31.116082 14723 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:31.114691 14696 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:31.088030 14661 raft_consensus.cc:697] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:15:31.127900  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 84368.6us
I20260709 13:15:31.104137 14656 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:31.122927 14732 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:31.129967 14691 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:31.130234 14732 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:31.130767 14732 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:31.130561 14661 consensus_queue.cc:237] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:31.131502  8696 leader_election.cc:400] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 18. Current term is 19.
I20260709 13:15:31.131793  8696 leader_election.cc:403] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:15:31.132238  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:31.132699  8696 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.133320 14440 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for earlier term 18. Current term is 19.
I20260709 13:15:31.136613  8696 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.137151  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:31.138352  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:31.105238 14685 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:31.104288 14722 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:31.135331  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:31.133893  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 49888.3us
I20260709 13:15:31.154115  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.122649 14696 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:31.157945  8026 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.163034  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:31.164023  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:31.166679  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:31.169656  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:31.135785 14666 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:31.155506 14670 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:31.189280 14666 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:31.190053 14657 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:31.165308 14440 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.026s
I20260709 13:15:31.190328 14666 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.157101 14685 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:31.164029 14722 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:31.163619 14754 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:31.163455 14696 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:31.128652 14668 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:15:31.185632 14687 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:31.203051  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 85412.2us
I20260709 13:15:31.203140  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:31.212468  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 59389.5us
I20260709 13:15:31.201037 14685 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:31.193037 14679 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:31.234580 14679 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:31.235158 14679 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:15:31.247079 14761 log.cc:927] Time spent T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:15:31.220688  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:31.221192  8104 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.286723 14687 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:31.293326  8101 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:31.295403 14655 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:31.321455  8693 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.322850 14685 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:31.333539  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 113453us
W20260709 13:15:31.334498  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 195998us
I20260709 13:15:31.306988 14776 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:15:31.346787  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 125725us
I20260709 13:15:31.347523 14774 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:31.356274 14691 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:31.356734 14691 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:31.357985  8104 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.359282  8104 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.359225 14687 consensus_queue.cc:1048] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:31.360435 14656 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:31.361336 14656 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:31.361382  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.362334 14656 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:31.353706  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:31.355533  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:31.364558  8058 raft_consensus.cc:2393] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 23: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:31.365567  8058 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:15:31.351177  7961 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37774 (request call id 146) took 4320 ms. Trace:
I20260709 13:15:31.363847  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:31.371450 14696 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.365420  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:31.367408  7961 rpcz_store.cc:276] 0709 13:15:27.030517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.031546 (+  1029us) service_pool.cc:224] Handling call
0709 13:15:31.350880 (+4319334us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:27.970843 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:28.120316 (+149473us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:28.125633 (+  5317us) write_op.cc:435] Acquired schema lock
0709 13:15:28.127940 (+  2307us) tablet.cc:662] Decoding operations
0709 13:15:28.178295 (+ 50355us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:28.184882 (+  6587us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:28.193748 (+  8866us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:15:28.247873 (+ 54125us) tablet.cc:701] Row locks acquired
0709 13:15:28.247893 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:15:28.269719 (+ 21826us) write_op.cc:270] Start()
0709 13:15:28.277280 (+  7561us) write_op.cc:276] Timestamp: P: 1783602928264848 usec, L: 0
0709 13:15:28.287545 (+ 10265us) op_driver.cc:348] REPLICATION: starting
0709 13:15:28.418491 (+130946us) log.cc:844] Serialized 533 byte log entry
0709 13:15:28.904576 (+486085us) op_driver.cc:464] REPLICATION: finished
0709 13:15:29.394049 (+489473us) write_op.cc:301] APPLY: starting
0709 13:15:29.422471 (+ 28422us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:29.636755 (+214284us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:29.644437 (+  7682us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:29.653334 (+  8897us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:29.653584 (+   250us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:15:29.653653 (+    69us) write_op.cc:312] APPLY: finished
0709 13:15:29.653891 (+   238us) log.cc:844] Serialized 25 byte log entry
0709 13:15:29.654102 (+   211us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:29.654208 (+   106us) write_op.cc:454] Released schema lock
0709 13:15:29.654263 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:29.654711 (+   448us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:29.654749 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1001836,"reported_tablets":3,"spinlock_wait_cycles":20736,"child_traces":[["op",{"apply.queue_time_us":467681,"apply.run_cpu_time_us":4708,"apply.run_wall_time_us":273187,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8307,"num_ops":1,"prepare.queue_time_us":62841,"prepare.run_cpu_time_us":4306,"prepare.run_wall_time_us":565838,"raft.queue_time_us":384584,"raft.run_cpu_time_us":5200,"raft.run_wall_time_us":322301,"replication_time_us":558115,"thread_start_us":394097,"threads_started":2,"wal-append.queue_time_us":17796,"wal-append.run_cpu_time_us":38654,"wal-append.run_wall_time_us":2213509}]]}
I20260709 13:15:31.395381  8054 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 20. Current term is 21. Ops: []
I20260709 13:15:31.362250 14732 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.410734  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:31.411970  8208 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 19: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:31.396039 14665 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:31.377010 14643 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.373874 14722 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.386054 14776 consensus_queue.cc:1059] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:15:31.419029  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:31.426736 14635 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:31.426826 14723 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.427434 14635 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:31.429302 14723 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:31.429589 14635 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:31.431238 14086 consensus_queue.cc:1059] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:15:31.433261 14668 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:31.433636 14668 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:31.434492 14668 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:31.437095 14382 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:31.438658  8057 raft_consensus.cc:1240] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:31.405085 14670 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.017s
I20260709 13:15:31.439563  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:31.440268 14661 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:31.439183  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:31.441685  8054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:31.443069 14643 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:31.404706 14754 consensus_queue.cc:1059] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:15:31.444078 14665 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:15:31.445621  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:31.446538 14661 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:31.447033 14661 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 21 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:31.446923 14656 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 20.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:31.447439 14653 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:15:31.447943 14656 raft_consensus.cc:3055] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:31.448443 14656 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:31.449054  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:31.449308 14656 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.8, Last appended by leader: 8, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.447937 14653 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:31.448043 14661 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.452361  8063 raft_consensus.cc:1240] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:31.456554  7957 catalog_manager.cc:5697] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:31.466886  8055 raft_consensus.cc:1240] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:31.396344 14718 raft_consensus.cc:515] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.471688  8060 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 23.
I20260709 13:15:31.459014  7957 catalog_manager.cc:5697] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 12 to 14, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 14 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:31.479522  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 73922.1us
I20260709 13:15:31.482125  8061 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:31.489432 14656 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:31.492727  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:31.493235  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.495097 14382 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:31.496012 14479 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:31.497046 14696 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:31.497586  8104 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.498353 14687 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:31.494901 14693 consensus_queue.cc:1059] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:15:31.500162  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 51099.6us
I20260709 13:15:31.500629  8104 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:31.500941  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 302420us
I20260709 13:15:31.504202  8175 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:15:31.480773 14771 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:15:31.514428  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:31.508701  8026 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.519344  8026 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.521534  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:31.525127 14721 raft_consensus.cc:2764] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:15:31.526319  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.526830 14721 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:31.527369 14721 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:31.528354  8026 leader_election.cc:400] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 22. Current term is 23.
I20260709 13:15:31.515466 14696 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:31.529278 14696 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.529687  8026 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.531347 14508 raft_consensus.cc:3055] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:31.531738 14508 raft_consensus.cc:740] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:31.531973 14778 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:15:31.532579  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 72018.1us
I20260709 13:15:31.532657 14508 consensus_queue.cc:260] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:31.499895 14777 raft_consensus.cc:3055] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:31.533667 14777 raft_consensus.cc:740] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:31.538215  8211 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 17: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:31.539506  8026 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.481226 14635 consensus_queue.cc:1059] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:15:31.517035 14722 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:31.548036 14778 raft_consensus.cc:697] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:31.544247  8104 leader_election.cc:304] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.545056  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:31.546106 14685 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:31.547138 14694 raft_consensus.cc:3055] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:15:31.535605 14440 consensus_queue.cc:1175] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" responder_term: 17 status { last_received { term: 16 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:15:31.528564 14314 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:31.543776 14655 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:31.551143  8104 leader_election.cc:304] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.551385 14777 consensus_queue.cc:260] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:31.552418 14657 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:31.562139 14693 raft_consensus.cc:2749] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:31.556021 14778 consensus_queue.cc:237] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:31.563609  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 59480.7us
I20260709 13:15:31.558243 14694 raft_consensus.cc:740] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:31.572497 14653 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:31.569208 14694 consensus_queue.cc:260] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.544216 14695 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:15:31.575925 14732 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:31.561380  7955 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34120 (request call id 250) took 20773 ms (client timeout 15000 ms). Trace:
I20260709 13:15:31.550884  8102 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:31.565596  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:31.579691  8026 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.582237 14782 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:31.583236 14723 raft_consensus.cc:2749] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:31.524011 14679 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.584877  8059 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 20. Current term is 21. Ops: []
I20260709 13:15:31.585884 14679 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:31.587370 14718 leader_election.cc:290] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:31.587118  7957 catalog_manager.cc:5697] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 8 to 13, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 13 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:31.587872 14668 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:31.589339  8104 leader_election.cc:400] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 19. Current term is 20.
I20260709 13:15:31.544725 14508 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:31.589655  8104 leader_election.cc:403] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:31.590668  8175 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:15:31.597127  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 57199us
I20260709 13:15:31.588979 14554 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:31.601418  7957 catalog_manager.cc:5697] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:31.606011  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:31.587932 14655 raft_consensus.cc:2749] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:31.601127 14479 raft_consensus.cc:3055] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:31.528378 14721 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.594590 14694 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:31.615068  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:31.611054  8175 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:31.611701 14643 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:31.617475  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:31.612555 14723 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:31.619496  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:31.600555 14718 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:31.623072  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:31.603240 14554 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:31.622587  7957 catalog_manager.cc:5697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:31.626046 14695 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:31.627321  8026 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.634475 14655 consensus_queue.cc:1059] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:31.627430 14086 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:31.633509  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:31.642553  8061 raft_consensus.cc:2393] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 17: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:31.614190 14653 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:31.609241 14685 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 19. Current term is 20.
I20260709 13:15:31.649382  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:31.637171  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:31.644546  8063 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:15:31.613173 14479 raft_consensus.cc:740] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:31.647344  8131 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:15:31.647900  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:31.641148 14643 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:31.652029  8133 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 18: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:31.652526  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:31.654712  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:31.608264 14691 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:31.635576 14554 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:31.661830  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:31.665699  8056 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:31.637451 14663 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:31.667264  8137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:31.681042  8131 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:31.681927  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:31.682783  8131 raft_consensus.cc:1240] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:31.684641  8136 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:15:31.685070  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:31.686050  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:31.687559  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69898.4us
I20260709 13:15:31.687395  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:31.688486  8104 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.645516 14696 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:31.693862  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b3101b6cc874539a962e23536fdabc9" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:31.686812  8133 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:31.692262  8130 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:31.698727  8058 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:31.698791  8059 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:15:31.662717 14732 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:31.708817  8135 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 21.
I20260709 13:15:31.645807 14723 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:31.712307 14479 consensus_queue.cc:260] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.713536 14479 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:31.714661 14661 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:31.716107  7957 catalog_manager.cc:5697] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 8 to 17, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 17 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:31.717846  8130 raft_consensus.cc:2468] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:31.718793 14685 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:31.717851  7957 catalog_manager.cc:5697] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:31.752907  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 65676.3us
I20260709 13:15:31.762514  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:15:31.768235  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 79848.7us
I20260709 13:15:31.770937  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:31.771795  8026 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.775789  7957 catalog_manager.cc:5697] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 9 to 17, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 17 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:31.746923 14681 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:15:31.744242 14732 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.780221  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:31.780535  8104 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.787150 14801 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:31.787923 14718 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:31.774994 14679 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:31.788648  8062 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:31.791706  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:15:31.579022  7955 rpcz_store.cc:269] 0709 13:15:10.787703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.837541 (+ 49838us) service_pool.cc:224] Handling call
0709 13:15:13.211021 (+2373480us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b5400172d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf80090 00007f6dbb01ef3a 00007f6dc0dd51cc 00007f6dc0ed337e 00007f6dbf3942a4 00007f6dbf394232 00007f6dbf394161 00007f6dbf3940dc 00007f6dbf393392 00007f6db935148c 00007f6db93508c6
0709 13:15:13.348740 (+137719us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b5400002210. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef2640 00007f6dc0dc9b76 00007f6dc0ed3c3e 00007f6dbf3942a4 00007f6dbf394232 00007f6dbf394161 00007f6dbf3940dc 00007f6dbf393392 00007f6db935148c 00007f6db93508c6
0709 13:15:31.561168 (+18212428us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:27.403682 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:27.404255 (+   573us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:27.404300 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:15:27.404336 (+    36us) tablet.cc:662] Decoding operations
0709 13:15:27.404719 (+   383us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:27.404784 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:27.404821 (+    37us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:15:27.405030 (+   209us) tablet.cc:701] Row locks acquired
0709 13:15:27.405053 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:15:27.405203 (+   150us) write_op.cc:270] Start()
0709 13:15:27.405334 (+   131us) write_op.cc:276] Timestamp: P: 1783602927405155 usec, L: 0
0709 13:15:27.405362 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:15:27.408992 (+  3630us) log.cc:844] Serialized 538 byte log entry
0709 13:15:27.410747 (+  1755us) op_driver.cc:464] REPLICATION: finished
0709 13:15:27.411147 (+   400us) write_op.cc:301] APPLY: starting
0709 13:15:27.411225 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:27.411977 (+   752us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:27.412061 (+    84us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:27.412734 (+   673us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:27.412879 (+   145us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:15:27.412923 (+    44us) write_op.cc:312] APPLY: finished
0709 13:15:27.413074 (+   151us) log.cc:844] Serialized 25 byte log entry
0709 13:15:27.413228 (+   154us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:27.413333 (+   105us) write_op.cc:454] Released schema lock
0709 13:15:27.413381 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:27.796159 (+382778us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b5400172d54. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb011e28 00007f6dbea4c581 00007f6dbea4bd58 00007f6dbea509f1 00007f6dbea509a9 00007f6dbea50939 00007f6dbea50901 00007f6dbea4fb2d 000000000038f354 000000000038f199
0709 13:15:27.818953 (+ 22794us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:27.825058 (+  6105us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1400695,"reported_tablets":51,"spinlock_wait_cycles":773871744,"child_traces":[["op",{"apply.queue_time_us":217,"apply.run_cpu_time_us":2536,"apply.run_wall_time_us":2535,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4180,"num_ops":1,"prepare.queue_time_us":36387,"prepare.run_cpu_time_us":16472,"prepare.run_wall_time_us":402809,"replication_time_us":5276,"spinlock_wait_cycles":7037312}]]}
I20260709 13:15:31.802603  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:31.810428  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:31.811353 14718 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:31.813378  8104 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:31.814471  8104 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
W20260709 13:15:31.816593 14791 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:15:31.819056 14691 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:31.820341 14723 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:31.820855 14723 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:31.823235  8104 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.822078 14723 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:15:31.824905  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:31.825721  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:31.802320 14766 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:15:31.826428 14695 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:31.831228  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet b578d88c1c1f4ca58b7291fed4909f09 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:50166
I20260709 13:15:31.832285  8725 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:15:31.834532  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:31.836699  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:31.809523 14661 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:31.839838  8026 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.841225  8026 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.847251 14732 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:31.851104  8729 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:31.855440  8062 raft_consensus.cc:1240] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer 965e59f272f143449b18f65db4e61d98 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:31.855400  8026 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.856244 14732 raft_consensus.cc:2749] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:15:31.855481 14695 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:15:31.859375 14732 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:31.859735 14732 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:31.860073 14732 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:31.862402 14656 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:31.864145 14798 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:31.868347  8026 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.870770  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:31.872867  8207 raft_consensus.cc:3055] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:31.873257  8207 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:31.873313  8205 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:15:31.839043 14666 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:31.874418  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:31.875021  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:31.868492 14765 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:31.880088  8207 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.884445  8207 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:31.868846 14695 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:31.872175 14784 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:31.886950  8175 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:15:31.888401  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 63616.6us
I20260709 13:15:31.889923  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:31.894654  8059 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:15:31.893610  8062 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:31.896944  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:31.897126  8134 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:31.899590  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:31.869268 14657 consensus_queue.cc:1059] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:15:31.886049 14605 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:31.882920 14666 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:31.902755 14440 raft_consensus.cc:3055] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:31.907564  8725 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:15:31.881922 14802 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:31.900753 14661 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:31.917323  8138 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 15.
I20260709 13:15:31.918808  8134 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:31.920975  8102 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:31.925271  8207 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:31.926824  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet b578d88c1c1f4ca58b7291fed4909f09 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:31.924367  8175 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
W20260709 13:15:31.940874  8694 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.943198  8026 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:31.944162 14779 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:31.944662 14779 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:31.945150 14779 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:31.918862 14440 raft_consensus.cc:740] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:31.946955 14440 consensus_queue.cc:260] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:31.948273 14440 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:31.958673 14666 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:31.960034 14795 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:15:31.960889 14801 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.961529 14801 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.932828 14685 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:15:31.964074 14801 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:31.965139 14777 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:31.922986 14718 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:15:31.974603  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 53711.4us
I20260709 13:15:31.974927  8175 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:31.927678 14802 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:31.972700  8027 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:31.972241 14695 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:15:31.981779 14783 consensus_queue.cc:1059] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:15:31.986307 14783 raft_consensus.cc:3055] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:31.986810 14783 raft_consensus.cc:740] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:31.987843 14783 consensus_queue.cc:260] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:31.990947  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.959918 14554 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:31.997872  8203 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:15:32.001292  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:32.001905  8131 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:32.003317  8131 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:32.004556  8026 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:31.979013 14655 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:32.011633 14661 consensus_queue.cc:1059] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:15:31.998333 14801 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:32.014636 14382 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:32.021993  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:32.022635  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:32.031621  8207 raft_consensus.cc:1240] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:31.993798 14718 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:31.992731 14695 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 23 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:32.039202  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:32.033824 14382 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.045452 14687 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.035761 14801 raft_consensus.cc:697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:32.030870 14605 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:15:32.056432 14783 raft_consensus.cc:3060] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:32.036736 14762 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:15:32.034101 14314 consensus_queue.cc:1059] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:15:32.061285 14382 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:32.056872 14732 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.115293  8026 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:32.076393 14687 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.108435 14554 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:32.081920 14802 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.8, Last appended by leader: 8, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.104290 14801 consensus_queue.cc:237] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:32.058380 14718 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
W20260709 13:15:32.113318  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 77663.1us
I20260709 13:15:32.118844  8058 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:15:32.168771  8129 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:15:32.169718  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
W20260709 13:15:32.171618  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 56402.9us
W20260709 13:15:32.180043  8028 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.184538 14779 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.185789  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:32.198897  8134 raft_consensus.cc:1275] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:15:32.191220  8029 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:32.203981 14779 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:32.204368 14818 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:32.201089 14775 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.126194 14695 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:32.188367 14314 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:32.207306 14775 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:32.207986 14314 raft_consensus.cc:697] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:32.208912 14314 consensus_queue.cc:237] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.211669 14687 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:32.213119 14732 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:32.220630  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:32.221421  8203 raft_consensus.cc:2410] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 22 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:15:32.209304 14775 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:32.174530 14668 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.188589 14814 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:32.233062  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 63766.2us
I20260709 13:15:32.241448  8203 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:15:32.244306  8725 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:32.231421 14668 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:32.257704 14811 log.cc:927] Time spent T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:15:32.257886 14668 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:32.266268  8209 raft_consensus.cc:1240] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 19. Current term is 20. Ops: []
W20260709 13:15:32.266937  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 76043.2us
I20260709 13:15:32.234826 14814 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:32.276006 14819 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:32.276599 14819 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:32.271512 14663 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:15:32.281239  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:32.282863  8135 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:15:32.285420  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:32.292354  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:32.293088  8135 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 21.
I20260709 13:15:32.293502  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:32.300868  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:32.301579  8131 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:32.303445  8133 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:15:32.318470  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:32.319474  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:32.319793 14382 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.323818  8175 leader_election.cc:304] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:32.299576 14693 raft_consensus.cc:3055] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:32.329895 14693 raft_consensus.cc:740] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:32.330582  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:32.331279  8208 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:15:32.330927 14693 consensus_queue.cc:260] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.331928 14693 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:32.333931 14813 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:32.334666 14819 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:32.291535 14732 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:15:32.335690 14822 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:32.293889 14685 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 13:15:32.329715 14820 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:15:32.326622 14783 consensus_queue.cc:1059] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:15:32.339968 14783 raft_consensus.cc:3055] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:32.340298 14783 raft_consensus.cc:740] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:32.341122 14783 consensus_queue.cc:260] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.341583  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:32.342453  8029 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:32.343690 14821 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:32.343762  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:32.344627 14821 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:32.345021 14821 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.345341 14821 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:32.348531 14382 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:32.349786  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:32.350777 14718 raft_consensus.cc:2804] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:32.351293 14718 raft_consensus.cc:697] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:32.352172 14718 consensus_queue.cc:237] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.353708  8131 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 20.
I20260709 13:15:32.355023  8056 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:15:32.353114 14605 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:15:32.361439  8061 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:15:32.367584  8059 raft_consensus.cc:1240] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:32.368621  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:32.298991 14661 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:15:32.375777  8063 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:32.390748 14661 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:32.393018  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:32.364935 14819 consensus_queue.cc:1059] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:15:32.405622  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:32.406281  8104 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:32.410408  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:32.427794  8026 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:32.429589 14382 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:32.429723  8026 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:32.451884  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:32.452625  8133 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:32.430186 14382 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.433890 14721 raft_consensus.cc:3055] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:32.405128 14813 consensus_queue.cc:1059] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:15:32.438045 14819 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:32.440791 14820 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:32.378782 14821 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:32.399381 14656 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 13:15:32.395812 14765 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:15:32.443481 14831 log.cc:927] Time spent T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:15:32.448269  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:15:32.450718  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44506.7us
I20260709 13:15:32.451982  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:32.431671 14830 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:32.454264 14709 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:32.479034 14813 raft_consensus.cc:3055] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:32.479413 14813 raft_consensus.cc:740] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:32.405426 14784 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.479951 14784 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:32.480273 14813 consensus_queue.cc:260] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.481417 14813 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:32.481662  8061 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 21.
I20260709 13:15:32.483544 14694 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:32.482066 14784 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:32.490737 14605 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:15:32.491479 14765 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:32.487399  8131 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 21.
I20260709 13:15:32.453246  8134 raft_consensus.cc:1240] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:32.489393 14709 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:32.493784  8061 raft_consensus.cc:1240] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 20. Current term is 21. Ops: []
I20260709 13:15:32.491410 14605 consensus_queue.cc:1048] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:32.485795  8104 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:32.499150  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:32.499924  8721 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 10.
I20260709 13:15:32.460930 14691 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:32.507105  8133 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:32.507834  8135 raft_consensus.cc:1240] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:15:32.454697 14687 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:15:32.539006  8131 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:32.478202 14814 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:32.511241  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:32.493119 14382 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:15:32.559803  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 44387.4us
I20260709 13:15:32.568282  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:32.569461  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:32.544237  8209 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:15:32.488333 14721 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:32.571589 14721 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.572762 14721 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:32.502869 14691 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:32.546576  8059 raft_consensus.cc:1240] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:32.487990 14819 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:32.575110 14691 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.575313 14819 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.576661 14819 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:32.539597 14479 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:32.553234  8026 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:32.580194 14830 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:32.580653 14830 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:32.581463 14830 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.584764  8135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:32.585372  8135 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 21.
I20260709 13:15:32.550289  8729 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 21.
I20260709 13:15:32.454453 14635 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.091s
I20260709 13:15:32.558167 14765 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.586942  8027 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:32.588029  8136 raft_consensus.cc:1240] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:32.458778 14762 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:32.607483 14605 consensus_queue.cc:1059] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:32.592199 14657 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:32.603204 14818 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:32.598471 14086 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:15:32.592555 14665 consensus_queue.cc:1059] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:15:32.638123 14818 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.606606 14762 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.623495 14766 consensus_queue.cc:1059] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
W20260709 13:15:32.703724  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 191847us
I20260709 13:15:32.722118  8723 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:15:32.705009 14818 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:15:32.736964  8175 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.713716 14382 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.782946  8060 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:15:32.742453 14314 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:15:32.896216  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:32.890657 14762 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:32.916236 14818 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:32.919166 14818 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:32.932626  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:32.933247  8723 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:32.934531 14858 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:32.939254 14858 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.941187 14858 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:32.944625  8723 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 22.
I20260709 13:15:32.945855  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:32.947448 14718 raft_consensus.cc:3055] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:32.947851 14718 raft_consensus.cc:740] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:15:32.948313  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 199425us
I20260709 13:15:32.953573  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:32.954103  8133 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:32.954157  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:32.956074  8138 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 20 index: 8. Preceding OpId from leader: term: 21 index: 9. (index mismatch)
I20260709 13:15:32.956629  8129 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:32.953048  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:32.966471 14314 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:32.966219 14718 consensus_queue.cc:260] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.966962 14314 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:32.968061 14718 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:32.956184 14687 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.973176 14687 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:32.974903 14314 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:32.975209 14687 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:32.977274 14821 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:32.978381 14845 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.367s	user 0.004s	sys 0.000s
I20260709 13:15:32.982419  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:32.982930  8205 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:32.984362  8204 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:32.985970  8134 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:15:32.987780  8207 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:15:32.993115  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:32.993682  8056 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
I20260709 13:15:32.997951  8133 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 22.
I20260709 13:15:33.003822  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:33.004587  8133 raft_consensus.cc:2393] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 15: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:32.998930 14086 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:15:33.019285 14657 raft_consensus.cc:493] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.019713 14657 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:32.998679 14732 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:15:33.023464 14696 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:15:33.038101 14382 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:33.039125  8205 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:15:33.040688  8204 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:33.050770  8056 raft_consensus.cc:1275] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:15:33.047534  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:33.061053  8101 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:33.066905  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:33.067570  8133 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 17: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:33.068734 14635 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:33.069154 14635 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.069499 14635 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:33.073455  8026 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:33.059950 14661 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:33.078917 14382 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:33.092278 14382 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:33.093433 14815 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:33.093230 14382 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:33.097548 14837 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.486s	user 0.000s	sys 0.002s
I20260709 13:15:33.095388  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:33.112504 14554 raft_consensus.cc:2749] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:33.115424 14765 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:33.105000 14657 raft_consensus.cc:515] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:33.150956  8178 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.133141 14554 consensus_queue.cc:1048] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:15:33.158134 14657 leader_election.cc:290] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:33.148294 14775 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.181847  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:33.184306  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf628848d7464a21aa8f76f7eb56a408" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:33.184834  8129 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:15:33.166556 14843 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.419s	user 0.004s	sys 0.000s
I20260709 13:15:33.197691 14853 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:33.201239  8129 raft_consensus.cc:2468] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 23.
I20260709 13:15:33.212102  8693 leader_election.cc:304] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:33.212684  8205 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:15:33.185913 14839 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.575s	user 0.004s	sys 0.001s
I20260709 13:15:33.217502  8212 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:33.218315  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:33.187662 14762 consensus_queue.cc:1059] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:15:33.226866 14844 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:33.227380 14844 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:33.236274  8212 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 20 index: 8. Preceding OpId from leader: term: 21 index: 9. (index mismatch)
I20260709 13:15:33.220075 14657 raft_consensus.cc:2804] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:15:33.246474  8029 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:33.202304 14775 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:33.236735 14844 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:33.253760  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:33.257795  8029 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:33.260102  8056 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 21: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:33.216167 14853 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:33.301050  8056 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:15:33.257326 14657 raft_consensus.cc:697] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 23 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:33.321847 14871 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.324882 14873 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.332782 14821 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:33.265983 14314 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:15:33.346397  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 98798.3us
I20260709 13:15:33.265048 14635 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:33.301548 14696 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:15:33.301321 14643 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:15:33.304544 14721 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:15:33.320858 14784 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.362574 14314 raft_consensus.cc:697] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:33.357990 14862 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
W20260709 13:15:33.398098 14876 log.cc:927] Time spent T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:15:33.399201 14819 raft_consensus.cc:3055] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:33.409080 14819 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:33.410002 14819 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:33.411330 14819 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:33.400893 14314 consensus_queue.cc:237] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:33.420518 14881 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.421069 14881 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:33.362769 14821 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:33.453111  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:33.357857 14873 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:33.386822 14784 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:33.468544  8207 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:15:33.469449  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:33.469995  8207 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 21.
I20260709 13:15:33.470891  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:33.472561  8207 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:33.349087 14871 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:33.483224  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:33.483819  8212 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:33.485612  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:33.448894 14657 consensus_queue.cc:237] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:33.481710 14881 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:33.481169 14775 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:33.482446 14693 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.493268 14670 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:15:33.507277 14846 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.894s	user 0.003s	sys 0.000s
I20260709 13:15:33.512353 14821 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:33.513172  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:33.516045 14852 log.cc:927] Time spent T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.689s	user 0.002s	sys 0.000s
I20260709 13:15:33.499758 14670 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.524629 14670 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:33.530104 14784 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:33.532791  7955 catalog_manager.cc:5697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 19 to 20, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 20 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:33.537569  7955 catalog_manager.cc:5697] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 13 to 14, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 14 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:33.535748 14775 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.540140 14871 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:33.542277  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:33.549862  8060 raft_consensus.cc:1240] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:15:33.550426  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:33.577570  8056 raft_consensus.cc:2393] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:33.497084 14693 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:33.552467 14873 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:33.495970 14853 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:33.518534  8133 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:15:33.561802  8104 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:33.561684 14847 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:33.571354 14775 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:33.572000 14816 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:33.556758  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:33.593645  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:33.597036  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:33.580606  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:33.585337  8059 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4489591b74b1469d81269da167e98f71"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:33.598719  8059 raft_consensus.cc:606] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:33.599022  8059 raft_consensus.cc:614] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:33.586220 14685 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:33.592033  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:33.600808  8721 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:33.597606  8058 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 16.
I20260709 13:15:33.598284  8723 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:33.603888  8129 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:33.610783  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:33.618010  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:33.623234  8723 raft_consensus.cc:2410] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for term 24 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 13:15:33.624336  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:33.624910  8060 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 23.
I20260709 13:15:33.625910  8132 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:33.629338  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:33.629930  8209 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:15:33.616907 14881 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:33.633215 14881 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:33.635162 14881 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:33.621415 14722 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 13:15:33.641192 14775 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:33.642758  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:33.621703  8027 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:33.645073  8027 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:33.641863 14665 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:15:33.651758  8209 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:33.652575  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:33.600217 14685 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:33.608153 14635 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:33.612046 14816 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:33.666850 14685 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:33.674431 14819 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:33.674906 14819 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:33.676193 14605 raft_consensus.cc:3055] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:15:33.676587 14605 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:33.676709 14819 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:33.654996 14778 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:33.677418 14605 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:33.679286  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 250840us
W20260709 13:15:33.681895  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.664362 14670 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:33.686173  8721 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 11.
I20260709 13:15:33.683534  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:33.674151 14783 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:33.712029  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:33.712627  8059 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:15:33.706887  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8454910b36474c90b678c57cea9773" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:33.713352  8062 raft_consensus.cc:3060] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:33.705788  8104 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:33.660028 14822 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:33.734758  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:33.761965  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:33.762374  8211 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 17.
I20260709 13:15:33.721081 14605 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:33.719816 14635 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.710615 14695 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.782380  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:33.783845  8721 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:33.721241 14881 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:15:33.776505  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 130091us
W20260709 13:15:33.777597  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 53746.3us
I20260709 13:15:33.778898  8062 raft_consensus.cc:2468] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:33.777132 14889 raft_consensus.cc:2804] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:15:33.785195  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 101731us
I20260709 13:15:33.733428 14822 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:33.782312  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:33.791824  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:33.792685  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:33.775226 14775 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:33.808686  8693 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:33.809326  8029 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:33.817962  8209 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:15:33.818917  8209 pending_rounds.cc:85] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Aborting all ops after (but not including) 10
I20260709 13:15:33.790710 14691 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.802827 14822 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:33.788556 14656 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:33.812681 14775 raft_consensus.cc:493] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.819265  8209 pending_rounds.cc:107] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:15:33.840270  8209 raft_consensus.cc:2889] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:33.786681 14635 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:33.819890 14718 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.814021 14889 raft_consensus.cc:697] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:33.782112 14784 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:15:33.849997  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:33.832274  8062 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:33.848577  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:33.829857 14885 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:33.788702 14695 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:33.795418 14881 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:33.857373  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:33.829721 14665 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:33.856617  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:33.838078 14653 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:15:33.851756  8062 raft_consensus.cc:606] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:33.842908 14775 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:33.860960 14653 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.872720 14653 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:33.872869 14670 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:33.874562 14890 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:15:33.852890 14718 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:33.854883 14314 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:15:33.888536  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:33.839795 14691 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:33.893718  8726 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 23 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 22. Current term is 23. Ops: []
I20260709 13:15:33.896588  8176 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:33.845891 14656 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:33.863868 14816 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:33.909540  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac625d2d87774bc7be4ff2189dee7eb0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:33.911986  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 89942.9us
I20260709 13:15:33.870074 14895 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:33.888706 14815 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:33.861099 14665 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:33.945758  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:33.958421  8060 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8dd7949df539418cbc00001111885883"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:33.959095  8060 raft_consensus.cc:606] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:15:33.941598 14895 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:33.970055  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 73582us
I20260709 13:15:33.943655 14721 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:33.971939  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 163323us
I20260709 13:15:33.943624 14693 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:33.953303 14816 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:33.970902  8176 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:33.987957  8726 raft_consensus.cc:1240] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:33.988330  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:33.940549 14889 consensus_queue.cc:237] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.034224  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:33.945248 14885 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 13:15:33.968714 14882 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:33.987051 14858 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:34.036042 14657 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:34.101245  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:15:34.081351  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 73080us
I20260709 13:15:34.037601 14820 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:33.991113 14721 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.052899 14653 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:34.054612 14695 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:34.058894 14892 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:15:34.130890  8721 raft_consensus.cc:1240] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 24 FOLLOWER]: Rejecting Update request from peer d9cd2999aa314b39997b423ec1ae509e for earlier term 23. Current term is 24. Ops: []
I20260709 13:15:34.070367 14665 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:34.072044 14635 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.071753 14822 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:34.133520 14653 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.133895 14822 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:34.135597 14896 log.cc:927] Time spent T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 13:15:34.075999  7955 catalog_manager.cc:5697] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:34.047434 14656 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:33.979964 14895 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:34.141139  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:34.138855  8058 raft_consensus.cc:2410] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:15:34.118472 14844 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.142792 14844 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.144747 14844 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:34.222723  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 99102.4us
I20260709 13:15:34.084719 14858 raft_consensus.cc:493] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:34.147076 14895 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:34.097455 14882 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.245218  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:34.115440 14657 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:34.228173 14858 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:34.235795 14718 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:34.274503  8696 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:34.239140 14899 consensus_queue.cc:1059] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:15:34.278095  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:34.245869 14820 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:34.287353  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:34.287935  8212 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 21.
I20260709 13:15:34.268791 14881 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:34.280200 14670 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:15:34.279227 14691 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.283973 14722 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:34.307585 14685 consensus_meta.cc:287] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.640s	user 0.008s	sys 0.002s
I20260709 13:15:34.308014 14685 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.310910 14685 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.312281 14895 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.314217  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:34.316371  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:34.316814  8134 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:15:34.316958  8129 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:34.317873  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:34.322036 14605 consensus_meta.cc:287] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.538s	user 0.004s	sys 0.004s
I20260709 13:15:34.325162 14657 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.347669 14721 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.328166 14695 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:34.295019 14885 consensus_queue.cc:1243] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 13:15:34.332415 14554 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.333039  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:34.334086 14665 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.340795 14722 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.294631 14687 raft_consensus.cc:3055] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:15:34.350652  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 236447us
I20260709 13:15:34.362814  8133 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:34.382009  8133 raft_consensus.cc:606] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:34.363272 14775 raft_consensus.cc:515] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:34.366201 14695 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.284629 14820 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.368616  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.368687  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.370474  8131 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 22. Current term is 23. Ops: []
I20260709 13:15:34.373395  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.374794  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
W20260709 13:15:34.367841 14783 consensus_meta.cc:287] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.637s	user 0.010s	sys 0.000s
I20260709 13:15:34.393399  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:34.393899  8061 raft_consensus.cc:2393] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 21: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:34.380162 14692 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.387563  8056 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 18.
I20260709 13:15:34.397543  8134 raft_consensus.cc:2393] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 20: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:34.377767 14687 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:34.399346  8723 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 18: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:15:34.400851  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:34.362862  7955 catalog_manager.cc:5697] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:34.405586 14907 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.098s	user 0.000s	sys 0.002s
I20260709 13:15:34.403447  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:34.386799 14914 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.376551 14554 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:34.419515  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 128185us
I20260709 13:15:34.421895  8729 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:15:34.379604 14665 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.398613 14732 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.410899 14692 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.453701  8029 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:34.457175 14917 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:34.457621 14917 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.457947 14917 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:34.458737  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.466828  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.467464  8056 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:34.433743 14605 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:34.470088  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:34.479305  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.479636  8134 raft_consensus.cc:2393] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 17: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:34.471410  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:34.479909  8056 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:34.480217  8138 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:34.473227 14665 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:34.475309 14904 log.cc:927] Time spent T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:15:34.468475  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.476230 14917 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.471494  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:34.490589  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:34.490764  8133 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 21.
I20260709 13:15:34.491142  8209 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:34.491411  8062 raft_consensus.cc:3055] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:15:34.491766  8062 raft_consensus.cc:740] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:34.492177  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:34.427256 14732 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.486006  8693 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:34.486585  8056 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:34.487421  8060 raft_consensus.cc:2393] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 17: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:15:34.485095  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 81720.4us
I20260709 13:15:34.489904  8207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:34.423739 14914 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.494753  8101 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:34.496132  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:34.499670  8056 raft_consensus.cc:606] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:34.500522  8207 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:34.483984 14382 consensus_queue.cc:1059] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:34.508291  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:34.511699  8134 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:34.478173 14605 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.516439  8026 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:34.521730  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.505525 14898 raft_consensus.cc:2749] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:15:34.525852  8057 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:34.528438 14775 leader_election.cc:290] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:34.529307 14695 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:34.488857 14813 consensus_queue.cc:1059] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.029s
I20260709 13:15:34.529803  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:34.552199  8133 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:34.537559 14643 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:34.539309 14692 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.541642  8026 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:34.546150 14722 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.437812 14687 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:34.512624 14685 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:34.554021  8209 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 21.
I20260709 13:15:34.519510 14605 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:34.505445 14895 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 13:15:34.496285 14909 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:15:34.567512  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:34.567766  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
W20260709 13:15:34.576598  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 141297us
I20260709 13:15:34.577215  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.560729 14643 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.584713 14643 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:34.571442 14685 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.595731  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:34.568704 14813 consensus_queue.cc:1243] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 22.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 13:15:34.583473  8134 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:34.571516 14382 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:34.603679 14914 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.604372 14554 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:34.583573  8131 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:34.596336  8209 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:34.597877  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:34.606164  8026 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:34.597142 14685 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:34.607247 14858 raft_consensus.cc:515] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:34.607987 14783 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:34.608355 14783 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.608676 14783 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:34.609304 14858 leader_election.cc:290] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:34.606537 14691 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:34.610404  8212 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:34.624325  8212 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:34.614465  8133 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:15:34.626155  8693 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:34.615222 14691 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.644843  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:34.644991  8208 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59868 (request call id 3984) took 1928 ms. Trace:
I20260709 13:15:34.657533  8205 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 17.
I20260709 13:15:34.636363 14762 raft_consensus.cc:2749] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:15:34.661901  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:34.643056 14775 raft_consensus.cc:2804] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:34.657460  8208 rpcz_store.cc:276] 0709 13:15:32.707414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.707988 (+   574us) service_pool.cc:224] Handling call
0709 13:15:33.539242 (+831254us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b5800168398. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef1a22 00007f6dc0e0ac09 00007f6dbff64ee0 00007f6dc002e0ab 00007f6dc002e131 00007f6dbffeffc0 00007f6dbffda929 00007f6dbffda3d2 00007f6db9551bd4
0709 13:15:33.539432 (+   190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.624406 (+ 84974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.624437 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:33.643777 (+ 19340us) log.cc:844] Serialized 26 byte log entry
0709 13:15:33.863121 (+219344us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:15:34.340703 (+477582us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b18000928c0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f293f3 00007f6db7f2b7c1 00007f6db7f2b779 00007f6db7f2b709 00007f6db7f2b6d1 00007f6db7f2a9cd 000000000038f354 000000000038f199 00007f6db7f0c83a
0709 13:15:34.546433 (+205730us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b18000928c0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dc0df1de1 00007f6dc0dfe5b1 00007f6db7f0be07 00007f6dc0d66ab9 00007f6db7f2869b 00007f6db7f2693f 00007f6db7f26455 00007f6dbea0e3c7 00007f6dbb0209a7 00007f6dbb0300bf 00007f6dbb030038
0709 13:15:34.634979 (+ 88546us) log.cc:844] Serialized 14 byte log entry
0709 13:15:34.635226 (+   247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.635294 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:34.635332 (+    38us) raft_consensus.cc:1685] finished
0709 13:15:34.635342 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.635452 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77634,"prepare.queue_time_us":23827,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":96439,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":1172,"spinlock_wait_cycles":23663872,"thread_start_us":224159,"threads_started":2,"wal-append.queue_time_us":198263}
I20260709 13:15:34.668712  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:34.660486 14691 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:34.673311  8724 raft_consensus.cc:2368] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:34.675913  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:34.676425  8132 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:34.683470 14643 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.683758 14882 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:34.686561 14643 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.688582  8062 consensus_queue.cc:260] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:34.694598  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:34.673926 14775 raft_consensus.cc:697] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 18 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:34.700242 14685 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.704813 14814 raft_consensus.cc:993] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
W20260709 13:15:34.708952  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 213448us
I20260709 13:15:34.711242 14917 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.712025 14783 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.730304  8205 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:34.730523 14679 raft_consensus.cc:1081] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:15:34.732582 14775 consensus_queue.cc:237] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:34.731950  8205 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
I20260709 13:15:34.736514  8133 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 19: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:34.744917 14783 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.745842 14685 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.747783  8058 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49222 (request call id 3776) took 1021 ms. Trace:
I20260709 13:15:34.748175  8058 rpcz_store.cc:276] 0709 13:15:33.726527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.761734 (+ 35207us) service_pool.cc:224] Handling call
0709 13:15:34.747606 (+985872us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b5400151154. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb01a2fa 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c 00007f6db95557c2 00007f6db935148c 00007f6db93508c6 00007f6db935026f
0709 13:15:34.747729 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864244480}
I20260709 13:15:34.748951  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:34.749809  8134 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:15:34.757762  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:34.759919  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:34.771502  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:34.799156  8056 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:34.799800  8056 raft_consensus.cc:606] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:34.811887  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:34.812484  8208 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:15:34.736150  8721 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8dd7949df539418cbc00001111885883"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55848
I20260709 13:15:34.819554  8721 raft_consensus.cc:493] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:34.773301  8136 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:34.820259  8721 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:34.822496  8176 leader_election.cc:400] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:34.823065  8176 leader_election.cc:403] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:15:34.766937  7955 catalog_manager.cc:5697] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 13 to 15, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 15 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:34.825012  7955 catalog_manager.cc:5697] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 18 to 20, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 20 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:34.826906  7955 catalog_manager.cc:5697] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 20 to 22, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 22 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:34.828425  7955 catalog_manager.cc:5697] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:34.766903 14635 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:34.829643 14635 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.835824 14605 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:34.837811 14605 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:34.838866  8210 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 19: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:34.840219 14661 raft_consensus.cc:3055] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:15:34.840598 14661 raft_consensus.cc:740] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:34.746174  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:34.841288 14635 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:34.841503  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 95407.6us
I20260709 13:15:34.841395 14661 consensus_queue.cc:260] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:34.841991  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:34.842718  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:34.843361 14722 raft_consensus.cc:493] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 965e59f272f143449b18f65db4e61d98)
I20260709 13:15:34.842686  8059 raft_consensus.cc:2393] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 18: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:34.843930  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:34.845078  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:15:34.845526 14919 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.251s	user 0.000s	sys 0.002s
I20260709 13:15:34.845630  8211 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:34.850243 14820 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:34.855868  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.856508  8061 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:34.859723  8060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.843765 14762 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:34.860787  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.860986 14661 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:15:34.825750 14691 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:34.864497  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 51281.7us
I20260709 13:15:34.864984  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1c85566831fd4f4283bc20a68e5ff448" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:34.874471  8058 raft_consensus.cc:3060] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:34.867230  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.871140  8208 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:34.866523  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51726a5d5cad44fd8e9b6803da998cb2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:34.877403  8059 raft_consensus.cc:3060] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:34.880375  8132 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 22.
I20260709 13:15:34.880636  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:34.881201  8208 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:34.881244  8211 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 22.
I20260709 13:15:34.884538  8205 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:15:34.893299  8134 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:34.896428  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:15:34.903680  8208 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 18.
I20260709 13:15:34.879329 14819 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:34.915594 14819 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:34.916009 14819 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:15:34.923074 14906 log.cc:927] Time spent T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.616s	user 0.000s	sys 0.003s
I20260709 13:15:34.930441  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.931510  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:34.932190  8061 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 19: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:15:34.938160  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 51415.5us
I20260709 13:15:34.901098  8026 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:15:34.910836  7957 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34134 (request call id 267) took 26772 ms (client timeout 15000 ms). Trace:
I20260709 13:15:34.910934 14691 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:34.879595 14858 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:34.886395 14762 raft_consensus.cc:2749] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573) with higher term. Message: Invalid argument: T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 18. Current term is 19.
I20260709 13:15:34.843881 14722 raft_consensus.cc:515] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:34.953760  8693 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:34.950117  8058 raft_consensus.cc:2468] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:34.902952  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:34.950829  8059 raft_consensus.cc:2468] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:15:34.944638  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:34.943727  8721 raft_consensus.cc:515] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:34.955333  8057 raft_consensus.cc:2393] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 22: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
W20260709 13:15:34.955633  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 55704.4us
W20260709 13:15:34.952808  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 50898.8us
I20260709 13:15:34.946858  8102 leader_election.cc:304] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:34.958945  8206 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59856 (request call id 2694) took 1166 ms. Trace:
W20260709 13:15:34.944455  7957 rpcz_store.cc:269] 0709 13:15:08.138115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.144535 (+  6420us) service_pool.cc:224] Handling call
0709 13:15:34.910766 (+26766231us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:31.981963 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:32.205917 (+223954us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:32.215267 (+  9350us) write_op.cc:435] Acquired schema lock
0709 13:15:32.215294 (+    27us) tablet.cc:662] Decoding operations
0709 13:15:32.216296 (+  1002us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:32.216411 (+   115us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:32.216449 (+    38us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:15:32.217363 (+   914us) tablet.cc:701] Row locks acquired
0709 13:15:32.217416 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:15:32.217597 (+   181us) write_op.cc:270] Start()
0709 13:15:32.217767 (+   170us) write_op.cc:276] Timestamp: P: 1783602932217538 usec, L: 0
0709 13:15:32.217811 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:15:32.270577 (+ 52766us) log.cc:844] Serialized 3234 byte log entry
0709 13:15:32.401377 (+130800us) op_driver.cc:464] REPLICATION: finished
0709 13:15:32.564446 (+163069us) write_op.cc:301] APPLY: starting
0709 13:15:32.564551 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:32.570207 (+  5656us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:32.616384 (+ 46177us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:32.622312 (+  5928us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:32.622737 (+   425us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:15:32.622813 (+    76us) write_op.cc:312] APPLY: finished
0709 13:15:32.739625 (+116812us) log.cc:844] Serialized 81 byte log entry
0709 13:15:32.752878 (+ 13253us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:32.753034 (+   156us) write_op.cc:454] Released schema lock
0709 13:15:32.753094 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:32.753471 (+   377us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:32.753499 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":75418,"reported_tablets":51,"spinlock_wait_cycles":9984,"child_traces":[["op",{"apply.queue_time_us":162746,"apply.run_cpu_time_us":13805,"apply.run_wall_time_us":189037,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37567,"num_ops":8,"prepare.queue_time_us":127877,"prepare.run_cpu_time_us":17742,"prepare.run_wall_time_us":327458,"raft.queue_time_us":23532,"raft.run_cpu_time_us":34967,"raft.run_wall_time_us":135659,"replication_time_us":183375,"spinlock_wait_cycles":22912,"thread_start_us":198998,"threads_started":4,"wal-append.queue_time_us":28607,"wal-append.run_cpu_time_us":15053,"wal-append.run_wall_time_us":1467715}]]}
I20260709 13:15:34.965503 14722 leader_election.cc:290] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:34.966215  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 65197.2us
I20260709 13:15:34.970638  8026 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:34.966698 14910 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:34.969520  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:34.974179  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:34.980631  8129 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:34.974141  8693 leader_election.cc:304] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:34.959820  8206 rpcz_store.cc:276] 0709 13:15:33.792602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.792761 (+   159us) service_pool.cc:224] Handling call
0709 13:15:33.801866 (+  9105us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.830139 (+ 28273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.830185 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:33.842734 (+ 12549us) log.cc:844] Serialized 26 byte log entry
0709 13:15:34.267914 (+425180us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:34.282010 (+ 14096us) log.cc:844] Serialized 14 byte log entry
0709 13:15:34.282225 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.282289 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:34.922584 (+640295us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4400022b4c. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7e4fc85 00007f6db7e506a2 00007f6dbabc21db 00007f6dbabbdaaf 00007f6dbabb186e 00007f6dbabb0bb6 00007f6dbabb07d8 00007f6dbabba221 00007f6dbabba1d9 00007f6dbabba169
0709 13:15:34.929574 (+  6990us) raft_consensus.cc:1685] finished
0709 13:15:34.958766 (+ 29192us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.958880 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41163,"prepare.queue_time_us":41031,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":28023,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":11505,"spinlock_wait_cycles":26285312,"thread_start_us":418506,"threads_started":1,"wal-append.queue_time_us":458016}
I20260709 13:15:35.000867  8721 leader_election.cc:290] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:34.956132 14903 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.957960 14653 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:35.004130  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:35.004364  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dd7949df539418cbc00001111885883" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:35.009287  8057 raft_consensus.cc:3055] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:35.009392  8134 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:35.009832  8057 raft_consensus.cc:740] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:35.000344  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:35.018044  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:34.990374 14635 raft_consensus.cc:2749] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:34.991367 14916 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:34.996004 14670 raft_consensus.cc:2804] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:34.997182 14910 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:35.024943 14867 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:15:35.032474  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 93669.4us
I20260709 13:15:35.033629  8029 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:35.022743 14554 raft_consensus.cc:2764] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:15:35.023867  8134 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:35.026522  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:35.039927  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:35.055341  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:35.056895  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:35.032549 14867 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.016917 14903 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.066314  8132 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:35.036245 14670 raft_consensus.cc:697] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:35.036286 14916 raft_consensus.cc:697] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:35.040194 14814 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:35.061035 14762 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:35.080374  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:35.082753 14670 consensus_queue.cc:237] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:35.066509 14858 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:35.070091 14867 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:35.087734  8693 leader_election.cc:304] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:35.088527  8059 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:15:35.094806 14930 log.cc:927] Time spent T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:15:35.095508 14932 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:15:35.104876 14873 raft_consensus.cc:2804] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:35.105398 14873 raft_consensus.cc:697] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:35.095835 14910 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.106264 14873 consensus_queue.cc:237] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:35.107136 14732 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:35.098462 14903 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:35.100766 14917 raft_consensus.cc:993] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:35.090612 14765 raft_consensus.cc:2749] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:35.072520 14314 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:15:35.114089  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:35.114791  8059 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:35.116034 14917 raft_consensus.cc:1081] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:35.117537 14903 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:35.110333 14916 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.118614 14819 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.119648  8101 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:35.120383 14903 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:35.120906 14903 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.121317 14903 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:35.090188 14814 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:35.127919 14814 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:35.132336  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 59173.6us
I20260709 13:15:35.136031 14679 raft_consensus.cc:993] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:35.136444 14679 raft_consensus.cc:1081] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
W20260709 13:15:35.142648  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 51140.7us
I20260709 13:15:35.143931  8129 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:35.145802  8026 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:35.154693 14762 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.155206 14762 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:35.159598  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:35.165882  8212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:35.169064  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:35.169672  8203 raft_consensus.cc:2393] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 19: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:35.120580 14819 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:35.180084  8208 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59856
I20260709 13:15:35.181324  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8fabe9526434f818c106b17c57b3b85" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:35.183171  8203 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59856
I20260709 13:15:35.183825  8203 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:35.184232  8203 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:35.187419  8058 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:15:35.157281 14679 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:35.188695 14679 raft_consensus.cc:697] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:35.188935  8723 raft_consensus.cc:1217] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 23 LEADER]: Deduplicated request from leader. Original: 20.15->[22.16-22.16]   Dedup: 22.16->[]
I20260709 13:15:35.189451  8723 raft_consensus.cc:1240] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 23 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 13:15:35.189543 14853 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.189072 14643 raft_consensus.cc:993] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:35.190421 14643 raft_consensus.cc:1081] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:35.190100 14853 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.180630  8208 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:35.192615  8208 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:35.200892  8208 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.209651  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:35.205426  8057 consensus_queue.cc:260] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:35.212915  8057 raft_consensus.cc:3060] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:15:35.213691 14314 consensus_queue.cc:1175] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8a968cada1e4446ebb2a5e167176e649" responder_term: 19 status { last_received { term: 18 index: 15 } last_committed_idx: 15 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:15:35.203867 14894 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 13:15:35.222312 14762 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:35.222620 14903 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.224639 14903 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:35.224783 14687 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:35.225569  8134 raft_consensus.cc:2368] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 20. Current term is 21.
I20260709 13:15:35.230298  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:35.231420  8062 raft_consensus.cc:3060] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:35.232563 14853 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:35.237890  8062 raft_consensus.cc:1275] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 13:15:35.238298  8062 pending_rounds.cc:85] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting all ops after (but not including) 15
I20260709 13:15:35.238593  8062 pending_rounds.cc:107] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 13:15:35.238875  8062 raft_consensus.cc:2889] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:35.236614  8104 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:35.243040  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:35.243781  8129 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:35.245592  8208 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:35.246250  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:35.246810  8061 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:15:35.245191 14784 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
I20260709 13:15:35.245257 14643 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:35.247951  8062 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37156 (request call id 1130) took 1108 ms. Trace:
I20260709 13:15:35.248175 14643 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.248420  8062 rpcz_store.cc:276] 0709 13:15:34.131130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.133759 (+  2629us) service_pool.cc:224] Handling call
0709 13:15:34.134036 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:34.491318 (+357282us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54004a2520. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf8ddc0 00007f6dbb0181c0 00007f6dbb019231 00007f6dbb015d8f 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62 00007f6db9551a91 00007f6db9551a0c
0709 13:15:34.688269 (+196951us) spinlock_profiling.cc:243] Waited 44.7 ms on lock 0x7b54004a2520. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf9406f 00007f6dbb00cf60 00007f6dbb00e7de 00007f6dbb018b49 00007f6dbb019416 00007f6dbb015d8f 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4 00007f6db9551b62
0709 13:15:35.219595 (+531326us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b580067d0d0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf720aa 00007f6dbaff4447 00007f6dbb00cfe7 00007f6dbb00e7de 00007f6dbb018b49 00007f6dbb019416 00007f6dbb015d8f 00007f6dbb01583e 00007f6dbffda419 00007f6db9551bd4
0709 13:15:35.230946 (+ 11351us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b54004a2520. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf8ba1d 00007f6dbaf721a4 00007f6dbaff4447 00007f6dbb00cfe7 00007f6dbb00e7de 00007f6dbb018b49 00007f6dbb019416 00007f6dbb015d8f 00007f6dbb01583e 00007f6dbffda419
0709 13:15:35.239526 (+  8580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.239748 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1027001984}
I20260709 13:15:35.250382 14657 consensus_queue.cc:1048] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:35.251991  8104 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:35.252920 14655 raft_consensus.cc:2764] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:15:35.189611 14679 consensus_queue.cc:237] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:35.256989  8176 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.247368  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:35.258793  8134 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 21.
I20260709 13:15:35.279392  8175 leader_election.cc:400] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 20. Current term is 21.
I20260709 13:15:35.280186  8175 leader_election.cc:403] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:15:35.191995 14867 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.255685  8026 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:35.285776  8057 raft_consensus.cc:2468] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:35.288340  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:35.289752  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:35.250219 14643 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:35.293706  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:35.294612  8062 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 22.
I20260709 13:15:35.296391  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:35.296921  8062 raft_consensus.cc:3055] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:35.297254  8062 raft_consensus.cc:740] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:35.298604  8208 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59856
I20260709 13:15:35.299331  8208 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:35.299741  8208 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:35.283924  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:35.312340  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dad90abaa8b04508a5b086ff94502e4e" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:35.313009  8134 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:35.306552  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:35.333320  8210 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:35.340109  8212 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 21: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:35.348845  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:35.350751  8029 leader_election.cc:304] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:35.353394  8062 consensus_queue.cc:260] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.283340 14819 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:35.348035  8134 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
W20260709 13:15:35.356668  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 65483.7us
I20260709 13:15:35.353196  8029 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:35.357602  8104 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:15:35.358121  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 76439us
I20260709 13:15:35.361625 14382 raft_consensus.cc:2749] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:35.372241 14722 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:35.372679 14722 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:35.373113 14722 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:35.290779 14911 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:35.375926 14911 raft_consensus.cc:493] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.376346 14911 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:35.336288 14663 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:15:35.386785 14663 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.387197 14663 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:35.390511 14867 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:35.341151 14765 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:35.402753 14765 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:35.407193  8129 raft_consensus.cc:1275] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:15:35.414685  8061 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 21.
I20260709 13:15:35.419796  8062 raft_consensus.cc:3060] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:35.419507  8203 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:35.458923  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 110296us
I20260709 13:15:35.460119  8203 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:35.460814  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:35.461755  8208 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:35.465288  8062 raft_consensus.cc:2468] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 13.
I20260709 13:15:35.448033 14914 raft_consensus.cc:2764] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader election decision vote started in defunct term 21: won
I20260709 13:15:35.485795  8729 raft_consensus.cc:1240] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:35.483549  8175 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:35.492813  8208 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:35.463296 14891 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:35.509083  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:35.510172  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:35.510732  8057 raft_consensus.cc:3055] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:15:35.511107  8057 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:35.511754  8057 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.9, Last appended by leader: 9, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.513862  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:35.514065  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:35.515007  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec8da6c869684688957d5323fb023bc5" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:35.515450  8138 raft_consensus.cc:3060] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:15:35.515746  8129 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:35.455107 14775 consensus_queue.cc:1048] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:15:35.510646  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:35.518865  8063 raft_consensus.cc:3055] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:35.519181  8063 raft_consensus.cc:740] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:35.531488  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:35.532570  8723 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 20: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:15:35.533540  8138 raft_consensus.cc:2468] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 21.
I20260709 13:15:35.489266 14635 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:35.498544 14685 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:35.541862  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:35.544359  8725 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:35.535483  8063 consensus_queue.cc:260] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:35.554781  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:35.555565  8138 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 19: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:35.520865 14820 raft_consensus.cc:2804] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:35.568097  8063 raft_consensus.cc:3060] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:35.570050 14936 log.cc:927] Time spent T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:15:35.585379  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 225356us
I20260709 13:15:35.593190  8129 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 22.
I20260709 13:15:35.603178  8138 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 21. Current term is 22. Ops: []
I20260709 13:15:35.578600 14820 raft_consensus.cc:697] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:35.548975 14635 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.586298  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:15:35.625742  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 67049.2us
I20260709 13:15:35.597211 14661 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:35.629886  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 43684.9us
I20260709 13:15:35.631397  8206 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:15:35.633142  8206 raft_consensus.cc:1275] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:15:35.637228  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:35.638212  8203 raft_consensus.cc:2393] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 19: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:35.639815  8175 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:35.661212  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:35.662766  8136 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:15:35.662950 14554 raft_consensus.cc:2804] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:35.663421  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:35.663754  8175 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:35.665100  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:35.674937 14661 raft_consensus.cc:2804] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:35.675489 14661 raft_consensus.cc:697] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:35.676422 14661 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.9, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.681241 14635 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:35.681478 14940 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:15:35.686403 14670 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:35.688154 14820 consensus_queue.cc:237] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.693563 14819 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e for earlier term 20. Current term is 21.
I20260709 13:15:35.694240  8203 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 21.
I20260709 13:15:35.663414 14554 raft_consensus.cc:697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 21 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:35.697412 14554 consensus_queue.cc:237] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.711608  8058 raft_consensus.cc:1275] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:15:35.712883  8725 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:15:35.713668  8729 raft_consensus.cc:1240] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:35.668043 14914 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:35.714821  8176 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:35.716835 14663 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.719807 14722 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.715292  8063 raft_consensus.cc:2468] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 14.
I20260709 13:15:35.722497  8059 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:15:35.722759  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 57157us
I20260709 13:15:35.727125  8130 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:15:35.744940  8212 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:15:35.716269 14914 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.749244  8206 raft_consensus.cc:2368] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 22. Current term is 23.
I20260709 13:15:35.725387 14949 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:15:35.757113  8130 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:35.763569  8211 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:35.744284 14661 raft_consensus.cc:2804] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:35.766059  8101 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:35.721581 14670 consensus_queue.cc:1048] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 13:15:35.750291 14914 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:15:35.770259  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 45570.9us
I20260709 13:15:35.770259  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:35.769955 14953 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:35.774924 14663 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 23 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:35.776465 14661 raft_consensus.cc:697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:15:35.831835  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 52779.5us
I20260709 13:15:35.777190 14784 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
W20260709 13:15:35.897372 14687 consensus_meta.cc:287] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Time spent flushing consensus metadata: real 0.672s	user 0.018s	sys 0.009s
I20260709 13:15:35.899329 14661 consensus_queue.cc:237] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:35.830617 14953 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:15:35.863677 14948 log.cc:927] Time spent T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.239s	user 0.002s	sys 0.002s
I20260709 13:15:35.928547  8134 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
W20260709 13:15:35.896515 14946 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
W20260709 13:15:35.930104  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 159001us
W20260709 13:15:35.927739 14939 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.330s	user 0.008s	sys 0.003s
I20260709 13:15:35.935421  8104 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:35.927141 14653 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.961747  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
W20260709 13:15:35.968115 14963 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:15:35.969224 14953 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.956471 14722 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:35.980644  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 49422.8us
I20260709 13:15:35.951012 14653 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:35.984004  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 48669us
I20260709 13:15:35.984233 14687 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:35.984889 14653 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:35.985039 14968 consensus_queue.cc:1048] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:15:35.990103 14911 consensus_meta.cc:287] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Time spent flushing consensus metadata: real 0.613s	user 0.009s	sys 0.002s
I20260709 13:15:35.990665 14911 raft_consensus.cc:515] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:35.992161  8056 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 21: Already voted for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in this term.
I20260709 13:15:35.997779 14914 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:36.000306  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:36.000862 14914 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:36.004669 14643 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:36.005419 14643 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:36.006798  8211 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:36.008036 14643 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:36.010528  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 79783.8us
I20260709 13:15:36.012907 14679 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:36.015211  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:36.032387  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:36.033084  8063 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:36.044215  8063 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:36.045614 14911 leader_election.cc:290] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:36.047931  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 64064.9us
I20260709 13:15:36.052951  8138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:36.053808  8138 raft_consensus.cc:2368] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 19. Current term is 20.
I20260709 13:15:36.054630  8131 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:15:36.054157  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:36.059340  8131 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:15:36.060292  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 227342us
I20260709 13:15:36.064105  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:36.064797  8062 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:36.072079  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ee839503bc34e5fb2b1f4e09dfe9784" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:36.072885  8132 raft_consensus.cc:2393] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 21: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:36.074178  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:36.076922  8211 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:15:36.079082  8026 leader_election.cc:400] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 19. Current term is 20.
I20260709 13:15:36.079483  8026 leader_election.cc:403] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:15:36.084822  8062 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 23.
I20260709 13:15:36.089030  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:36.089708  8132 raft_consensus.cc:3060] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:15:36.088246  8175 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:36.103418  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:36.104177  8211 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:36.061275 14663 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:36.124332  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:36.133512  8211 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:36.109385 14814 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807) with higher term. Message: Invalid argument: T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 19. Current term is 20.
I20260709 13:15:36.144815  8132 raft_consensus.cc:2468] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 23.
I20260709 13:15:36.108193 14554 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:36.126003 14958 raft_consensus.cc:2804] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:15:36.119130 14784 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:15:36.120712 14691 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:15:36.159680 14765 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:36.160213 14554 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:36.135006 14663 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:36.163313 14663 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:36.144021 14980 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:36.180867 14964 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:36.181631 14964 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:36.182157  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 76782.7us
I20260709 13:15:36.184669 14964 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:36.188287  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:36.192744 14889 raft_consensus.cc:993] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:36.193130  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:36.193753  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:36.194998  8132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:36.195300  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:36.198988  8175 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:36.199738 14891 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:36.200186  8175 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:36.200956 14891 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:36.169988 14958 raft_consensus.cc:697] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 23 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:36.174664 14980 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:36.209287 14958 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:36.210666 14980 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:36.211529  8027 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:36.203980 14979 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:15:36.174536 14554 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:36.231819  8104 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:36.233467 14982 raft_consensus.cc:2749] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:36.220671 14889 raft_consensus.cc:1081] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:36.234589  8104 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:36.245510  8130 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:15:36.250867  8104 leader_election.cc:400] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 22. Current term is 23.
I20260709 13:15:36.251212  8104 leader_election.cc:403] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:36.253491  8058 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:36.259327 14954 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:36.260028 14954 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:36.261154 14954 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:36.272771 14944 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.675s	user 0.003s	sys 0.000s
I20260709 13:15:36.281649 14889 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 for earlier term 22. Current term is 23.
W20260709 13:15:36.286279  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 180799us
W20260709 13:15:36.289341  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 208171us
I20260709 13:15:36.292775  8209 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59868
I20260709 13:15:36.293329  8209 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:36.293704  8209 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:36.294515  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:36.295413  8204 raft_consensus.cc:2410] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 for term 25 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 18.
I20260709 13:15:36.249257 14891 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:36.298152 14891 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:36.300318 14891 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:36.302269 14816 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:36.304323  8211 raft_consensus.cc:1275] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:15:36.297634  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:36.309370  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:36.319854  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:36.331537  8693 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:36.338776  8132 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:36.342654  8138 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 21 index: 9. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
I20260709 13:15:36.343765  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cbbc2cb4def4ea09659a9cdc5087f65" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:36.346256 14993 consensus_queue.cc:1048] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:15:36.345578  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:36.334012  8209 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:36.345077 14657 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:15:36.363763  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:36.364056  8209 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:36.364347  8131 raft_consensus.cc:3055] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:36.364652  8131 raft_consensus.cc:740] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:36.360093 14765 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:36.343917 14663 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:15:36.368639  8131 consensus_queue.cc:260] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:36.348110 14816 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:15:36.377136 14985 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:15:36.378852  8131 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:36.387897  8131 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:36.393983  8059 raft_consensus.cc:1275] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:15:36.407256  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:36.410508  8175 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:36.410574 14765 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:36.411116 14765 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:36.411499 14765 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:36.433547 14661 raft_consensus.cc:2804] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:36.434119 14661 raft_consensus.cc:697] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:36.431437 14985 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.447165 14985 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:36.452528 14985 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:36.435081 14661 consensus_queue.cc:237] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:36.462432 14911 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:36.463644 14775 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:36.464214 14775 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:36.464901 14692 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:36.468621 14775 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:36.469969 14822 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:36.470669  8029 leader_election.cc:304] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:36.471570 14917 raft_consensus.cc:2749] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:15:36.473133  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:36.473412  8029 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:36.474228 14885 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:36.475621 14885 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:36.477124 14643 consensus_queue.cc:1059] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: INVALID_TERM, Last received: 21.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:36.479737  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:36.480384  8730 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 24.
I20260709 13:15:36.484421  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3e2b86010604477955f19a56daf6d04" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:36.485028  8721 raft_consensus.cc:3060] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:36.486796  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:36.487138 14657 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:36.487726 14657 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:36.490191 14657 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:36.379102  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:36.509979  8057 raft_consensus.cc:3060] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:36.510680  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:36.516054 14783 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:36.516530 14783 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:36.519477 14783 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:36.520310  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:36.532891  8057 raft_consensus.cc:2468] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 22.
I20260709 13:15:36.533286  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4fa00ecc35497e823a1329fe771042" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:36.535485  8057 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49210 (request call id 2339) took 1026 ms. Trace:
I20260709 13:15:36.535794  8057 rpcz_store.cc:276] 0709 13:15:35.509474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.509949 (+   475us) service_pool.cc:224] Handling call
0709 13:15:36.474958 (+965009us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b180016f320. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbb051646 00007f6dbaf8b58e 00007f6dbb00cf9e 00007f6dbb00e7de 00007f6dbb01a27c 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c
0709 13:15:36.509775 (+ 34817us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b58005afbd0. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6dbaf7f571 00007f6dbaf720aa 00007f6dbaff4447 00007f6dbb00cfe7 00007f6dbb00e7de 00007f6dbb01a27c 00007f6dbffdbb17 00007f6db95566d4 00007f6db9556662 00007f6db9556591 00007f6db955650c
0709 13:15:36.535435 (+ 25660us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1622381056}
I20260709 13:15:36.540012  8206 raft_consensus.cc:1240] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 21. Current term is 22. Ops: []
I20260709 13:15:36.548735  8721 raft_consensus.cc:2468] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
W20260709 13:15:36.553859  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 195447us
I20260709 13:15:36.555905  8729 raft_consensus.cc:1240] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:36.565157  8176 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:36.572479  8176 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:36.526880 14765 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:36.550559 14985 consensus_queue.cc:1048] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:15:36.578392  8058 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:15:36.579553  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:36.580763  7955 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37774 (request call id 147) took 4280 ms. Trace:
I20260709 13:15:36.581128  7955 rpcz_store.cc:276] 0709 13:15:32.325043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.325213 (+   170us) service_pool.cc:224] Handling call
0709 13:15:36.580707 (+4255494us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:35.100044 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:35.100593 (+   549us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:35.100627 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:15:35.100644 (+    17us) tablet.cc:662] Decoding operations
0709 13:15:35.101499 (+   855us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:35.101570 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:35.101607 (+    37us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:15:35.102277 (+   670us) tablet.cc:701] Row locks acquired
0709 13:15:35.102296 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:15:35.126440 (+ 24144us) write_op.cc:270] Start()
0709 13:15:35.126555 (+   115us) write_op.cc:276] Timestamp: P: 1783602935126385 usec, L: 0
0709 13:15:35.126578 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:15:35.127199 (+   621us) log.cc:844] Serialized 3190 byte log entry
0709 13:15:35.961419 (+834220us) op_driver.cc:464] REPLICATION: finished
0709 13:15:36.156117 (+194698us) write_op.cc:301] APPLY: starting
0709 13:15:36.156233 (+   116us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:36.222731 (+ 66498us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:36.227740 (+  5009us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:36.260937 (+ 33197us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:36.261217 (+   280us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:15:36.261276 (+    59us) write_op.cc:312] APPLY: finished
0709 13:15:36.261601 (+   325us) log.cc:844] Serialized 81 byte log entry
0709 13:15:36.262587 (+   986us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:36.262725 (+   138us) write_op.cc:454] Released schema lock
0709 13:15:36.262777 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:36.264609 (+  1832us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:36.264641 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1137158,"reported_tablets":11,"spinlock_wait_cycles":1408,"child_traces":[["op",{"apply.queue_time_us":191920,"apply.run_cpu_time_us":14958,"apply.run_wall_time_us":108484,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37567,"num_ops":8,"prepare.queue_time_us":164096,"prepare.run_cpu_time_us":36691,"prepare.run_wall_time_us":355081,"raft.queue_time_us":255571,"raft.run_cpu_time_us":34850,"raft.run_wall_time_us":174987,"replication_time_us":834663,"thread_start_us":710637,"threads_started":4,"wal-append.queue_time_us":477054}]]}
I20260709 13:15:36.585886  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:36.588267  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:36.589632 14853 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:15:36.595590  8029 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:36.597774 14814 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:36.572993 14816 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:15:36.580626 14956 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:36.607306  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:36.615972  8693 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:36.600525 14816 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:36.615070 14873 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:36.675588 14816 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:15:36.746140  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 130213us
I20260709 13:15:36.700278 14949 raft_consensus.cc:2749] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:36.748348 15006 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:15:36.864560 14765 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:36.867067 14999 log.cc:927] Time spent T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:15:36.870843  8063 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:15:36.871977 14979 log.cc:927] Time spent T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:15:36.876169 14982 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:36.882385  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:36.883141  8729 raft_consensus.cc:2393] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 20: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
W20260709 13:15:36.884528 15005 log.cc:927] Time spent T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.143s	user 0.003s	sys 0.000s
I20260709 13:15:36.887874  8057 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:15:36.817353 15003 log.cc:927] Time spent T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:15:36.899394  8729 raft_consensus.cc:1275] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:15:36.901759  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 44449.1us
W20260709 13:15:36.822369 15004 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
W20260709 13:15:36.912858 15002 log.cc:927] Time spent T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:15:36.914583 14969 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:15:36.916499 15007 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:15:36.923679 14989 consensus_queue.cc:1048] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:15:36.933351  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:36.961925 14972 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:36.962560 14972 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:36.964630 14972 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:36.985038  8136 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:36.986150  8175 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 965e59f272f143449b18f65db4e61d98
I20260709 13:15:36.996721 14554 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:36.996573 14692 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:37.007586 14692 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:37.009318 14554 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:37.009598 14692 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:37.013209  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:37.014165  8726 raft_consensus.cc:2393] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in current term 24: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:15:37.043030 14956 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:37.015148 14554 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:37.122200  8211 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:15:37.124080  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 131760us
I20260709 13:15:37.124382  8211 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:15:37.062280 14956 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:37.134212  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 213645us
I20260709 13:15:37.138373  8726 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 13:15:37.120782  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 128576us
I20260709 13:15:37.169548 14661 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:37.174300  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:37.199239 15011 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.285s	user 0.002s	sys 0.001s
W20260709 13:15:37.363526 15012 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.317s	user 0.001s	sys 0.000s
W20260709 13:15:37.414151  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 508717us
I20260709 13:15:37.414880  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:37.415652 14873 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:15:37.421383 15008 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.509s	user 0.007s	sys 0.002s
W20260709 13:15:37.412842 15004 log.cc:927] Time spent T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 13:15:37.437803  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 278353us
I20260709 13:15:37.467535 14956 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:37.474975 14816 consensus_meta.cc:287] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e: Time spent flushing consensus metadata: real 0.664s	user 0.010s	sys 0.000s
I20260709 13:15:37.475407 14816 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:37.476579  8211 raft_consensus.cc:2368] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 24. Current term is 25.
I20260709 13:15:37.477478 14816 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 25 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:37.480693  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:37.481411  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:37.481945  8725 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:15:37.485913 15020 log.cc:927] Time spent T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.321s	user 0.000s	sys 0.009s
W20260709 13:15:37.489204 15022 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.325s	user 0.000s	sys 0.009s
I20260709 13:15:37.490550  8729 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:37.502686  8725 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 25.
I20260709 13:15:37.506162  8176 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:37.510998 14605 raft_consensus.cc:2804] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:15:37.535866  8130 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:15:37.522491 14605 raft_consensus.cc:697] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 25 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:37.543960 14605 consensus_queue.cc:237] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:37.561096  8029 leader_election.cc:400] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 24 pre-election: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 24. Current term is 25.
I20260709 13:15:37.561481  8029 leader_election.cc:403] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:37.562147 14722 raft_consensus.cc:3060] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:15:37.565037  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
W20260709 13:15:37.626284  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 63938.3us
W20260709 13:15:37.626384  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 55876.3us
I20260709 13:15:37.639038  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:37.588274 14819 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:37.598531 14816 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:37.659929  8063 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:15:37.669943  8725 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:15:37.671267  8055 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 21 index: 9. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
I20260709 13:15:37.655692  8130 raft_consensus.cc:1275] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:15:37.670102 14949 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:37.698262  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:37.669667 14819 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:37.681367 14989 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.018s
W20260709 13:15:37.716971  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 146267us
I20260709 13:15:37.720798  8129 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:15:37.722069  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc4cbdf31b174089be94b5f87a22f629" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:37.677556 14816 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:37.725391 14554 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.776230 14819 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:37.780524 14969 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:15:37.761209 14918 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:15:37.783393 15021 log.cc:927] Time spent T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.532s	user 0.009s	sys 0.000s
I20260709 13:15:37.814600 14661 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:15:37.782148 14969 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
W20260709 13:15:37.832827  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 48594.7us
W20260709 13:15:37.842778  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 50039.7us
I20260709 13:15:37.886021 14956 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.886597 14956 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:37.890813  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 57354.7us
I20260709 13:15:37.784565 14554 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:37.818434 14918 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:37.918910  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 200776us
W20260709 13:15:37.920352  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 256079us
I20260709 13:15:37.899547 14382 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:37.891471 15013 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:15:37.928262  8175 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:37.947554 14918 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:37.951447 14956 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:37.948037 14554 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:37.932849 14816 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:37.940778 14765 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:37.936316 14382 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:37.971504 14382 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:37.979064 14853 consensus_queue.cc:1048] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:37.979763  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:37.980899  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:37.981987  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:15:37.982975  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 61146.2us
I20260709 13:15:37.983713  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:37.984546  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:37.985584  8057 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:37.986332  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:37.986961  8131 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:37.986677  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:37.980160 14816 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:15:38.007215  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:38.008208  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:37.963176 15014 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:15:38.015566  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d013f25ff51143beb65d31cc33e5a248" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:38.004983 14722 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745) with higher term. Message: Invalid argument: T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 24. Current term is 25.
I20260709 13:15:38.019021  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:38.011270 14765 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:38.082690 14765 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.080856  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:38.091082 14956 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:38.099884 14956 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.101840 14956 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:38.083145 15016 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:38.114785  8210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6b3f42115d64c26b2b412d669ed6010" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:38.125206  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:38.129649  8055 raft_consensus.cc:2468] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 25.
W20260709 13:15:38.129812  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 49040.8us
I20260709 13:15:38.130692  8175 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:38.115061 15016 raft_consensus.cc:493] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:38.137560 15016 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:38.111341 14964 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:38.111240 14661 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:15:38.150403 15051 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:15:38.155530  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:38.129431 14956 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:38.140959 14867 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:38.190385  8176 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:38.196235 14765 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:38.196744 15057 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:15:38.198588 14956 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.210609  8061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:38.210935  8063 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 25 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:15:38.211977 14956 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:38.215670  8026 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:38.216531 14382 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:38.217974  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:38.218730 14867 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:15:38.218751  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:38.220337  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:38.221133  8026 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:38.223798 14916 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:38.148005 14661 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.201083  8730 raft_consensus.cc:1275] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 25 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:15:38.202646  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:38.202436 14762 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:38.227685 14661 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:38.208276 15022 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:15:38.208679 15012 log.cc:927] Time spent T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:15:38.142823 14964 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.234727  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:38.238054  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:38.244866  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:38.245832  8175 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:38.246570 15014 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:38.246966 14853 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:38.247701  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:38.248383 14958 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:38.249557  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:38.251785 14964 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:38.252282 14989 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:15:38.260823  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 58176.7us
I20260709 13:15:38.288184  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:38.305276  8138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4489591b74b1469d81269da167e98f71"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:38.313158  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:38.319514  8138 raft_consensus.cc:606] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:38.319903  8138 raft_consensus.cc:614] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:38.320155  8175 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:38.298508 14867 consensus_queue.cc:1048] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.015s
I20260709 13:15:38.328861 14956 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:38.339788 15016 raft_consensus.cc:515] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.329736 14867 consensus_queue.cc:1243] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Peer 965e59f272f143449b18f65db4e61d98 log is divergent from this leader: its last log entry 22.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 13:15:38.341768 15016 leader_election.cc:290] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:38.320928 14903 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:38.344094 14903 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.348377  8062 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:38.348973  8062 raft_consensus.cc:606] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:15:38.349284  8062 raft_consensus.cc:614] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:38.353691  8208 raft_consensus.cc:1240] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:15:38.360491  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:38.366595  8138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:38.367163  8138 raft_consensus.cc:606] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 22 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:15:38.377472 14903 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:38.381198  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:38.398949 15013 raft_consensus.cc:2749] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:38.404608  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4489591b74b1469d81269da167e98f71" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" is_pre_election: true
I20260709 13:15:38.413141  8138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:38.413725  8138 raft_consensus.cc:606] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:38.418983  8101 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:38.418900 14732 consensus_queue.cc:1059] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:15:38.419828 14903 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:15:38.420173 14909 raft_consensus.cc:3055] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:38.420529 14909 raft_consensus.cc:740] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:38.423507  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80e5571c873417aa73c79508290e836" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:38.421303 14909 consensus_queue.cc:260] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.429967 14909 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:38.444052  8138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:38.444650  8138 raft_consensus.cc:606] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:38.446507  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:15:38.448041  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:38.472045  8726 pending_rounds.cc:85] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Aborting all ops after (but not including) 17
I20260709 13:15:38.472936  8726 pending_rounds.cc:107] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: Aborting uncommitted NO_OP operation due to leader change: 22.18
I20260709 13:15:38.473999  8726 raft_consensus.cc:2889] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:38.488147 14912 raft_consensus.cc:993] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:38.488735 14968 raft_consensus.cc:1081] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:38.495159  8204 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59868
I20260709 13:15:38.495725  8204 raft_consensus.cc:493] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:38.496119  8204 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:15:38.503959 15063 log.cc:927] Time spent T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:15:38.512639  8062 raft_consensus.cc:2468] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 21.
I20260709 13:15:38.513943  7958 catalog_manager.cc:5697] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 11 to 12, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 12 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
W20260709 13:15:38.527354 15059 log.cc:927] Time spent T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.091s	user 0.002s	sys 0.000s
I20260709 13:15:38.543548  8204 raft_consensus.cc:515] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.545487  8204 leader_election.cc:290] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:38.564718 15067 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
W20260709 13:15:38.570117  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 48696.5us
I20260709 13:15:38.580802  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:38.581411  8056 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:38.593907  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e529be00baff4bfab3a3d0a45d84c238" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:38.594548  8136 raft_consensus.cc:3055] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:15:38.594832  8136 raft_consensus.cc:740] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:38.595520  8136 consensus_queue.cc:260] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.596722  8136 raft_consensus.cc:3060] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:38.605095  8136 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 22.
I20260709 13:15:38.626919  8056 raft_consensus.cc:2468] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 22.
I20260709 13:15:38.642769  8175 leader_election.cc:304] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:38.643970 14853 raft_consensus.cc:2804] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:38.644800 14853 raft_consensus.cc:697] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 21 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:38.645676 14853 consensus_queue.cc:237] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.684989  8175 leader_election.cc:304] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:38.685843 14853 raft_consensus.cc:2804] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:38.686415 14853 raft_consensus.cc:697] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:38.687343 14853 consensus_queue.cc:237] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.687477  7955 catalog_manager.cc:5697] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.688932  7955 catalog_manager.cc:5697] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 12 to 13, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 13 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:38.704205  7955 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 17 to 18, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 18 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:38.714373 14912 raft_consensus.cc:993] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:38.715551 14912 raft_consensus.cc:1081] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:38.714578  7955 catalog_manager.cc:5697] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 15 to 18, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 18 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.718515  8056 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49222
I20260709 13:15:38.719028  8056 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:38.719362  8056 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:38.717765  7955 catalog_manager.cc:5697] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 20 to 23, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 23 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.722697  7955 catalog_manager.cc:5697] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 18 to 19, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 19 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.728899  7955 catalog_manager.cc:5697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 16 to 17, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 17 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:38.731921  8056 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.736977  8056 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:38.737071  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:38.737450  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:38.737643  8131 raft_consensus.cc:3055] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:38.738016  8204 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:38.738018  8131 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:38.739588  8131 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.735782  7955 catalog_manager.cc:5697] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:38.740801  8131 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:38.747651  8131 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:38.748627  8026 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:38.751935 14909 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:38.752398 14909 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:38.753468 14909 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:38.759382  8204 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:38.759589  8198 tablet_service.cc:1558] Processing DeleteTablet for tablet b186be823643475982e269b73d886a42 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 8)) from {username='slave'} at 127.0.0.1:48166
W20260709 13:15:38.759657  7958 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34146 (request call id 260) took 47652 ms (client timeout 15000 ms). Trace:
W20260709 13:15:38.767365  7958 rpcz_store.cc:269] 0709 13:14:51.107848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.289507 (+181659us) service_pool.cc:224] Handling call
0709 13:15:38.759551 (+47470044us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:38.572792 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:38.573311 (+   519us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:38.573341 (+    30us) write_op.cc:435] Acquired schema lock
0709 13:15:38.573359 (+    18us) tablet.cc:662] Decoding operations
0709 13:15:38.573668 (+   309us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:38.573737 (+    69us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:38.573764 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:15:38.573930 (+   166us) tablet.cc:701] Row locks acquired
0709 13:15:38.573947 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:15:38.574074 (+   127us) write_op.cc:270] Start()
0709 13:15:38.574178 (+   104us) write_op.cc:276] Timestamp: P: 1783602938574030 usec, L: 0
0709 13:15:38.574202 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:15:38.574867 (+   665us) log.cc:844] Serialized 538 byte log entry
0709 13:15:38.655313 (+ 80446us) op_driver.cc:464] REPLICATION: finished
0709 13:15:38.656587 (+  1274us) write_op.cc:301] APPLY: starting
0709 13:15:38.656678 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:38.657836 (+  1158us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:38.657940 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:38.672247 (+ 14307us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:38.672428 (+   181us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:15:38.672469 (+    41us) write_op.cc:312] APPLY: finished
0709 13:15:38.672637 (+   168us) log.cc:844] Serialized 25 byte log entry
0709 13:15:38.678978 (+  6341us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:38.679062 (+    84us) write_op.cc:454] Released schema lock
0709 13:15:38.679099 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:38.681626 (+  2527us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:38.723628 (+ 42002us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10397,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":1022,"apply.run_cpu_time_us":3197,"apply.run_wall_time_us":24758,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8307,"num_ops":1,"prepare.queue_time_us":39161,"prepare.run_cpu_time_us":2714,"prepare.run_wall_time_us":9616,"raft.queue_time_us":73263,"raft.run_cpu_time_us":1270,"raft.run_wall_time_us":22066,"replication_time_us":80978,"thread_start_us":131887,"threads_started":4,"wal-append.queue_time_us":5402}]]}
I20260709 13:15:38.770707 14968 raft_consensus.cc:993] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:15:38.771240 14968 raft_consensus.cc:1081] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 22 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:15:38.779505  8726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55826
I20260709 13:15:38.780072  8726 raft_consensus.cc:493] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:38.780545  8726 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:38.805855  8726 raft_consensus.cc:515] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:38.820322  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:38.820863  8204 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:38.825961  8726 leader_election.cc:290] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:38.831166  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5124ddc96f984ec2ae0ee97be6732a90" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:38.831679  8131 raft_consensus.cc:3055] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:15:38.831889  8204 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 23.
I20260709 13:15:38.831969  8131 raft_consensus.cc:740] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:38.833226  8696 leader_election.cc:304] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:38.833909  8131 consensus_queue.cc:260] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:38.834944  8131 raft_consensus.cc:3060] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:38.834556 14894 raft_consensus.cc:2804] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:15:38.842404  8131 raft_consensus.cc:2468] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 23.
I20260709 13:15:38.842404 14894 raft_consensus.cc:697] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 23 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:38.853487  7948 catalog_manager.cc:5028] TS d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): tablet b186be823643475982e269b73d886a42 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:38.857021 14894 consensus_queue.cc:237] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:38.884955  8208 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a003b4584d3747a381522d67299aaed2"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:48166
I20260709 13:15:38.885632  8208 raft_consensus.cc:606] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Received request to transfer leadership
I20260709 13:15:38.889281  7955 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:37774 (request call id 148) took 2070 ms. Trace:
I20260709 13:15:38.897323  8208 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a003b4584d3747a381522d67299aaed2"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:48166
I20260709 13:15:38.897928  8208 raft_consensus.cc:606] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Received request to transfer leadership
W20260709 13:15:38.900081  7948 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for a003b4584d3747a381522d67299aaed2 already in progress [suppressed 8 similar messages]
I20260709 13:15:38.891672  7955 rpcz_store.cc:276] 0709 13:15:36.850329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:36.857478 (+  7149us) service_pool.cc:224] Handling call
0709 13:15:38.889196 (+2031718us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:38.751219 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:38.751728 (+   509us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:38.751759 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:15:38.751772 (+    13us) tablet.cc:662] Decoding operations
0709 13:15:38.752589 (+   817us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:38.752643 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:38.752687 (+    44us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:15:38.753566 (+   879us) tablet.cc:701] Row locks acquired
0709 13:15:38.753587 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:15:38.753720 (+   133us) write_op.cc:270] Start()
0709 13:15:38.753830 (+   110us) write_op.cc:276] Timestamp: P: 1783602938753678 usec, L: 0
0709 13:15:38.753852 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:15:38.754423 (+   571us) log.cc:844] Serialized 3256 byte log entry
0709 13:15:38.756442 (+  2019us) op_driver.cc:464] REPLICATION: finished
0709 13:15:38.760258 (+  3816us) write_op.cc:301] APPLY: starting
0709 13:15:38.760335 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:38.764502 (+  4167us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:38.764603 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:38.858075 (+ 93472us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:38.858333 (+   258us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:15:38.858413 (+    80us) write_op.cc:312] APPLY: finished
0709 13:15:38.861953 (+  3540us) log.cc:844] Serialized 81 byte log entry
0709 13:15:38.877523 (+ 15570us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:38.877608 (+    85us) write_op.cc:454] Released schema lock
0709 13:15:38.877641 (+    33us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:38.877942 (+   301us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:38.877969 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10971,"reported_tablets":8,"spinlock_wait_cycles":1408,"child_traces":[["op",{"apply.queue_time_us":3588,"apply.run_cpu_time_us":11623,"apply.run_wall_time_us":117654,"cfile_cache_hit":32,"cfile_cache_hit_bytes":33440,"num_ops":8,"prepare.queue_time_us":92,"prepare.run_cpu_time_us":3330,"prepare.run_wall_time_us":3442,"replication_time_us":2464}]]}
I20260709 13:15:38.936558  8208 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a003b4584d3747a381522d67299aaed2"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:48166
I20260709 13:15:38.937204  8208 raft_consensus.cc:606] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Received request to transfer leadership
I20260709 13:15:38.918829  7959 catalog_manager.cc:5697] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:38.969580  7959 catalog_manager.cc:5697] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.973097  7959 catalog_manager.cc:5697] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 17 to 19, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 19 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:38.983049  7959 catalog_manager.cc:5697] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:38.986157  7959 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 15 to 16, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 16 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:38.999001  7959 catalog_manager.cc:5697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.006876  7959 catalog_manager.cc:5697] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 13 to 18, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 18 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.008790  7959 catalog_manager.cc:5697] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:39.010380  7959 catalog_manager.cc:5697] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 15 to 17, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 17 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:39.013743  7959 catalog_manager.cc:5697] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:39.015501  7959 catalog_manager.cc:5697] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 18 to 20, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 20 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.017014  7959 catalog_manager.cc:5697] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 13 to 17, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 17 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.018527  7959 catalog_manager.cc:5697] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.021064  7959 catalog_manager.cc:5697] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:39.022812  7959 catalog_manager.cc:5697] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 15 to 18, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 18 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:39.024459  7959 catalog_manager.cc:5697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:39.038170  8009 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:15:39.038674  8009 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for a003b4584d3747a381522d67299aaed2 already in progress
W20260709 13:15:39.038975  8009 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for a003b4584d3747a381522d67299aaed2 already in progress
I20260709 13:15:39.057952 14853 raft_consensus.cc:993] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:39.061817 14765 raft_consensus.cc:1081] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:39.073319  8059 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a003b4584d3747a381522d67299aaed2"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49210
I20260709 13:15:39.073915  8059 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:39.074419  8059 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:39.081532  8059 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:39.103228  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:39.103849  8131 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:39.116046  8131 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:39.119231  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:39.119810  8208 raft_consensus.cc:3055] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:39.120103  8208 raft_consensus.cc:740] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:39.120880  8208 consensus_queue.cc:260] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:39.122252  8208 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:39.122306  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:39.123029  8730 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:39.131023  8208 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:39.134246  8029 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:39.137850 14783 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:39.138984 14783 raft_consensus.cc:697] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:39.139874 14783 consensus_queue.cc:237] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:39.148945  8056 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:15:39.155325  7955 catalog_manager.cc:5697] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 17 to 18, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 18 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:39.156055 14853 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:39.157150  7955 catalog_manager.cc:5697] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 13 to 14, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 14 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:39.160007  8056 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:15:39.160626  8134 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:15:39.162623 14721 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:39.165490 14765 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:39.177248  7955 catalog_manager.cc:5697] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 17 to 20, leader changed from 965e59f272f143449b18f65db4e61d98 (127.6.198.68) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 20 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:39.178845  7955 catalog_manager.cc:5697] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 21 to 25, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 25 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:39.180248  7955 catalog_manager.cc:5697] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 22 to 23, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 23 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:39.189046  8730 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 14.
I20260709 13:15:39.220722  8059 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 14 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:39.239787  8136 raft_consensus.cc:3060] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:39.247267  8136 raft_consensus.cc:1275] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:15:39.248015  8131 raft_consensus.cc:1275] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:15:39.248521 15016 consensus_queue.cc:1048] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:39.249871 15016 consensus_queue.cc:1048] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:15:39.264693  7959 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34134 (request call id 268) took 12791 ms (client timeout 15000 ms). Trace:
W20260709 13:15:39.265102  7959 rpcz_store.cc:269] 0709 13:15:26.472897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.473641 (+   744us) service_pool.cc:224] Handling call
0709 13:15:39.264616 (+12790975us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:39.043653 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:39.044230 (+   577us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:39.044287 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:15:39.044303 (+    16us) tablet.cc:662] Decoding operations
0709 13:15:39.045608 (+  1305us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:39.045664 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:39.045696 (+    32us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:15:39.047196 (+  1500us) tablet.cc:701] Row locks acquired
0709 13:15:39.047218 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:15:39.047411 (+   193us) write_op.cc:270] Start()
0709 13:15:39.047549 (+   138us) write_op.cc:276] Timestamp: P: 1783602939047352 usec, L: 0
0709 13:15:39.047584 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 13:15:39.048429 (+   845us) log.cc:844] Serialized 6314 byte log entry
0709 13:15:39.051112 (+  2683us) op_driver.cc:464] REPLICATION: finished
0709 13:15:39.051558 (+   446us) write_op.cc:301] APPLY: starting
0709 13:15:39.051639 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:39.075641 (+ 24002us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:39.075749 (+   108us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:39.117696 (+ 41947us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:39.126557 (+  8861us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 13:15:39.126608 (+    51us) write_op.cc:312] APPLY: finished
0709 13:15:39.126967 (+   359us) log.cc:844] Serialized 148 byte log entry
0709 13:15:39.127717 (+   750us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:39.127877 (+   160us) write_op.cc:454] Released schema lock
0709 13:15:39.127942 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:39.175632 (+ 47690us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:39.175667 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3330,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":242,"apply.run_cpu_time_us":21832,"apply.run_wall_time_us":123931,"cfile_cache_hit":64,"cfile_cache_hit_bytes":66880,"num_ops":16,"prepare.queue_time_us":12281,"prepare.run_cpu_time_us":5091,"prepare.run_wall_time_us":26853,"replication_time_us":3411}]]}
I20260709 13:15:39.271588  8208 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:15:39.278121 14783 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:39.289139  7960 catalog_manager.cc:5697] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 14 to 19, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 19 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.343724  8204 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:15:39.345537 14775 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:15:39.380251  8138 raft_consensus.cc:1275] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:15:39.383364 14775 consensus_queue.cc:1048] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:15:39.437831  7960 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34120 (request call id 251) took 12964 ms (client timeout 15000 ms). Trace:
I20260709 13:15:39.438053  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet b578d88c1c1f4ca58b7291fed4909f09 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:50166
W20260709 13:15:39.438413  7960 rpcz_store.cc:269] 0709 13:15:26.473303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.473724 (+   421us) service_pool.cc:224] Handling call
0709 13:15:26.575742 (+102018us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b5400002210. stack: 00007f6db7ef7343 00007f6db7ef7240 00007f6db78861d4 00007f6dc15c4304 00007f6dc15c4299 00007f6db7db1db1 00007f6db7ef1b71 00007f6db7ef20b2 00007f6dc0e6ba89 00007f6dc0e03da6 00007f6dc0dd5088 00007f6dc0ed337e 00007f6dbf3942a4 00007f6dbf394232 00007f6dbf394161 00007f6dbf3940dc
0709 13:15:39.437769 (+12862027us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:39.340179 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:39.340832 (+   653us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:39.340876 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:15:39.340897 (+    21us) tablet.cc:662] Decoding operations
0709 13:15:39.341347 (+   450us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:39.341420 (+    73us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:39.341458 (+    38us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:15:39.341698 (+   240us) tablet.cc:701] Row locks acquired
0709 13:15:39.341718 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:15:39.341867 (+   149us) write_op.cc:270] Start()
0709 13:15:39.341963 (+    96us) write_op.cc:276] Timestamp: P: 1783602939341825 usec, L: 0
0709 13:15:39.341978 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 13:15:39.342793 (+   815us) log.cc:844] Serialized 538 byte log entry
0709 13:15:39.346407 (+  3614us) op_driver.cc:464] REPLICATION: finished
0709 13:15:39.346810 (+   403us) write_op.cc:301] APPLY: starting
0709 13:15:39.346889 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:39.347704 (+   815us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:39.347800 (+    96us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:39.348502 (+   702us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:39.348659 (+   157us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:15:39.348712 (+    53us) write_op.cc:312] APPLY: finished
0709 13:15:39.348875 (+   163us) log.cc:844] Serialized 25 byte log entry
0709 13:15:39.349039 (+   164us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:39.349117 (+    78us) write_op.cc:454] Released schema lock
0709 13:15:39.349155 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:39.349497 (+   342us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:39.349543 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":51,"spinlock_wait_cycles":15582080,"child_traces":[["op",{"apply.queue_time_us":225,"apply.run_cpu_time_us":2642,"apply.run_wall_time_us":2643,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4180,"num_ops":1,"prepare.queue_time_us":549,"prepare.run_cpu_time_us":2654,"prepare.run_wall_time_us":2893,"replication_time_us":4284}]]}
I20260709 13:15:39.449100  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:39.444136  7954 catalog_manager.cc:5697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 18 to 19, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 19 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:39.498286  7954 catalog_manager.cc:5697] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 19 to 20, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 20 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:39.509296  7954 catalog_manager.cc:5697] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 15 to 16, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 16 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:39.539754  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 77700.7us
I20260709 13:15:39.534158  7954 catalog_manager.cc:5697] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:39.575935  7954 catalog_manager.cc:5697] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 13 to 18, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 18 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:39.596151  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet b578d88c1c1f4ca58b7291fed4909f09 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:39.615001  8138 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:39.619261 14732 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:39.641956  8204 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:39.644220 14732 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:39.688185  8726 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:39.691457 14918 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:39.728415  7959 catalog_manager.cc:5697] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 18 to 19, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 19 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.738806  7959 catalog_manager.cc:5697] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 19 to 22, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 22 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
W20260709 13:15:39.740272  7954 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34146 (request call id 262) took 12913 ms (client timeout 15000 ms). Trace:
I20260709 13:15:39.740849  7959 catalog_manager.cc:5697] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 16 to 19, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 19 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.743237  7959 catalog_manager.cc:5697] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.745283  7959 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.746706  7959 catalog_manager.cc:5697] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.747852  7959 catalog_manager.cc:5697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 19 to 20, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 20 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.750305  7959 catalog_manager.cc:5697] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 20 to 22, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 22 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.752173  7959 catalog_manager.cc:5697] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 13 to 19, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 19 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.752951  7959 catalog_manager.cc:5697] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:39.754504  7959 catalog_manager.cc:5697] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 18 to 20, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 20 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.755726  7959 catalog_manager.cc:5697] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 17 to 19, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 19 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.760133  7959 catalog_manager.cc:5697] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 17 to 21, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 21 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.761142  7959 catalog_manager.cc:5697] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 12 to 13, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 13 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.762056  7959 catalog_manager.cc:5697] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.764577  7959 catalog_manager.cc:5697] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 20 to 21, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 21 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.766855  7959 catalog_manager.cc:5697] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 14 to 15, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 15 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.768353  7959 catalog_manager.cc:5697] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 20 to 22, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 22 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.769222  7959 catalog_manager.cc:5697] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 16 to 21. New cstate: current_term: 21 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.770000  7959 catalog_manager.cc:5697] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 21 to 23, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 23 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:39.740860  7954 rpcz_store.cc:269] 0709 13:15:26.826458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:26.828897 (+  2439us) service_pool.cc:224] Handling call
0709 13:15:39.740187 (+12911290us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:39.619404 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:39.619967 (+   563us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:39.620002 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:15:39.620022 (+    20us) tablet.cc:662] Decoding operations
0709 13:15:39.620653 (+   631us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:39.620706 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:39.620734 (+    28us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:15:39.621267 (+   533us) tablet.cc:701] Row locks acquired
0709 13:15:39.621286 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:15:39.621422 (+   136us) write_op.cc:270] Start()
0709 13:15:39.621538 (+   116us) write_op.cc:276] Timestamp: P: 1783602939621378 usec, L: 0
0709 13:15:39.621560 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:15:39.622199 (+   639us) log.cc:844] Serialized 2079 byte log entry
0709 13:15:39.624493 (+  2294us) op_driver.cc:464] REPLICATION: finished
0709 13:15:39.624922 (+   429us) write_op.cc:301] APPLY: starting
0709 13:15:39.625004 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:39.627984 (+  2980us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:39.628087 (+   103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:39.630993 (+  2906us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:39.631189 (+   196us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 13:15:39.631242 (+    53us) write_op.cc:312] APPLY: finished
0709 13:15:39.631476 (+   234us) log.cc:844] Serialized 57 byte log entry
0709 13:15:39.632229 (+   753us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:39.632338 (+   109us) write_op.cc:454] Released schema lock
0709 13:15:39.632386 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:39.674424 (+ 42038us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:39.674452 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4121,"reported_tablets":51,"spinlock_wait_cycles":16128,"child_traces":[["op",{"apply.queue_time_us":238,"apply.run_cpu_time_us":7131,"apply.run_wall_time_us":36268,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20900,"num_ops":5,"prepare.queue_time_us":108,"prepare.run_cpu_time_us":3090,"prepare.run_wall_time_us":4884,"replication_time_us":2827}]]}
I20260709 13:15:39.771217  7959 catalog_manager.cc:5697] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 18 to 20, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 20 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:39.782781  7959 catalog_manager.cc:5697] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 16 to 18, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 18 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } }
I20260709 13:15:39.878033  7960 catalog_manager.cc:5697] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 13 to 14, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 14 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:40.598361  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:40.605118  8208 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:40.642670  8138 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:15:40.643355  8176 consensus_peers.cc:597] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f265c49b04e34e9fa83eed8ecdb90607. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:40.644616 15108 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:40.647861  8056 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:15:40.653050 14765 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:40.695364  8208 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:48152
I20260709 13:15:40.696128  8208 raft_consensus.cc:606] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:15:40.696732 15106 raft_consensus.cc:2955] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:40.698264  8058 raft_consensus.cc:2955] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:40.710188  8056 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a003b4584d3747a381522d67299aaed2"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:40.725342  7954 catalog_manager.cc:5697] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 14, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 20 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:40.732517  8056 raft_consensus.cc:606] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:40.735575  8056 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:40.736153  8056 raft_consensus.cc:606] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:40.740892  8058 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ade1b48685164d64a62772d186ac959f"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:40.741434  8058 raft_consensus.cc:606] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:40.755383  8208 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:48152
I20260709 13:15:40.755869  8208 raft_consensus.cc:606] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:40.758008  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:15:40.760147  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:40.859393 15121 raft_consensus.cc:993] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:40.859964 15121 raft_consensus.cc:1081] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Signalling peer 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:40.904701 14721 raft_consensus.cc:993] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:40.905730 14721 raft_consensus.cc:1081] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Signalling peer 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:40.901089  8138 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ade1b48685164d64a62772d186ac959f"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:36064
I20260709 13:15:40.910414  8138 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:40.910858  8138 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:15:40.922933  8009 auto_rebalancer.cc:629] Failed to schedule move for tablet a4992d168cb240009f8bfc8df07668e0: Remote error: Service unavailable: leader transfer in progress
W20260709 13:15:40.943603  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 106972us
I20260709 13:15:40.960999  8134 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:36064
I20260709 13:15:40.961580  8134 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:40.961943  8134 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:40.971462 15108 raft_consensus.cc:993] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e: : Instructing follower 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:40.971988 15108 raft_consensus.cc:1081] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Signalling peer 8a968cada1e4446ebb2a5e167176e649 to start an election
W20260709 13:15:40.979624  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 84722us
I20260709 13:15:40.986357  8138 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:40.981797  8056 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:40.988494  8138 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:40.989284  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 92150.1us
I20260709 13:15:40.990062  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:40.990722  8208 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:40.996672  8058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:40.996747  8131 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:36048
I20260709 13:15:40.997313  8058 raft_consensus.cc:3055] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:40.997413  8131 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:40.997968  8131 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:40.997535  8136 raft_consensus.cc:2955] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:40.997684  8058 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:41.002282  8058 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:40.997505  8134 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:41.003381  8058 raft_consensus.cc:3060] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:41.004338  8134 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:41.011889  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:41.012457  8204 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:41.014029  8208 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:41.027040  8129 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:15:41.027050 15106 raft_consensus.cc:993] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:15:41.028028 15106 raft_consensus.cc:1081] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
W20260709 13:15:41.029080  8027 consensus_peers.cc:597] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d4f33fbb7b794128a4668f6fda160937. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:41.029891  8726 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55838
I20260709 13:15:41.030450  8726 raft_consensus.cc:493] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:41.030797  8726 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:41.031318  8104 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:41.032327 15131 raft_consensus.cc:2804] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:41.033632 15131 raft_consensus.cc:697] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:41.038872 15131 consensus_queue.cc:237] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:41.045625  8203 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:15:41.046677  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:41.022516  8131 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:41.066664  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:41.067234  8208 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:41.067584  8208 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:41.069208 14909 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:41.069604  8726 raft_consensus.cc:515] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:41.047253  8056 raft_consensus.cc:3055] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:41.069680  8208 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:41.070533  8056 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:41.071122  8208 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:41.071924  8204 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:41.053264  8058 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:41.071379  8056 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:41.075435  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:41.081980 14909 raft_consensus.cc:993] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:41.082549 14909 raft_consensus.cc:1081] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Signalling peer 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:41.086436  8131 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:41.090538  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:41.091109  8721 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:41.094498  8726 leader_election.cc:290] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:41.094677  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:41.095295  8203 raft_consensus.cc:3055] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:41.095563  8203 raft_consensus.cc:740] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:41.096472 14903 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:41.096230  8203 consensus_queue.cc:260] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:41.096961 14903 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:41.097508  8203 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:41.100641  8056 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:41.102216  8136 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a003b4584d3747a381522d67299aaed2"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:36064
I20260709 13:15:41.102964  8136 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:41.106864  8136 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:41.097807 14903 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:41.108659  8208 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 21.
I20260709 13:15:41.112566  8203 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 21.
I20260709 13:15:41.116695  8104 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:41.122967  8721 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 21.
I20260709 13:15:41.118971  8696 leader_election.cc:304] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:41.128389  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f229a5eb4514abc9183538dd97073d2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:41.131246 15099 raft_consensus.cc:2804] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:41.131783 15099 raft_consensus.cc:697] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 21 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:41.131880 14909 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:41.132716 15099 consensus_queue.cc:237] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:41.136044 15130 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:41.136621 15130 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:41.137549 15130 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:41.151494  8131 raft_consensus.cc:3060] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:41.155721 15123 raft_consensus.cc:2955] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Committing config change with OpId 21.13: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:41.160450  8138 raft_consensus.cc:2955] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:41.181273  7955 catalog_manager.cc:5697] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 20 to 21, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 21 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:41.182999 15135 ts_tablet_manager.cc:933] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:41.187343  8056 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:41.208119  7954 catalog_manager.cc:5697] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 19 to 20, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 20 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:41.225176  8058 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:41.228725  8136 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:41.232551  8131 raft_consensus.cc:2468] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 21.
I20260709 13:15:41.233235  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:41.233783  8056 raft_consensus.cc:3055] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:41.234038  8056 raft_consensus.cc:740] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:41.234774  8056 consensus_queue.cc:260] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:41.236058  8056 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:41.243685  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:41.244285  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a003b4584d3747a381522d67299aaed2" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:41.244987  8730 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:41.244335  8203 raft_consensus.cc:3060] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:41.247421  8136 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:41.249029  8056 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:41.256186  8203 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:41.257344  8104 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:41.261113 14903 raft_consensus.cc:2804] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:41.261623 14903 raft_consensus.cc:697] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:41.263144 14903 consensus_queue.cc:237] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:41.266284  8730 raft_consensus.cc:2468] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:41.282737  8131 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:15:41.287433  8027 consensus_peers.cc:597] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f7fb8453eda243f9856f489af50e67a2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:41.289927 15132 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:41.297734 15135 tablet_copy_client.cc:323] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.67:35745
I20260709 13:15:41.298242  8208 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:15:41.300277  8222 tablet_copy_service.cc:140] P d9cd2999aa314b39997b423ec1ae509e: Received BeginTabletCopySession request for tablet f265c49b04e34e9fa83eed8ecdb90607 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:55942)
I20260709 13:15:41.300891  8222 tablet_copy_service.cc:161] P d9cd2999aa314b39997b423ec1ae509e: Beginning new tablet copy session on tablet f265c49b04e34e9fa83eed8ecdb90607 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:55942: session id = 965e59f272f143449b18f65db4e61d98-f265c49b04e34e9fa83eed8ecdb90607
I20260709 13:15:41.301363  8208 raft_consensus.cc:2955] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Committing config change with OpId 21.13: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:41.279975  7954 catalog_manager.cc:5697] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 19 to 20, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 20 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:41.306186 15132 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:41.308882  7954 catalog_manager.cc:5697] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 20 to 21, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 21 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:41.310060  7954 catalog_manager.cc:5697] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 21 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:41.331265  8222 tablet_copy_source_session.cc:215] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:41.339366 15135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f265c49b04e34e9fa83eed8ecdb90607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.393252 15132 raft_consensus.cc:2955] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 LEADER]: Committing config change with OpId 22.13: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:41.393857  7954 catalog_manager.cc:5697] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 14 to 15, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 15 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:41.398976  8131 raft_consensus.cc:2955] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Committing config change with OpId 22.13: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:41.438560 15135 tablet_copy_client.cc:806] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:41.439325 15135 tablet_copy_client.cc:670] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:41.440438  7954 catalog_manager.cc:5697] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 22 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:41.450500 15135 tablet_copy_client.cc:538] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:41.460671  8209 raft_consensus.cc:2955] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Committing config change with OpId 22.13: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:41.505033  8208 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:15:41.507342 14912 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:15:41.532421 15147 ts_tablet_manager.cc:933] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
I20260709 13:15:41.551834 15135 tablet_bootstrap.cc:492] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:15:41.560801 15147 tablet_copy_client.cc:323] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.65:45919
I20260709 13:15:41.562810  8073 tablet_copy_service.cc:140] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Received BeginTabletCopySession request for tablet d4f33fbb7b794128a4668f6fda160937 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:37156)
I20260709 13:15:41.563387  8073 tablet_copy_service.cc:161] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Beginning new tablet copy session on tablet d4f33fbb7b794128a4668f6fda160937 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:37156: session id = 965e59f272f143449b18f65db4e61d98-d4f33fbb7b794128a4668f6fda160937
I20260709 13:15:41.586668  8131 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:15:41.588323 15099 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:15:41.617501  8073 tablet_copy_source_session.cc:215] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:41.619231  8058 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:15:41.625913  8730 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:15:41.637310 15147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f33fbb7b794128a4668f6fda160937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:41.637410 14903 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.003s
I20260709 13:15:41.649536  8209 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:15:41.651239 14903 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:41.686640  8203 raft_consensus.cc:1275] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:15:41.688149 15129 consensus_queue.cc:1048] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:15:41.695396 15131 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:41.699631 15147 tablet_copy_client.cc:806] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:41.700270 15147 tablet_copy_client.cc:670] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:41.706296  8056 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:15:41.714246 15130 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:41.737684  8204 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:15:41.746799 14903 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:15:41.886089  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 116717us
I20260709 13:15:41.891287 15135 tablet_bootstrap.cc:492] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=1 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:41.892509 15135 tablet_bootstrap.cc:492] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:15:41.893396 15135 ts_tablet_manager.cc:1403] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.342s	user 0.041s	sys 0.007s
I20260709 13:15:41.915266  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:42.017865 15135 raft_consensus.cc:359] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:42.151479  8725 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:15:42.151875 15154 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:15:42.092780 15135 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:15:42.158918 15135 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:42.161828 15135 ts_tablet_manager.cc:1434] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.268s	user 0.010s	sys 0.002s
W20260709 13:15:42.239995 15152 log.cc:927] Time spent T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:15:42.240046  8222 tablet_copy_service.cc:342] P d9cd2999aa314b39997b423ec1ae509e: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-f265c49b04e34e9fa83eed8ecdb90607 received from {username='slave'} at 127.0.0.1:55942
I20260709 13:15:42.240769  8222 tablet_copy_service.cc:434] P d9cd2999aa314b39997b423ec1ae509e: ending tablet copy session 965e59f272f143449b18f65db4e61d98-f265c49b04e34e9fa83eed8ecdb90607 on tablet f265c49b04e34e9fa83eed8ecdb90607 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:15:42.240881 15147 tablet_copy_client.cc:538] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:15:42.247174 15158 log.cc:927] Time spent T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:15:42.369819  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 74747.1us
W20260709 13:15:42.372459  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 50113.1us
W20260709 13:15:42.393308 15159 log.cc:927] Time spent T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 13:15:42.444108 15147 tablet_bootstrap.cc:492] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:15:42.456753 15172 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:15:42.488806  8721 raft_consensus.cc:1217] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.13->[20.14-20.14]   Dedup: 20.14->[]
W20260709 13:15:42.516533  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 84119.9us
I20260709 13:15:42.524397 15135 ts_tablet_manager.cc:933] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
I20260709 13:15:42.552326  8063 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:15:42.554306  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 80989.6us
I20260709 13:15:42.658778 15205 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
W20260709 13:15:42.677838  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 53326us
I20260709 13:15:42.699445  8211 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:42.698851 15135 tablet_copy_client.cc:323] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.65:45919
I20260709 13:15:42.711755 15172 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:42.720189  8073 tablet_copy_service.cc:140] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Received BeginTabletCopySession request for tablet f7fb8453eda243f9856f489af50e67a2 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:37156)
I20260709 13:15:42.720768  8073 tablet_copy_service.cc:161] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Beginning new tablet copy session on tablet f7fb8453eda243f9856f489af50e67a2 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:37156: session id = 965e59f272f143449b18f65db4e61d98-f7fb8453eda243f9856f489af50e67a2
I20260709 13:15:42.730671  8073 tablet_copy_source_session.cc:215] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:15:42.734822  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 57986.1us
I20260709 13:15:42.743016 15135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7fb8453eda243f9856f489af50e67a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:42.858007 15135 tablet_copy_client.cc:806] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:42.858719 15135 tablet_copy_client.cc:670] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:42.870218 15135 tablet_copy_client.cc:538] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:42.964695 15147 tablet_bootstrap.cc:492] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:42.981789 15147 tablet_bootstrap.cc:492] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:15:42.982579 15147 ts_tablet_manager.cc:1403] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.547s	user 0.048s	sys 0.010s
I20260709 13:15:42.984395 15135 tablet_bootstrap.cc:492] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:15:43.014223 15199 raft_consensus.cc:1064] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:15:43.019999 15199 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:43.027602 15147 raft_consensus.cc:359] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 21 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:43.029338 15147 raft_consensus.cc:740] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 21 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:15:43.029981 15147 consensus_queue.cc:260] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:43.032411 15147 ts_tablet_manager.cc:1434] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.048s	user 0.000s	sys 0.005s
I20260709 13:15:43.038515  8073 tablet_copy_service.cc:342] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-d4f33fbb7b794128a4668f6fda160937 received from {username='slave'} at 127.0.0.1:37156
I20260709 13:15:43.038954  8073 tablet_copy_service.cc:434] P 84d775d1767e4cc4aa7b9c627d8f0fb8: ending tablet copy session 965e59f272f143449b18f65db4e61d98-d4f33fbb7b794128a4668f6fda160937 on tablet d4f33fbb7b794128a4668f6fda160937 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:15:43.074685  8136 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:15:43.076126  8730 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 20 LEARNER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:15:43.076911 15136 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:43.078060 15200 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:15:43.094587  8059 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:15:43.096268 15199 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:43.107955 15200 raft_consensus.cc:2955] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Committing config change with OpId 20.15: config changed from index 14 to 15, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:43.121148  8129 raft_consensus.cc:2955] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:43.151242  8730 raft_consensus.cc:2955] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:43.203127  7954 catalog_manager.cc:5697] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 14 to 15, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:43.271029  8059 raft_consensus.cc:2955] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:43.292389 15135 tablet_bootstrap.cc:492] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=1 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:43.293138 15135 tablet_bootstrap.cc:492] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:15:43.293659 15135 ts_tablet_manager.cc:1403] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.321s	user 0.031s	sys 0.044s
I20260709 13:15:43.300266 15135 raft_consensus.cc:359] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:43.301427 15135 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:15:43.302078 15135 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:43.304194 15135 ts_tablet_manager.cc:1434] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:15:43.305514  8073 tablet_copy_service.cc:342] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-f7fb8453eda243f9856f489af50e67a2 received from {username='slave'} at 127.0.0.1:37156
I20260709 13:15:43.305869  8073 tablet_copy_service.cc:434] P 84d775d1767e4cc4aa7b9c627d8f0fb8: ending tablet copy session 965e59f272f143449b18f65db4e61d98-f7fb8453eda243f9856f489af50e67a2 on tablet f7fb8453eda243f9856f489af50e67a2 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:15:43.521214  8211 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:43.543938  8721 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:15:43.545583  8134 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:15:43.551434 15119 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:43.556370 15119 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:43.557313  8721 raft_consensus.cc:1217] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 21 LEARNER]: Deduplicated request from leader. Original: 21.12->[21.13-21.13]   Dedup: 21.13->[]
I20260709 13:15:43.567440 15106 raft_consensus.cc:2955] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Committing config change with OpId 20.16: config changed from index 15 to 16, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:43.573050  8721 raft_consensus.cc:2955] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Committing config change with OpId 20.16: config changed from index 15 to 16, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:43.573376  8134 raft_consensus.cc:2955] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Committing config change with OpId 20.16: config changed from index 15 to 16, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:43.610458  7948 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f265c49b04e34e9fa83eed8ecdb90607 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:43.609608  7961 catalog_manager.cc:5697] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 15 to 16, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New cstate: current_term: 20 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:43.638125  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet f265c49b04e34e9fa83eed8ecdb90607 with delete_type TABLET_DATA_TOMBSTONED (TS 84d775d1767e4cc4aa7b9c627d8f0fb8 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:50166
I20260709 13:15:43.672375  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:43.684055  8721 raft_consensus.cc:1217] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [term 22 LEARNER]: Deduplicated request from leader. Original: 22.12->[22.13-22.13]   Dedup: 22.13->[]
I20260709 13:15:43.743718 15231 tablet_replica.cc:333] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:43.745123 15231 raft_consensus.cc:2243] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:43.745971 15231 raft_consensus.cc:2272] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:43.757206 15231 ts_tablet_manager.cc:1916] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:43.793545  8211 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6771ba07746943f6a958ca7be1990296"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:48152
I20260709 13:15:43.794155  8211 raft_consensus.cc:606] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:43.802299  8211 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:48152
I20260709 13:15:43.802888  8211 raft_consensus.cc:606] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:43.806247  8134 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:43.806785  8134 raft_consensus.cc:606] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:43.808105  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:15:43.809746  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:43.838956 15182 raft_consensus.cc:993] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:43.839563 15182 raft_consensus.cc:1081] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:43.842520  8059 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49210
I20260709 13:15:43.843072  8059 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:43.843413  8059 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:43.861614  8059 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:43.875116  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:43.875707  8204 raft_consensus.cc:3055] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:43.876209  8204 raft_consensus.cc:740] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:43.876955  8204 consensus_queue.cc:260] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:43.878027  8204 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:43.878789  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:43.879339  8721 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:43.875638  8059 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:43.889564 15231 ts_tablet_manager.cc:1929] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.15
I20260709 13:15:43.890028 15231 log.cc:1199] T f265c49b04e34e9fa83eed8ecdb90607 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/wals/f265c49b04e34e9fa83eed8ecdb90607
I20260709 13:15:43.891911  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet f265c49b04e34e9fa83eed8ecdb90607 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:43.913682  8721 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 21.
I20260709 13:15:43.916162  8027 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:43.917035 15217 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:43.917539 15217 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:43.918407 15217 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:43.935268  8204 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 21.
I20260709 13:15:43.937143  7961 catalog_manager.cc:5697] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 20 to 21, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 21 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:44.129688  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44094.3us
I20260709 13:15:44.132972 15180 raft_consensus.cc:1064] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:15:44.134519 15207 raft_consensus.cc:993] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:44.135020 15131 raft_consensus.cc:1081] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:44.135715 15180 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 11, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:44.137008  8056 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49222
I20260709 13:15:44.137588  8056 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:44.137979  8056 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:15:44.146718  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 55189.7us
I20260709 13:15:44.161221  8056 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:44.172705  8056 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:44.173105  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:44.173432  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:44.173666  8211 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:44.174166  8134 raft_consensus.cc:3055] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:44.174542  8134 raft_consensus.cc:740] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:44.175326  8134 consensus_queue.cc:260] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:44.178978  8730 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 21 LEARNER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:15:44.176591  8134 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:44.180562 15213 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:44.187459  8134 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 21.
I20260709 13:15:44.202189 15119 raft_consensus.cc:993] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: : Instructing follower 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:44.202693 15119 raft_consensus.cc:1081] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Signalling peer 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:44.203559  8204 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:15:44.205425  8134 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6771ba07746943f6a958ca7be1990296"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:36048
I20260709 13:15:44.205945  8134 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:44.206269  8134 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:44.206645  8129 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:15:44.210556 15180 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:44.211493  8026 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
I20260709 13:15:44.216593  8134 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:44.219655  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:44.220028  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:44.220590  8721 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:44.221894  8134 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:44.220187  8208 raft_consensus.cc:3055] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:44.224690  8208 raft_consensus.cc:740] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:44.225508  8208 consensus_queue.cc:260] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:44.226692  8208 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:44.259024 15213 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:44.259656 15213 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:44.260572 15213 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:44.269132 15180 raft_consensus.cc:1064] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
W20260709 13:15:44.270145  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 58737.1us
I20260709 13:15:44.271696 15234 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:44.271816 15180 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 11, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:44.297667  8721 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
I20260709 13:15:44.299232  8102 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:44.301471 15131 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:44.302026 15131 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 19 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:44.302932 15131 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:44.313427  8211 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 21.
I20260709 13:15:44.314396  8209 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:15:44.319725 15235 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:44.310654  8208 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
I20260709 13:15:44.328619  8721 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:15:44.330036 15235 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:15:44.342046  7961 catalog_manager.cc:5697] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 20 to 21, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 21 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:44.404815 15241 log.cc:927] Time spent T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 13:15:44.426622  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 69936us
W20260709 13:15:44.429816  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 72661.3us
I20260709 13:15:44.533077  8134 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:15:44.541038  8136 raft_consensus.cc:2955] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:44.549420 15217 raft_consensus.cc:2955] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Committing config change with OpId 21.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:44.545464 15235 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
W20260709 13:15:44.622860  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 110502us
I20260709 13:15:44.630724  8208 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:15:44.632807  8208 raft_consensus.cc:2955] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:44.636680  8721 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [term 22 LEARNER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:15:44.638726  8721 raft_consensus.cc:2955] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Committing config change with OpId 21.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:44.655830 15180 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:44.674220  7960 catalog_manager.cc:5697] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 18 to 19, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 19 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:44.677363 15234 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:44.692230  8208 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:15:44.699827 15236 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:44.700767  7955 catalog_manager.cc:5697] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 21 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:15:44.756124 15259 log.cc:927] Time spent T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:15:44.772212 15180 raft_consensus.cc:2955] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 LEADER]: Committing config change with OpId 22.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:44.776095  8730 raft_consensus.cc:2955] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:15:44.801717  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 44357.8us
I20260709 13:15:44.810775  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:44.824749  7961 catalog_manager.cc:5697] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:44.859817  8209 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:15:44.883162  8730 raft_consensus.cc:1275] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:15:44.867298  8211 raft_consensus.cc:2955] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:44.894138 15131 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:44.895851 15264 consensus_queue.cc:1048] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:44.948290  8136 raft_consensus.cc:2955] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:44.950987  8134 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:15:44.959069  8056 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:44.972712 15252 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:44.997612  8721 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:15:45.001729 15255 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:45.010573  8208 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:15:45.013787 15252 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:15:45.019605 15236 raft_consensus.cc:2955] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Committing config change with OpId 21.15: config changed from index 14 to 15, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:45.029081  8721 raft_consensus.cc:2955] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 14 to 15, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:45.031260  8208 raft_consensus.cc:2955] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Committing config change with OpId 21.15: config changed from index 14 to 15, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:45.048130  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d4f33fbb7b794128a4668f6fda160937 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:45.085460  8063 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:45.092597  7961 catalog_manager.cc:5697] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 14 to 15, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New cstate: current_term: 21 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:45.112716  8730 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:15:45.115207 15255 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:45.119578  8204 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:15:45.122675 15255 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:45.135495 15252 raft_consensus.cc:2955] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 LEADER]: Committing config change with OpId 22.15: config changed from index 14 to 15, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:45.144941  8725 raft_consensus.cc:2955] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:45.168462  8123 tablet_service.cc:1558] Processing DeleteTablet for tablet d4f33fbb7b794128a4668f6fda160937 with delete_type TABLET_DATA_TOMBSTONED (TS 8a968cada1e4446ebb2a5e167176e649 not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:36736
I20260709 13:15:45.179872  8209 raft_consensus.cc:2955] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Committing config change with OpId 22.15: config changed from index 14 to 15, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:45.192715  7957 catalog_manager.cc:5697] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 14 to 15, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New cstate: current_term: 22 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:45.195854  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f7fb8453eda243f9856f489af50e67a2 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:45.211663  8124 tablet_service.cc:1558] Processing DeleteTablet for tablet f7fb8453eda243f9856f489af50e67a2 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 14 (current committed config index is 15)) from {username='slave'} at 127.0.0.1:36736
I20260709 13:15:45.240001 15270 tablet_replica.cc:333] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:15:45.243068 15270 raft_consensus.cc:2243] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:45.243840 15270 raft_consensus.cc:2272] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:45.256675 15270 ts_tablet_manager.cc:1916] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:45.273125 15270 ts_tablet_manager.cc:1929] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 21.14
I20260709 13:15:45.273488 15270 log.cc:1199] T d4f33fbb7b794128a4668f6fda160937 P 8a968cada1e4446ebb2a5e167176e649: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/wals/d4f33fbb7b794128a4668f6fda160937
I20260709 13:15:45.275122  7945 catalog_manager.cc:5028] TS 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): tablet d4f33fbb7b794128a4668f6fda160937 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:45.276321 15270 tablet_replica.cc:333] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:15:45.277411 15270 raft_consensus.cc:2243] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:45.279052 15270 raft_consensus.cc:2272] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:45.282662 15270 ts_tablet_manager.cc:1916] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:45.299206 15270 ts_tablet_manager.cc:1929] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.14
I20260709 13:15:45.300904 15270 log.cc:1199] T f7fb8453eda243f9856f489af50e67a2 P 8a968cada1e4446ebb2a5e167176e649: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/wals/f7fb8453eda243f9856f489af50e67a2
I20260709 13:15:45.302731  7945 catalog_manager.cc:5028] TS 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): tablet f7fb8453eda243f9856f489af50e67a2 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:45.963843  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:46.098935  8056 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:46.099540  8056 raft_consensus.cc:606] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:46.104822  8056 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:46.105374  8056 raft_consensus.cc:606] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:46.108601  8136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:46.109151  8136 raft_consensus.cc:606] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:46.115870  8136 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:46.116446  8136 raft_consensus.cc:606] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:15:46.119364  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:15:46.121096  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:46.319984 15271 raft_consensus.cc:993] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:15:46.320515 15271 raft_consensus.cc:1081] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:15:46.328135  8730 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55826
I20260709 13:15:46.328685  8730 raft_consensus.cc:493] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:46.329006  8730 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:46.338447 15263 raft_consensus.cc:993] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:46.339017 15263 raft_consensus.cc:1081] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:46.345595  8062 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49222
I20260709 13:15:46.346210  8062 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:46.352033  8062 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:46.351327  8730 raft_consensus.cc:515] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:46.392957  8062 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:46.396124  8136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:46.396770  8136 raft_consensus.cc:3055] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:15:46.397277  8136 raft_consensus.cc:740] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:46.396690  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd0294d309cb4f8696e693285a7b5b04" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:46.399940  8209 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:46.403250  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:46.403795  8725 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:46.405730  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:46.406276  8134 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:46.406612  8134 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:46.406761  8136 consensus_queue.cc:260] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:46.407418  8134 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:46.416723  8134 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:46.410434  8062 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:46.413342 15255 raft_consensus.cc:993] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:46.418330 15255 raft_consensus.cc:1081] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:46.420017  8204 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59856
I20260709 13:15:46.420586  8204 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:46.420928  8204 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:46.415920  8136 raft_consensus.cc:3060] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:46.431551  8725 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:46.435035  8027 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:46.415002  8209 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 22.
I20260709 13:15:46.437060  8696 leader_election.cc:304] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:46.437907 15190 raft_consensus.cc:2804] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:46.438431 15190 raft_consensus.cc:697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 22 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:46.414225  8730 leader_election.cc:290] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:46.439332 15190 consensus_queue.cc:237] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:46.435755 15213 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:46.440845 15213 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:46.441668 15213 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:46.433475  8204 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:46.448700  8204 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:46.449975  8056 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:46.450572  8056 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:46.450846  8056 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:46.451557  8056 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:46.452680  8056 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:46.453889  8136 raft_consensus.cc:2468] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 22.
I20260709 13:15:46.454636  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:46.455149  8131 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:46.446322  8134 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 19.
I20260709 13:15:46.466532  8131 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:46.467562  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:46.468389 15106 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:46.468854 15106 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:46.469653  7957 catalog_manager.cc:5697] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 21 to 22, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 22 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:46.469667 15106 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:46.474460  8056 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:15:46.495324  7955 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 18 to 19, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 19 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:46.517313  7957 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 18 to 19, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 19 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:46.586107  8136 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:46.591272 15255 raft_consensus.cc:993] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:46.592984 15255 raft_consensus.cc:1081] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:46.594686  8204 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59856
I20260709 13:15:46.595204  8204 raft_consensus.cc:493] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:46.595517  8204 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:46.665329  8209 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:15:46.666919 15271 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:15:46.667821  8102 consensus_peers.cc:597] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e397a319b11487fbbacec77723b5739. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:46.680171  8056 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:15:46.616117  8204 raft_consensus.cc:515] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:46.685835 15271 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:46.730458  8131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:46.731158  8131 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:46.731201  8204 leader_election.cc:290] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:46.741541  8131 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:46.742971  8175 leader_election.cc:304] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:15:46.761173  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 109837us
I20260709 13:15:46.746101 15106 raft_consensus.cc:2804] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:46.774544  8062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d785324a51949a79a4de6f0d836be56" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:46.775311  8062 raft_consensus.cc:3055] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:46.775691  8062 raft_consensus.cc:740] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:46.776721  8062 consensus_queue.cc:260] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:46.777776  8062 raft_consensus.cc:3060] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:46.784610  8204 raft_consensus.cc:2955] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:46.771194 15106 raft_consensus.cc:697] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:46.787901 15106 consensus_queue.cc:237] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:46.788679  8062 raft_consensus.cc:2468] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:46.761255 15271 raft_consensus.cc:2955] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
W20260709 13:15:46.799490  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 56612.5us
I20260709 13:15:46.807665  8062 raft_consensus.cc:2955] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:46.831483  7957 catalog_manager.cc:5697] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 20 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:46.862161  7957 catalog_manager.cc:5697] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 15 to 16, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 16 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:46.874804  8009 auto_rebalancer.cc:629] Failed to schedule move for tablet 07fe988cafa34bcaa51af457b7248ccc: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:15:46.879149  8131 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:46.908635  8209 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:46.910568 15289 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:46.917073  8204 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:15:46.919313 15272 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:15:46.920358  8102 consensus_peers.cc:597] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 17c830696ad4418b88d85b6a97e41851. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:46.922696  8730 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:15:46.924070 15213 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:15:46.949625  8131 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:15:46.952240 15213 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:15:46.960552  8062 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:46.970685 15290 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:15:46.975978  8131 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:15:46.990658 15288 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:15:47.019995  8063 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:15:47.056434 15292 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
W20260709 13:15:47.094506  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 105495us
I20260709 13:15:47.104800  8131 raft_consensus.cc:1275] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:15:47.109378  8208 raft_consensus.cc:2955] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:47.122078  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:15:47.186302 15297 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 13:15:47.220889 15300 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:15:47.222304 15301 log.cc:927] Time spent T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:15:47.223201 15299 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:15:47.175663 15303 consensus_queue.cc:1048] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.056s
W20260709 13:15:47.328001  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 76698.5us
W20260709 13:15:47.318300 15295 log.cc:927] Time spent T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:15:47.331192 15313 ts_tablet_manager.cc:933] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:47.373802  8131 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:47.379241  8063 raft_consensus.cc:1275] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:15:47.379235  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 114829us
W20260709 13:15:47.387667  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 61579.6us
I20260709 13:15:47.426297 15330 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:15:47.470829 15182 consensus_queue.cc:1048] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:47.484715 15313 tablet_copy_client.cc:323] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:15:47.490784  8147 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet 8e397a319b11487fbbacec77723b5739 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:15:47.491559  8147 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet 8e397a319b11487fbbacec77723b5739 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-8e397a319b11487fbbacec77723b5739
I20260709 13:15:47.518787 15263 raft_consensus.cc:2955] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:47.527347  8147 tablet_copy_source_session.cc:215] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:47.528193  7960 catalog_manager.cc:5697] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index -1 to 11, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 18 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
W20260709 13:15:47.549871  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 83232.8us
I20260709 13:15:47.551576 15313 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e397a319b11487fbbacec77723b5739. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:47.555776  8062 raft_consensus.cc:2955] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:47.640630 15351 ts_tablet_manager.cc:933] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:47.718636 15351 tablet_copy_client.cc:323] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:15:47.731129  8147 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet 17c830696ad4418b88d85b6a97e41851 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:15:47.731671  8147 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet 17c830696ad4418b88d85b6a97e41851 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-17c830696ad4418b88d85b6a97e41851
W20260709 13:15:47.736671 15353 log.cc:927] Time spent T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.066s	user 0.001s	sys 0.001s
I20260709 13:15:47.738432  8208 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.10, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:47.824043  8147 tablet_copy_source_session.cc:215] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:47.827315 15351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17c830696ad4418b88d85b6a97e41851. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:47.839279 15313 tablet_copy_client.cc:806] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:47.840008 15313 tablet_copy_client.cc:670] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:47.875564 15313 tablet_copy_client.cc:538] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:47.891223 15351 tablet_copy_client.cc:806] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:47.892572 15313 tablet_bootstrap.cc:492] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
W20260709 13:15:47.900373  8176 consensus_peers.cc:597] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ec8da6c869684688957d5323fb023bc5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:47.908524  8059 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:15:47.915032  8136 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:15:47.918581 15346 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:47.920234 15345 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:47.922155 15351 tablet_copy_client.cc:670] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:47.939121 15351 tablet_copy_client.cc:538] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:47.944729 15292 raft_consensus.cc:2955] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Committing config change with OpId 22.11: config changed from index -1 to 11, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:47.950403  8063 raft_consensus.cc:2955] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Committing config change with OpId 22.11: config changed from index -1 to 11, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:47.997128  7957 catalog_manager.cc:5697] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 11, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 22 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:48.005494 15351 tablet_bootstrap.cc:492] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:15:48.092163  8136 raft_consensus.cc:2955] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Committing config change with OpId 22.11: config changed from index -1 to 11, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
W20260709 13:15:48.172936 15313 log.cc:927] Time spent T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.079s	user 0.002s	sys 0.002s
W20260709 13:15:48.282539  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 55816.6us
I20260709 13:15:48.333155 15313 tablet_bootstrap.cc:492] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:48.346490 15313 tablet_bootstrap.cc:492] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:15:48.361634 15313 ts_tablet_manager.cc:1403] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.469s	user 0.043s	sys 0.020s
I20260709 13:15:48.372108  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:48.374236 15313 raft_consensus.cc:359] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:48.375157 15313 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:15:48.375805 15313 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:48.386724 15313 ts_tablet_manager.cc:1434] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:15:48.390638  8147 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-8e397a319b11487fbbacec77723b5739 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:15:48.391180  8147 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-8e397a319b11487fbbacec77723b5739 on tablet 8e397a319b11487fbbacec77723b5739 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:15:48.415297 15351 tablet_bootstrap.cc:492] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:48.416198 15351 tablet_bootstrap.cc:492] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:15:48.416853 15351 ts_tablet_manager.cc:1403] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.412s	user 0.048s	sys 0.018s
I20260709 13:15:48.419842 15351 raft_consensus.cc:359] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:48.420635 15351 raft_consensus.cc:740] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:15:48.421238 15351 consensus_queue.cc:260] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:48.457808 15351 ts_tablet_manager.cc:1434] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 13:15:48.459722  8147 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-17c830696ad4418b88d85b6a97e41851 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:15:48.460166  8147 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-17c830696ad4418b88d85b6a97e41851 on tablet 17c830696ad4418b88d85b6a97e41851 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:15:48.467499 15351 ts_tablet_manager.cc:933] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:48.492553  8206 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:48152
I20260709 13:15:48.493177  8206 raft_consensus.cc:606] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:48.514235  8055 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:48.514801  8055 raft_consensus.cc:606] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:48.519078  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:48.522744  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:48.533589 15351 tablet_copy_client.cc:323] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.67:35745
I20260709 13:15:48.536233  8222 tablet_copy_service.cc:140] P d9cd2999aa314b39997b423ec1ae509e: Received BeginTabletCopySession request for tablet ec8da6c869684688957d5323fb023bc5 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:55942)
I20260709 13:15:48.536805  8222 tablet_copy_service.cc:161] P d9cd2999aa314b39997b423ec1ae509e: Beginning new tablet copy session on tablet ec8da6c869684688957d5323fb023bc5 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:55942: session id = 965e59f272f143449b18f65db4e61d98-ec8da6c869684688957d5323fb023bc5
I20260709 13:15:48.545320  8222 tablet_copy_source_session.cc:215] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:48.548597 15351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec8da6c869684688957d5323fb023bc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:48.557772  8725 raft_consensus.cc:1217] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.12->[20.13-20.13]   Dedup: 20.13->[]
I20260709 13:15:48.580564 15292 raft_consensus.cc:993] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:48.581014 15292 raft_consensus.cc:1081] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:48.583384  8059 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49210
I20260709 13:15:48.583931  8059 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:48.584263  8059 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:48.590327 15351 tablet_copy_client.cc:806] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:48.590973 15351 tablet_copy_client.cc:670] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:48.597079 15351 tablet_copy_client.cc:538] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:48.601563  8059 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:48.603835  8725 raft_consensus.cc:1217] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 18 LEARNER]: Deduplicated request from leader. Original: 18.10->[18.11-18.11]   Dedup: 18.11->[]
I20260709 13:15:48.604451  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:48.604650  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:48.604986  8129 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:48.605237  8206 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:48.605574  8206 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:48.606218  8206 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:48.607239  8206 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:48.613996  8206 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 20.
I20260709 13:15:48.623742  8059 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:48.625527  8029 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:48.627501 15316 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:48.628050 15316 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:48.628813 15316 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:48.634574 15351 tablet_bootstrap.cc:492] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:15:48.642277  7960 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 19 to 20, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 20 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:48.662856  8129 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 20.
I20260709 13:15:48.760509 15364 raft_consensus.cc:993] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:48.761008 15364 raft_consensus.cc:1081] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Signalling peer 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:48.771225  8136 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:36064
I20260709 13:15:48.771811  8136 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:48.772140  8136 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:48.788473 15351 tablet_bootstrap.cc:492] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=3 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:48.789429 15351 tablet_bootstrap.cc:492] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:15:48.790097 15351 ts_tablet_manager.cc:1403] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.156s	user 0.043s	sys 0.001s
I20260709 13:15:48.814990  8136 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:48.836984 15351 raft_consensus.cc:359] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:48.837787 15351 raft_consensus.cc:740] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:15:48.853915  8059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:48.854324  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:48.854630  8059 raft_consensus.cc:3055] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:48.854976  8059 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:48.856389  8059 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:48.857554  8059 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:48.857266 15351 consensus_queue.cc:260] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:48.855175  8136 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:48.854983  8208 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:48.866742 15351 ts_tablet_manager.cc:1434] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.076s	user 0.004s	sys 0.002s
I20260709 13:15:48.868368  8222 tablet_copy_service.cc:342] P d9cd2999aa314b39997b423ec1ae509e: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-ec8da6c869684688957d5323fb023bc5 received from {username='slave'} at 127.0.0.1:55942
I20260709 13:15:48.868768  8222 tablet_copy_service.cc:434] P d9cd2999aa314b39997b423ec1ae509e: ending tablet copy session 965e59f272f143449b18f65db4e61d98-ec8da6c869684688957d5323fb023bc5 on tablet ec8da6c869684688957d5323fb023bc5 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:15:48.879444  8208 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:48.880554  8104 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:48.881496 15340 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:48.881964 15340 raft_consensus.cc:697] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:48.882783 15340 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:48.910172  8059 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 15.
I20260709 13:15:48.926698  7960 catalog_manager.cc:5697] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 14 to 15, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 15 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:48.929389  8730 raft_consensus.cc:1217] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [term 22 LEARNER]: Deduplicated request from leader. Original: 22.10->[22.11-22.11]   Dedup: 22.11->[]
I20260709 13:15:48.969875 15263 raft_consensus.cc:1064] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:15:48.972362 15263 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 11, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:49.000526 15340 raft_consensus.cc:1064] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:15:49.003262 15340 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 9, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:49.026309  8208 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:15:49.027987 15305 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:49.033088  8208 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:15:49.034487 15320 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:49.050529  8730 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 18 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:15:49.051987 15323 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:49.055619  8059 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:15:49.056994 15323 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:49.204478  8208 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:15:49.217408 15263 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:15:49.238608  8059 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:15:49.248973  8730 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 20 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:15:49.252367 15390 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:15:49.261346 15374 log.cc:927] Time spent T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:15:49.275032 15263 raft_consensus.cc:2955] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.287457  8203 raft_consensus.cc:2955] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:15:49.307005  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 76189.6us
I20260709 13:15:49.318768  8059 raft_consensus.cc:2955] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.320806  8129 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:15:49.324952  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 46721.6us
I20260709 13:15:49.330273  7957 catalog_manager.cc:5697] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 18 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.336910 15305 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:49.347476  8730 raft_consensus.cc:2955] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.360500 15382 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:49.380426  8131 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:49.398890  8204 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:15:49.400434 15263 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:49.410588 15389 raft_consensus.cc:2955] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index 12 to 13, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.411916  8204 raft_consensus.cc:2955] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.430188 15381 raft_consensus.cc:2955] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.446125  8725 raft_consensus.cc:2955] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.452626  8730 raft_consensus.cc:1275] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:15:49.458210  7955 catalog_manager.cc:5697] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 12 to 13, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New cstate: current_term: 18 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.465598  8055 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:15:49.467183  8059 raft_consensus.cc:2955] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.475771  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet 17c830696ad4418b88d85b6a97e41851 with delete_type TABLET_DATA_TOMBSTONED (TS 84d775d1767e4cc4aa7b9c627d8f0fb8 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:50166
I20260709 13:15:49.476605 15390 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:49.478907 15390 consensus_queue.cc:1048] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:49.486327  8721 raft_consensus.cc:2955] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.497913  8204 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:15:49.499457 15390 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:49.517230  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 17c830696ad4418b88d85b6a97e41851 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:49.521824  8203 raft_consensus.cc:2955] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.523113 15396 tablet_replica.cc:333] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:49.524315 15396 raft_consensus.cc:2243] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:49.525071 15396 raft_consensus.cc:2272] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:49.528180  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:49.530318 15391 raft_consensus.cc:1064] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:15:49.541601 15391 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 9, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:49.560464 15396 ts_tablet_manager.cc:1916] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:49.564735  8129 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:15:49.565250  8726 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [term 22 LEARNER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:15:49.566754  8055 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:15:49.571619 15391 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:49.573585 15391 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:49.584646 15106 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:49.607409 15391 raft_consensus.cc:2955] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Committing config change with OpId 22.12: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.521271  7955 catalog_manager.cc:5697] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.615528  8055 raft_consensus.cc:2955] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Committing config change with OpId 22.12: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.628813  8725 raft_consensus.cc:2955] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Committing config change with OpId 22.12: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.631460 15396 ts_tablet_manager.cc:1929] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.12
I20260709 13:15:49.631862 15396 log.cc:1199] T 17c830696ad4418b88d85b6a97e41851 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/wals/17c830696ad4418b88d85b6a97e41851
I20260709 13:15:49.637247  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet 17c830696ad4418b88d85b6a97e41851 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:49.661793  8129 raft_consensus.cc:2955] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Committing config change with OpId 22.12: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.677119  8131 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:49.686134  8055 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:15:49.687528 15305 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:49.695251 15389 raft_consensus.cc:2955] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.696544  7960 catalog_manager.cc:5697] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.695515  8721 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:15:49.699115 15381 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:49.705708  8726 raft_consensus.cc:2955] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.721661  8059 raft_consensus.cc:2955] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.768029  7955 catalog_manager.cc:5697] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 14 to 15, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New cstate: current_term: 20 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.771461  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8e397a319b11487fbbacec77723b5739 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:49.798158  8198 tablet_service.cc:1558] Processing DeleteTablet for tablet 8e397a319b11487fbbacec77723b5739 with delete_type TABLET_DATA_TOMBSTONED (TS d9cd2999aa314b39997b423ec1ae509e not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:48166
I20260709 13:15:49.822584  8204 consensus_queue.cc:237] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:49.834152  8726 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:15:49.837231 15391 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:49.842566  8129 raft_consensus.cc:1275] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:15:49.844594 15106 consensus_queue.cc:1048] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:49.846933 15106 raft_consensus.cc:2955] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Committing config change with OpId 22.13: config changed from index 12 to 13, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.848681  8726 raft_consensus.cc:2955] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Committing config change with OpId 22.13: config changed from index 12 to 13, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.859001 15403 tablet_replica.cc:333] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:15:49.860411 15403 raft_consensus.cc:2243] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:49.871592 15403 raft_consensus.cc:2272] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:49.871510  7955 catalog_manager.cc:5697] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 12 to 13, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New cstate: current_term: 22 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.865816  8129 raft_consensus.cc:2955] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Committing config change with OpId 22.13: config changed from index 12 to 13, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:49.886098 15403 ts_tablet_manager.cc:1916] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:49.896019  7948 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ec8da6c869684688957d5323fb023bc5 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:49.897935  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet ec8da6c869684688957d5323fb023bc5 with delete_type TABLET_DATA_TOMBSTONED (TS 84d775d1767e4cc4aa7b9c627d8f0fb8 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:50166
I20260709 13:15:49.899259 15396 tablet_replica.cc:333] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:49.900326 15396 raft_consensus.cc:2243] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:49.901088 15396 raft_consensus.cc:2272] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:49.905267 15396 ts_tablet_manager.cc:1916] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:49.923261 15396 ts_tablet_manager.cc:1929] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.12
I20260709 13:15:49.923765 15396 log.cc:1199] T ec8da6c869684688957d5323fb023bc5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/wals/ec8da6c869684688957d5323fb023bc5
I20260709 13:15:49.926102  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet ec8da6c869684688957d5323fb023bc5 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:49.966187 15403 ts_tablet_manager.cc:1929] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.14
I20260709 13:15:49.966830 15403 log.cc:1199] T 8e397a319b11487fbbacec77723b5739 P d9cd2999aa314b39997b423ec1ae509e: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/wals/8e397a319b11487fbbacec77723b5739
I20260709 13:15:49.968752  7948 catalog_manager.cc:5028] TS d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): tablet 8e397a319b11487fbbacec77723b5739 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:50.780738  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:50.980286  8131 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:50.981204  8131 raft_consensus.cc:606] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:50.994515  8212 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:48152
I20260709 13:15:50.995060  8212 raft_consensus.cc:606] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:50.998494  8059 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:50.999056  8059 raft_consensus.cc:606] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:15:51.000548  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:15:51.003383  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:51.076663  8129 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:51.107904  8212 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:15:51.109766 15407 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:51.111682  8059 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:15:51.114101 15316 raft_consensus.cc:993] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:15:51.114602 15316 raft_consensus.cc:1081] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:15:51.113492 15390 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:15:51.118541  8721 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55848
I20260709 13:15:51.119064  8721 raft_consensus.cc:493] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:51.119441  8721 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:15:51.122031  8102 consensus_peers.cc:597] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 97bd47fbf9aa4d1bb8ec05287f69e0b0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:51.154806 15391 raft_consensus.cc:993] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: : Instructing follower 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:51.155401 15391 raft_consensus.cc:1081] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Signalling peer 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:51.160420  8131 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:36048
I20260709 13:15:51.160991  8131 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:51.161309  8131 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:51.250217  8131 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:51.326948  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:51.264112  8721 raft_consensus.cc:515] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:51.327697  8204 raft_consensus.cc:3055] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:51.327992  8204 raft_consensus.cc:740] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:51.328871  8204 consensus_queue.cc:260] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:51.330021  8204 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:51.330502  8063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:51.331174  8063 raft_consensus.cc:3055] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:15:51.331501  8063 raft_consensus.cc:740] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:51.332324  8063 consensus_queue.cc:260] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:51.332957 15411 log.cc:927] Time spent T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:15:51.333534  8063 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:51.335580  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:51.336155  8726 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:51.340045  8204 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 21.
I20260709 13:15:51.341964  8063 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 23.
I20260709 13:15:51.355613  8693 leader_election.cc:304] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:51.356722 15408 raft_consensus.cc:2804] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:15:51.357232 15408 raft_consensus.cc:697] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [term 23 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:51.358194 15408 consensus_queue.cc:237] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:51.345116  8131 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Requested vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:51.376144  8104 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:51.379720  8726 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 21.
I20260709 13:15:51.382290 15390 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:51.382972 15390 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:51.385861  8721 leader_election.cc:290] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:51.388128 15390 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:51.394217  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7fb8453eda243f9856f489af50e67a2" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: true dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:51.394989  8204 raft_consensus.cc:3060] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:15:51.426882  8204 raft_consensus.cc:2468] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 23.
I20260709 13:15:51.427764  8212 raft_consensus.cc:2955] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:51.427778 15407 raft_consensus.cc:2955] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:51.459365  7955 catalog_manager.cc:5697] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 22 to 23, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 23 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:51.462858  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 73327.9us
I20260709 13:15:51.475133  8059 raft_consensus.cc:2955] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index -1 to 11, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:51.492743  8131 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:51.509824  7959 catalog_manager.cc:5697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 11, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 16 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:51.537003  7957 catalog_manager.cc:5697] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 20 to 21, leader changed from d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 21 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:51.541922  8212 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:15:51.544596 15390 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:15:51.549788  8102 consensus_peers.cc:597] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0e4fa00ecc35497e823a1329fe771042. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:51.550241  8059 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:15:51.552476 15416 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:51.569512 15407 raft_consensus.cc:993] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:51.570150 15407 raft_consensus.cc:1081] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:51.603664  8063 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49222
I20260709 13:15:51.604442  8063 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:51.604900  8063 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:51.613037 15426 ts_tablet_manager.cc:933] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:51.623356 15381 raft_consensus.cc:2955] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 23 LEADER]: Committing config change with OpId 23.13: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:51.627816  8208 raft_consensus.cc:2955] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Committing config change with OpId 23.13: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:51.640496  8059 raft_consensus.cc:2955] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Committing config change with OpId 23.13: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:51.650463  8063 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:51.689399  8063 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:51.690698  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:51.691195  8721 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:51.693707 15426 tablet_copy_client.cc:323] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:15:51.696593  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:51.697247  8129 raft_consensus.cc:3055] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:51.697516  8129 raft_consensus.cc:740] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:51.698297  8129 consensus_queue.cc:260] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:51.699571  8129 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:51.700845  8721 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:51.702135  8027 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:51.708038 15316 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:51.708453  8147 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet 97bd47fbf9aa4d1bb8ec05287f69e0b0 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:15:51.708637 15316 raft_consensus.cc:697] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:51.709003  8147 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet 97bd47fbf9aa4d1bb8ec05287f69e0b0 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-97bd47fbf9aa4d1bb8ec05287f69e0b0
I20260709 13:15:51.709641 15316 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:51.712190  8059 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:51.725971  8147 tablet_copy_source_session.cc:215] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:51.706606  8129 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 16.
I20260709 13:15:51.734826  8212 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
W20260709 13:15:51.735178  8027 consensus_peers.cc:597] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1002e40536a34729929022702d03444c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:51.736157  8129 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:15:51.736635 15428 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:15:51.738139 15316 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:51.744213 15426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97bd47fbf9aa4d1bb8ec05287f69e0b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:51.759977 15316 raft_consensus.cc:2955] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:51.761919  8208 raft_consensus.cc:2955] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:51.777230  8059 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:15:51.778760 15408 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:15:51.817440  8208 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:15:51.819273 15390 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:15:51.820329  8131 raft_consensus.cc:2955] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index -1 to 12, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:51.827284  7959 catalog_manager.cc:5697] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index -1 to 13, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 23 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:51.832244  7959 catalog_manager.cc:5697] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 15 to 16, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 16 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:51.848143  8212 raft_consensus.cc:1275] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:15:51.850189 15418 consensus_queue.cc:1048] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:15:51.852154  8129 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:51.896531  8204 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:51.898160 15415 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:15:51.899029  8102 consensus_peers.cc:597] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b8b8bb0a8e3545218708596da648262f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:51.902299  8059 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:51.904493 15381 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:51.934360  7957 catalog_manager.cc:5697] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index -1 to 12, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 18 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:51.986807 15426 tablet_copy_client.cc:806] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:51.987419 15426 tablet_copy_client.cc:670] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:52.004449  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:52.007824 15448 ts_tablet_manager.cc:933] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:52.018877 15390 raft_consensus.cc:2955] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Committing config change with OpId 15.9: config changed from index -1 to 9, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:52.020599 15426 tablet_copy_client.cc:538] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:52.032428 15448 tablet_copy_client.cc:323] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:15:52.034220  8147 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet 0e4fa00ecc35497e823a1329fe771042 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:15:52.034797  8147 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet 0e4fa00ecc35497e823a1329fe771042 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-0e4fa00ecc35497e823a1329fe771042
I20260709 13:15:52.035792  8721 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:15:52.037101 15381 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:15:52.034003  8208 raft_consensus.cc:2955] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Committing config change with OpId 15.9: config changed from index -1 to 9, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:52.034013  8059 raft_consensus.cc:2955] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Committing config change with OpId 15.9: config changed from index -1 to 9, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:52.044108 15426 tablet_bootstrap.cc:492] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:15:52.059013  8147 tablet_copy_source_session.cc:215] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:52.097004  7960 catalog_manager.cc:5697] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index -1 to 9, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 15 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:52.115852 15448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e4fa00ecc35497e823a1329fe771042. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:52.130549 15447 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:15:52.160343  8131 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:15:52.181370 15430 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:52.226025 15448 tablet_copy_client.cc:806] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:52.226670 15448 tablet_copy_client.cc:670] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:52.232658 15448 tablet_copy_client.cc:538] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:52.243989  8721 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:15:52.248947 15316 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:52.272179 15448 tablet_bootstrap.cc:492] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:15:52.286011 15426 tablet_bootstrap.cc:492] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:52.290424 15426 tablet_bootstrap.cc:492] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:15:52.291246 15426 ts_tablet_manager.cc:1403] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.247s	user 0.034s	sys 0.012s
I20260709 13:15:52.300009 15426 raft_consensus.cc:359] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:52.302925 15426 raft_consensus.cc:740] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:15:52.306744 15426 consensus_queue.cc:260] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:52.324896 15426 ts_tablet_manager.cc:1434] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.002s
I20260709 13:15:52.335106  8147 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-97bd47fbf9aa4d1bb8ec05287f69e0b0 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:15:52.335610  8147 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-97bd47fbf9aa4d1bb8ec05287f69e0b0 on tablet 97bd47fbf9aa4d1bb8ec05287f69e0b0 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:15:52.343829 15426 ts_tablet_manager.cc:933] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
I20260709 13:15:52.374716 15426 tablet_copy_client.cc:323] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.65:45919
I20260709 13:15:52.376591  8073 tablet_copy_service.cc:140] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Received BeginTabletCopySession request for tablet 1002e40536a34729929022702d03444c from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:37156)
I20260709 13:15:52.377120  8073 tablet_copy_service.cc:161] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Beginning new tablet copy session on tablet 1002e40536a34729929022702d03444c from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:37156: session id = 965e59f272f143449b18f65db4e61d98-1002e40536a34729929022702d03444c
I20260709 13:15:52.388015  8073 tablet_copy_source_session.cc:215] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:52.391309 15426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1002e40536a34729929022702d03444c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:52.415311 15426 tablet_copy_client.cc:806] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:52.416934 15426 tablet_copy_client.cc:670] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:52.432897 15426 tablet_copy_client.cc:538] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:52.433610 15459 ts_tablet_manager.cc:933] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:52.449213 15426 tablet_bootstrap.cc:492] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:15:52.472607  8726 raft_consensus.cc:1217] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.10->[16.11-16.11]   Dedup: 16.11->[]
I20260709 13:15:52.486949 15448 tablet_bootstrap.cc:492] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=1 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:52.494385 15448 tablet_bootstrap.cc:492] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:15:52.494763 15459 tablet_copy_client.cc:323] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:15:52.495141 15448 ts_tablet_manager.cc:1403] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.223s	user 0.049s	sys 0.012s
I20260709 13:15:52.497457  8147 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet b8b8bb0a8e3545218708596da648262f from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:15:52.498057  8147 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet b8b8bb0a8e3545218708596da648262f from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-b8b8bb0a8e3545218708596da648262f
I20260709 13:15:52.498270 15448 raft_consensus.cc:359] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:52.499167 15448 raft_consensus.cc:740] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:15:52.507136  8147 tablet_copy_source_session.cc:215] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:52.499795 15448 consensus_queue.cc:260] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:52.514608 15459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8b8bb0a8e3545218708596da648262f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:52.515017 15448 ts_tablet_manager.cc:1434] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.003s
I20260709 13:15:52.524533  8147 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-0e4fa00ecc35497e823a1329fe771042 received from {username='slave'} at 127.0.0.1:46944
I20260709 13:15:52.524996  8147 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-0e4fa00ecc35497e823a1329fe771042 on tablet 0e4fa00ecc35497e823a1329fe771042 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:15:52.579059 15459 tablet_copy_client.cc:806] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:52.582120 15459 tablet_copy_client.cc:670] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:52.586414 15459 tablet_copy_client.cc:538] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:52.595094 15459 tablet_bootstrap.cc:492] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:15:52.609803 15426 tablet_bootstrap.cc:492] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:52.616245 15426 tablet_bootstrap.cc:492] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:15:52.616917 15426 ts_tablet_manager.cc:1403] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.168s	user 0.045s	sys 0.000s
I20260709 13:15:52.627863 15426 raft_consensus.cc:359] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:52.628654 15426 raft_consensus.cc:740] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:15:52.629338 15426 consensus_queue.cc:260] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:52.631793 15426 ts_tablet_manager.cc:1434] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:15:52.640358  8073 tablet_copy_service.cc:342] P 84d775d1767e4cc4aa7b9c627d8f0fb8: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-1002e40536a34729929022702d03444c received from {username='slave'} at 127.0.0.1:37156
I20260709 13:15:52.640815  8073 tablet_copy_service.cc:434] P 84d775d1767e4cc4aa7b9c627d8f0fb8: ending tablet copy session 965e59f272f143449b18f65db4e61d98-1002e40536a34729929022702d03444c on tablet 1002e40536a34729929022702d03444c with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:15:52.755693 15459 tablet_bootstrap.cc:492] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=1 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:52.756719 15459 tablet_bootstrap.cc:492] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:15:52.757505 15459 ts_tablet_manager.cc:1403] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.163s	user 0.034s	sys 0.004s
I20260709 13:15:52.761207 15459 raft_consensus.cc:359] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:52.762089 15459 raft_consensus.cc:740] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:15:52.762892 15459 consensus_queue.cc:260] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:52.768847 15459 ts_tablet_manager.cc:1434] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:15:52.775971  8147 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-b8b8bb0a8e3545218708596da648262f received from {username='slave'} at 127.0.0.1:46944
I20260709 13:15:52.776383  8147 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-b8b8bb0a8e3545218708596da648262f on tablet b8b8bb0a8e3545218708596da648262f with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:15:52.856426 15390 raft_consensus.cc:1064] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:15:52.859586 15390 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:52.864090  8721 raft_consensus.cc:1217] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 18 LEARNER]: Deduplicated request from leader. Original: 18.11->[18.12-18.12]   Dedup: 18.12->[]
I20260709 13:15:52.922701  8721 raft_consensus.cc:1217] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 LEARNER]: Deduplicated request from leader. Original: 23.12->[23.13-23.13]   Dedup: 23.13->[]
I20260709 13:15:52.930692  8203 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:15:52.932284 15390 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:52.932971  8063 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:15:52.945609  8721 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:15:52.953939 15454 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:52.966746 15454 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:53.018891 15454 raft_consensus.cc:2955] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:53.029728  8726 raft_consensus.cc:2955] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:53.056120  8203 raft_consensus.cc:2955] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:53.058974  8721 raft_consensus.cc:1217] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 15 LEARNER]: Deduplicated request from leader. Original: 15.8->[15.9-15.9]   Dedup: 15.9->[]
I20260709 13:15:53.055452  7957 catalog_manager.cc:5697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:53.079792  8063 raft_consensus.cc:2955] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Committing config change with OpId 16.12: config changed from index 11 to 12, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:53.132071  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:15:53.213490  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 59088.6us
I20260709 13:15:53.339351  8136 consensus_queue.cc:237] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:53.345880  8726 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:15:53.347433 15390 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:53.354094  8063 raft_consensus.cc:1275] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:15:53.356026 15474 consensus_queue.cc:1048] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:53.360445 15473 raft_consensus.cc:2955] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Committing config change with OpId 16.13: config changed from index 12 to 13, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:53.366741  8726 raft_consensus.cc:2955] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:53.378515 15474 raft_consensus.cc:1064] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:15:53.381081 15474 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 11, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:53.398662  7955 catalog_manager.cc:5697] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 12 to 13, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New cstate: current_term: 16 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:53.403726  8063 raft_consensus.cc:2955] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Committing config change with OpId 16.13: config changed from index 12 to 13, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:53.406556  8059 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:53.407383  8059 raft_consensus.cc:606] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Received request to transfer leadership to TS 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:53.413105  8129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8e397a319b11487fbbacec77723b5739"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:53.413594  8129 raft_consensus.cc:606] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Received request to transfer leadership to TS 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:53.418529  8212 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:15:53.419159  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:53.420574 15407 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:53.421190  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:53.424484  8198 tablet_service.cc:1558] Processing DeleteTablet for tablet 97bd47fbf9aa4d1bb8ec05287f69e0b0 with delete_type TABLET_DATA_TOMBSTONED (TS d9cd2999aa314b39997b423ec1ae509e not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:48166
I20260709 13:15:53.427305  8059 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:15:53.429149 15407 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:53.432093 15467 raft_consensus.cc:1064] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:15:53.435062 15467 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:53.419341  8726 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:15:53.445133 15407 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:53.488108 15478 tablet_replica.cc:333] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:15:53.519011  8725 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 18 LEARNER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:15:53.520875  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 97bd47fbf9aa4d1bb8ec05287f69e0b0 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:53.529183 15478 raft_consensus.cc:2243] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:53.529944 15478 raft_consensus.cc:2272] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:53.533490  8203 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:15:53.620800 15428 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:53.632778  8136 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:15:53.642307 15316 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:53.645195 15478 ts_tablet_manager.cc:1916] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:15:53.687969  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 152114us
W20260709 13:15:53.727222  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 65113.7us
W20260709 13:15:53.763864  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 98454.9us
W20260709 13:15:53.764544  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 120812us
I20260709 13:15:53.859849 15381 raft_consensus.cc:2955] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 23 LEADER]: Committing config change with OpId 23.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:53.884771 15484 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:15:53.890949  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 90360.4us
I20260709 13:15:53.897032  8726 raft_consensus.cc:2955] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Committing config change with OpId 23.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:53.907958  8211 raft_consensus.cc:2955] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Committing config change with OpId 23.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:53.925987 15501 raft_consensus.cc:1064] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:15:53.956717  8055 raft_consensus.cc:2955] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Committing config change with OpId 23.14: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:53.937731 15501 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 7, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:53.976828 15478 ts_tablet_manager.cc:1929] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.12
I20260709 13:15:53.992002 15478 log.cc:1199] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P d9cd2999aa314b39997b423ec1ae509e: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/wals/97bd47fbf9aa4d1bb8ec05287f69e0b0
I20260709 13:15:53.996310  7948 catalog_manager.cc:5028] TS d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): tablet 97bd47fbf9aa4d1bb8ec05287f69e0b0 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:54.014587  8730 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 15 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:54.018878 15474 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:54.023648  8203 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:54.034520 15474 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:15:54.043802 15504 log.cc:927] Time spent T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:15:54.075847  8204 raft_consensus.cc:2955] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:54.074752 15431 raft_consensus.cc:2955] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:54.099936  8730 raft_consensus.cc:2955] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:54.104410  7959 catalog_manager.cc:5697] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 13 to 14, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:54.112564  7957 catalog_manager.cc:5697] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: config changed from index 12 to 13, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 18 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:54.179960 15484 raft_consensus.cc:993] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:54.180215  8059 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:54.184377 15484 raft_consensus.cc:1081] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Signalling peer 8a968cada1e4446ebb2a5e167176e649 to start an election
I20260709 13:15:54.231094  8136 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
 from {username='slave'} at 127.0.0.1:36064
I20260709 13:15:54.231833  8136 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:54.232256  8136 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:54.246451  8136 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:54.252432  8136 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:54.264008 15474 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:54.273175 15470 raft_consensus.cc:993] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:54.273614 15470 raft_consensus.cc:1081] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Signalling peer 84d775d1767e4cc4aa7b9c627d8f0fb8 to start an election
I20260709 13:15:54.340025  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:54.341042  8055 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:54.341501  8055 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:54.343151  8063 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8e397a319b11487fbbacec77723b5739"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
 from {username='slave'} at 127.0.0.1:49222
I20260709 13:15:54.343685  8063 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:54.344084  8063 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:54.345494  8131 raft_consensus.cc:1240] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Rejecting Update request from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:54.352690  8055 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:54.357410  8055 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:54.241559  8134 raft_consensus.cc:2955] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:54.391564  8055 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:54.392902  8063 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:54.444351  8063 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:54.397581  8101 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649; no voters: 
W20260709 13:15:54.467763  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 70243.4us
W20260709 13:15:54.467958  8027 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 93650.5us
I20260709 13:15:54.469527  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:54.470150  8730 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:15:54.514801  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 146758us
I20260709 13:15:54.524094  8730 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 21.
I20260709 13:15:54.476969 15473 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:54.560071  8027 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
W20260709 13:15:54.570974  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 103520us
I20260709 13:15:54.567442 15413 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:54.565407 15473 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
W20260709 13:15:54.574872  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 71935.7us
I20260709 13:15:54.575771  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:54.576500  8133 raft_consensus.cc:3055] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:54.576880  8133 raft_consensus.cc:740] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:54.581489 15413 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Becoming Leader. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:54.577849  8133 consensus_queue.cc:260] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:54.584889 15473 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:54.682466  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 55922.2us
I20260709 13:15:54.711972 15470 raft_consensus.cc:2955] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index 9 to 10, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:54.683898 15413 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:54.745286  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 123211us
W20260709 13:15:54.747792  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 138571us
I20260709 13:15:54.773597  8063 raft_consensus.cc:2955] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 9 to 10, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:54.783726  8059 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:54.795158  8725 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:15:54.797549 15428 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:54.810122  8132 consensus_queue.cc:237] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:54.811846 15467 raft_consensus.cc:2955] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Committing config change with OpId 18.14: config changed from index 13 to 14, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:54.822198  8062 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:15:54.824062 15496 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:54.821573  8725 raft_consensus.cc:2955] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 13 to 14, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:54.832499 15390 raft_consensus.cc:2955] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 23 LEADER]: Committing config change with OpId 23.15: config changed from index 14 to 15, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:54.839046  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:54.843566  8133 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:54.852152  8203 raft_consensus.cc:2955] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 9 to 10, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:54.857894  8208 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:15:54.868609  8062 raft_consensus.cc:2955] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index 14 to 15, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:54.875573  8133 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84d775d1767e4cc4aa7b9c627d8f0fb8 in term 21.
I20260709 13:15:54.882504 15431 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:54.907260  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1002e40536a34729929022702d03444c with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:54.914629  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: true dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:54.915210  8721 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:54.922061  8721 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:54.927233  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0e4fa00ecc35497e823a1329fe771042 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:15:54.927045  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 54099us
I20260709 13:15:54.945312  8208 raft_consensus.cc:2955] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Committing config change with OpId 18.14: config changed from index 13 to 14, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:54.960999  8730 raft_consensus.cc:1240] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Rejecting Update request from peer 8a968cada1e4446ebb2a5e167176e649 for earlier term 20. Current term is 21. Ops: []
I20260709 13:15:54.977520  7955 catalog_manager.cc:5697] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 13 to 14, VOTER 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) evicted. New cstate: current_term: 18 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:54.993512  8726 raft_consensus.cc:1275] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:15:54.995517  8724 raft_consensus.cc:2955] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 9 to 10, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:55.057961 15496 consensus_queue.cc:1048] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:55.059543  7960 catalog_manager.cc:5697] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 14 to 15, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New cstate: current_term: 23 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:55.067528  7960 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 19 to 20, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 20 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:55.071784  7960 catalog_manager.cc:5697] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 9 to 10, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:55.078606  8123 tablet_service.cc:1558] Processing DeleteTablet for tablet 1002e40536a34729929022702d03444c with delete_type TABLET_DATA_TOMBSTONED (TS 8a968cada1e4446ebb2a5e167176e649 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:36736
I20260709 13:15:55.121202 15532 tablet_replica.cc:333] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:15:55.121925  8725 raft_consensus.cc:2955] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index 14 to 15, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:55.126026  7957 catalog_manager.cc:5697] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 reported cstate change: term changed from 20 to 21, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65). New cstate: current_term: 21 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:55.134877  8063 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:15:55.136405 15474 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:15:55.137847 15532 raft_consensus.cc:2243] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:55.144148  8724 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:15:55.144613 15532 raft_consensus.cc:2272] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:55.145480 15474 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:15:55.145792  8159 debug-util.cc:398] Leaking SignalData structure 0x7b080034a720 after lost signal to thread 6940
W20260709 13:15:55.151598  8159 debug-util.cc:398] Leaking SignalData structure 0x7b080034a280 after lost signal to thread 7995
W20260709 13:15:55.152871  8159 debug-util.cc:398] Leaking SignalData structure 0x7b08001f7200 after lost signal to thread 8087
W20260709 13:15:55.153954  8159 debug-util.cc:398] Leaking SignalData structure 0x7b08000b4340 after lost signal to thread 8162
W20260709 13:15:55.155100  8159 debug-util.cc:398] Leaking SignalData structure 0x7b08002aef40 after lost signal to thread 8236
W20260709 13:15:55.156266  8159 debug-util.cc:398] Leaking SignalData structure 0x7b08003ffa80 after lost signal to thread 8754
I20260709 13:15:55.186439  8198 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e4fa00ecc35497e823a1329fe771042 with delete_type TABLET_DATA_TOMBSTONED (TS d9cd2999aa314b39997b423ec1ae509e not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:48166
I20260709 13:15:55.200058  8134 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:55.210793 15532 ts_tablet_manager.cc:1916] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:55.215174  8134 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:15:55.217908  8211 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:15:55.220222 15493 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:55.219581 15526 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 13:15:55.242245 15497 raft_consensus.cc:2955] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 10 to 11, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:55.250965  8726 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:15:55.252600 15493 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:55.256980  8726 raft_consensus.cc:2955] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:15:55.267169 15533 log.cc:927] Time spent T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:15:55.271360  8211 raft_consensus.cc:2955] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 10 to 11, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:55.302429  8725 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:15:55.295135 15532 ts_tablet_manager.cc:1929] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 18.13
I20260709 13:15:55.309116 15532 log.cc:1199] T 1002e40536a34729929022702d03444c P 8a968cada1e4446ebb2a5e167176e649: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/wals/1002e40536a34729929022702d03444c
I20260709 13:15:55.339239 15535 tablet_replica.cc:333] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:15:55.340423 15535 raft_consensus.cc:2243] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:55.341208 15535 raft_consensus.cc:2272] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:55.342446  7945 catalog_manager.cc:5028] TS 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): tablet 1002e40536a34729929022702d03444c (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:55.345878 15535 ts_tablet_manager.cc:1916] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:55.346468  7961 catalog_manager.cc:5697] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 10 to 11, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New cstate: current_term: 15 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:55.308542 15413 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 13:15:55.374001  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b8b8bb0a8e3545218708596da648262f with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:15:55.374891  8050 tablet_service.cc:1558] Processing DeleteTablet for tablet b8b8bb0a8e3545218708596da648262f with delete_type TABLET_DATA_TOMBSTONED (TS 84d775d1767e4cc4aa7b9c627d8f0fb8 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:50166
W20260709 13:15:55.453904  8029 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 92077.8us
I20260709 13:15:55.461468 15542 tablet_replica.cc:333] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:55.463963 15542 raft_consensus.cc:2243] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:55.470500 15542 raft_consensus.cc:2272] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:55.493520 15542 ts_tablet_manager.cc:1916] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:15:55.517601 15535 ts_tablet_manager.cc:1929] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 23.14
I20260709 13:15:55.518950 15535 log.cc:1199] T 0e4fa00ecc35497e823a1329fe771042 P d9cd2999aa314b39997b423ec1ae509e: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/wals/0e4fa00ecc35497e823a1329fe771042
I20260709 13:15:55.521591  7948 catalog_manager.cc:5028] TS d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): tablet 0e4fa00ecc35497e823a1329fe771042 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:55.552409 15542 ts_tablet_manager.cc:1929] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.10
I20260709 13:15:55.552897 15542 log.cc:1199] T b8b8bb0a8e3545218708596da648262f P 84d775d1767e4cc4aa7b9c627d8f0fb8: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/wals/b8b8bb0a8e3545218708596da648262f
I20260709 13:15:55.555047  7945 catalog_manager.cc:5028] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): tablet b8b8bb0a8e3545218708596da648262f (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
I20260709 13:15:55.968667  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:56.046521  8133 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
new_leader_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:33946
I20260709 13:15:56.047076  8133 raft_consensus.cc:606] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Received request to transfer leadership to TS 965e59f272f143449b18f65db4e61d98
I20260709 13:15:56.054888  8055 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37"
dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
mode: GRACEFUL
new_leader_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:50170
I20260709 13:15:56.055665  8055 raft_consensus.cc:606] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Received request to transfer leadership to TS d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:56.058867  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:15:56.060642  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:56.283535 15526 raft_consensus.cc:993] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:56.284097 15526 raft_consensus.cc:1081] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:15:56.285941  8208 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59856
I20260709 13:15:56.286497  8208 raft_consensus.cc:493] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:56.286821  8208 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:56.304225  8208 raft_consensus.cc:515] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:56.310738  8208 leader_election.cc:290] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:56.312842  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:56.313459  8133 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:56.324781  8133 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:56.327271  8175 leader_election.cc:304] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:56.331967 15499 raft_consensus.cc:2804] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:56.332509 15499 raft_consensus.cc:697] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:56.333518 15499 consensus_queue.cc:237] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:56.339871  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62c9c8ea5bb04f99add9743c31017f37" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:56.340441  8055 raft_consensus.cc:3055] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:56.340711  8055 raft_consensus.cc:740] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 84d775d1767e4cc4aa7b9c627d8f0fb8, State: Running, Role: LEADER
I20260709 13:15:56.341404  8055 consensus_queue.cc:260] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:56.342464  8055 raft_consensus.cc:3060] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:56.352746  8055 raft_consensus.cc:2468] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:56.377206  7960 catalog_manager.cc:5697] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 19 to 20, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 20 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:56.383877 15493 raft_consensus.cc:993] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:15:56.384428 15522 raft_consensus.cc:1081] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Signalling peer 965e59f272f143449b18f65db4e61d98 to start an election
I20260709 13:15:56.386544  8725 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe"
dest_uuid: "965e59f272f143449b18f65db4e61d98"
 from {username='slave'} at 127.0.0.1:55826
I20260709 13:15:56.387086  8725 raft_consensus.cc:493] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:56.387450  8725 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:56.399708  8725 raft_consensus.cc:515] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:56.407374  8725 leader_election.cc:290] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:56.407555  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:56.408541  8134 raft_consensus.cc:3055] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:56.408911  8134 raft_consensus.cc:740] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:56.409700  8134 consensus_queue.cc:260] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:56.410974  8134 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:56.421479  8055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481500a64cdb4a7fb94658807295dcbe" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: true dest_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8"
I20260709 13:15:56.423672  8055 raft_consensus.cc:3060] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:56.446986  8055 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 21.
I20260709 13:15:56.476563  8693 leader_election.cc:304] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:56.478574 15518 raft_consensus.cc:2804] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:56.479168 15518 raft_consensus.cc:697] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 21 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:56.480806 15518 consensus_queue.cc:237] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:56.448414  8134 raft_consensus.cc:2468] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 21.
I20260709 13:15:56.511412  7960 catalog_manager.cc:5697] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 20 to 21, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 21 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:56.633839  8133 consensus_queue.cc:237] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:56.662333  8208 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:15:56.662521  8059 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:15:56.666579 15493 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:15:56.667603 15553 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:15:56.668277  8102 consensus_peers.cc:597] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ade1b48685164d64a62772d186ac959f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:56.692826 15497 raft_consensus.cc:2955] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:56.704108  8208 raft_consensus.cc:2955] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:56.720487  8063 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:56.735646  8055 raft_consensus.cc:2955] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Committing config change with OpId 20.12: config changed from index -1 to 12, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:56.729627  7960 catalog_manager.cc:5697] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index -1 to 12, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 20 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:56.789307  8027 consensus_peers.cc:597] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 07fe988cafa34bcaa51af457b7248ccc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:56.792011  8208 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:15:56.797736 15413 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:56.808620  8133 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer 84d775d1767e4cc4aa7b9c627d8f0fb8: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:15:56.810863 15545 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:56.821511  8063 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:15:56.821684 15526 raft_consensus.cc:2955] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:56.827869 15547 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:56.853609  8208 raft_consensus.cc:2955] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:56.885026  7960 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index -1 to 14, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 20 leader_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:56.886992  8133 raft_consensus.cc:2955] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:56.920356  8059 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 21 index: 19. (index mismatch)
I20260709 13:15:56.929665 15518 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:15:56.932283  8133 raft_consensus.cc:1275] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:15:56.938076 15544 consensus_queue.cc:1048] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:15:56.966689  8026 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 47103.9us
I20260709 13:15:56.970984  8208 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:56.987675  8134 raft_consensus.cc:1275] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 21 index: 19. (index mismatch)
I20260709 13:15:57.004470 15565 consensus_queue.cc:1048] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:15:57.061870  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:15:57.128252  8176 consensus_peers.cc:597] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4489591b74b1469d81269da167e98f71. This is attempt 1: this message will repeat every 5th retry.
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 1
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:15:57.171228  6937 tablet_server.cc:179] TabletServer@127.6.198.65:0 shutting down...
I20260709 13:15:57.170833 15583 ts_tablet_manager.cc:933] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:57.178383  8693 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:15:57.182531  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 92842.6us
I20260709 13:15:57.184741  8133 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:15:57.190919 15583 tablet_copy_client.cc:323] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.66:45807
I20260709 13:15:57.196614 15547 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:15:57.199620  8147 tablet_copy_service.cc:140] P 8a968cada1e4446ebb2a5e167176e649: Received BeginTabletCopySession request for tablet ade1b48685164d64a62772d186ac959f from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:46944)
I20260709 13:15:57.200168  8147 tablet_copy_service.cc:161] P 8a968cada1e4446ebb2a5e167176e649: Beginning new tablet copy session on tablet ade1b48685164d64a62772d186ac959f from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:46944: session id = 965e59f272f143449b18f65db4e61d98-ade1b48685164d64a62772d186ac959f
W20260709 13:15:57.200992  8693 consensus_peers.cc:597] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.217339  8147 tablet_copy_source_session.cc:215] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:57.221120 15583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade1b48685164d64a62772d186ac959f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:15:57.236882  8175 consensus_peers.cc:597] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.239142 15569 raft_consensus.cc:2955] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 23 LEADER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
W20260709 13:15:57.244138  8175 consensus_peers.cc:597] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.244712  8175 consensus_peers.cc:597] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.244624  8101 consensus_peers.cc:597] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.246614  8693 consensus_peers.cc:597] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.247725  8133 raft_consensus.cc:2955] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
W20260709 13:15:57.263525  8693 consensus_peers.cc:597] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.275481  7960 catalog_manager.cc:5697] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index -1 to 15, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 23 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:57.339208 15583 tablet_copy_client.cc:806] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:57.342200  8204 consensus_queue.cc:237] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:57.378613  8130 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:15:57.379261  8693 consensus_peers.cc:597] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.380999 15574 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:15:57.385113  8175 consensus_peers.cc:597] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.389806  8175 consensus_peers.cc:597] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.391767  8176 consensus_peers.cc:597] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dad90abaa8b04508a5b086ff94502e4e. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.397327  8101 consensus_peers.cc:597] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.397934  8101 consensus_peers.cc:597] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.339958 15583 tablet_copy_client.cc:670] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
W20260709 13:15:57.408646  8693 consensus_peers.cc:597] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.413237  8101 consensus_peers.cc:597] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.426651 15583 tablet_copy_client.cc:538] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:15:57.450011  8175 consensus_peers.cc:597] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.453807  8101 consensus_peers.cc:597] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.473325 15592 ts_tablet_manager.cc:933] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
W20260709 13:15:57.481308  8693 consensus_peers.cc:597] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.481703  8175 consensus_peers.cc:597] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.486562  8693 consensus_peers.cc:597] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.491313  8175 consensus_peers.cc:597] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.506680 15547 raft_consensus.cc:2955] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
W20260709 13:15:57.508252  8101 consensus_peers.cc:597] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.512943  8693 consensus_peers.cc:597] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.518018  8134 raft_consensus.cc:2955] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } } }
I20260709 13:15:57.525864 15583 tablet_bootstrap.cc:492] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
W20260709 13:15:57.529258  8693 consensus_peers.cc:597] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.535247  8175 consensus_peers.cc:597] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.538095  8175 consensus_peers.cc:597] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.551602  7960 catalog_manager.cc:5697] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index -1 to 10, NON_VOTER 965e59f272f143449b18f65db4e61d98 (127.6.198.68) added. New cstate: current_term: 13 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:57.550503 15592 tablet_copy_client.cc:323] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.65:45919
I20260709 13:15:57.559276  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:15:57.579192  6937 tablet_replica.cc:333] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.580112  6937 raft_consensus.cc:2243] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.580765  6937 raft_consensus.cc:2272] T a003b4584d3747a381522d67299aaed2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.583905  6937 tablet_replica.cc:333] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
W20260709 13:15:57.608841  8175 consensus_peers.cc:597] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.605751  6937 raft_consensus.cc:2243] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.611028  6937 raft_consensus.cc:2272] T bf628848d7464a21aa8f76f7eb56a408 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:57.612320  8175 consensus_peers.cc:597] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.644019  6937 tablet_replica.cc:333] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.645025  6937 raft_consensus.cc:2243] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.645718  6937 raft_consensus.cc:2272] T ade1b48685164d64a62772d186ac959f P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.652647  6937 tablet_replica.cc:333] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.653504  6937 raft_consensus.cc:2243] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.654458  6937 raft_consensus.cc:2272] T 8e397a319b11487fbbacec77723b5739 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:57.660583  8101 consensus_peers.cc:597] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.660705  8175 consensus_peers.cc:597] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.661073  6937 tablet_replica.cc:333] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.662074  6937 raft_consensus.cc:2243] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.663064  6937 raft_consensus.cc:2272] T 1002e40536a34729929022702d03444c P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.665582  6937 tablet_replica.cc:333] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
W20260709 13:15:57.671507  8101 consensus_peers.cc:597] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:57.682783  8101 consensus_peers.cc:597] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:57.683899  6937 raft_consensus.cc:2243] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.684544  6937 raft_consensus.cc:2272] T f7fb8453eda243f9856f489af50e67a2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.693228 15583 tablet_bootstrap.cc:492] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:57.694156 15583 tablet_bootstrap.cc:492] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:15:57.694779  6937 tablet_replica.cc:333] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.707042  6937 raft_consensus.cc:2243] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.707994 15583 ts_tablet_manager.cc:1403] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.182s	user 0.032s	sys 0.011s
I20260709 13:15:57.711208 15583 raft_consensus.cc:359] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:57.712059 15583 raft_consensus.cc:740] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:15:57.712697 15583 consensus_queue.cc:260] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:57.713408  6937 raft_consensus.cc:2272] T 1b3101b6cc874539a962e23536fdabc9 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.716475  6937 tablet_replica.cc:333] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.717288  6937 raft_consensus.cc:2243] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.717813  6937 raft_consensus.cc:2272] T d80e5571c873417aa73c79508290e836 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.719584 15583 ts_tablet_manager.cc:1434] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:15:57.734181  8724 raft_consensus.cc:1217] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.11->[20.12-20.12]   Dedup: 20.12->[]
I20260709 13:15:57.738464  8147 tablet_copy_service.cc:342] P 8a968cada1e4446ebb2a5e167176e649: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-ade1b48685164d64a62772d186ac959f received from {username='slave'} at 127.0.0.1:46944
I20260709 13:15:57.738898  8147 tablet_copy_service.cc:434] P 8a968cada1e4446ebb2a5e167176e649: ending tablet copy session 965e59f272f143449b18f65db4e61d98-ade1b48685164d64a62772d186ac959f on tablet ade1b48685164d64a62772d186ac959f with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:15:57.740371  6937 tablet_replica.cc:333] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.741765  6937 raft_consensus.cc:2243] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.747123 15596 ts_tablet_manager.cc:933] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:57.748615  6937 raft_consensus.cc:2272] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.753969  6937 tablet_replica.cc:333] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.758844  6937 raft_consensus.cc:2243] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.767593 15596 tablet_copy_client.cc:323] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.67:35745
I20260709 13:15:57.768895  6937 raft_consensus.cc:2272] T fe73c60f6d9f4d25a3dffad1e7198694 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.769855  8222 tablet_copy_service.cc:140] P d9cd2999aa314b39997b423ec1ae509e: Received BeginTabletCopySession request for tablet 4489591b74b1469d81269da167e98f71 from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:55942)
I20260709 13:15:57.770505  8222 tablet_copy_service.cc:161] P d9cd2999aa314b39997b423ec1ae509e: Beginning new tablet copy session on tablet 4489591b74b1469d81269da167e98f71 from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:55942: session id = 965e59f272f143449b18f65db4e61d98-4489591b74b1469d81269da167e98f71
I20260709 13:15:57.772009  6937 tablet_replica.cc:333] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.772909  6937 raft_consensus.cc:2243] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.778393  8222 tablet_copy_source_session.cc:215] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:57.780771  6937 raft_consensus.cc:2272] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.783794  6937 tablet_replica.cc:333] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.786554  6937 raft_consensus.cc:2243] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.787165  6937 raft_consensus.cc:2272] T d013f25ff51143beb65d31cc33e5a248 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.796562 15596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4489591b74b1469d81269da167e98f71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.813936 15583 ts_tablet_manager.cc:933] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:57.823216  6937 tablet_replica.cc:333] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.824246  6937 raft_consensus.cc:2243] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.827458  6937 raft_consensus.cc:2272] T 9fbc44c6e33e41698020173171d4ff21 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.837019  6937 tablet_replica.cc:333] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.838107  6937 raft_consensus.cc:2243] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.838558 15583 tablet_copy_client.cc:323] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.67:35745
I20260709 13:15:57.838835  6937 raft_consensus.cc:2272] T bc4cbdf31b174089be94b5f87a22f629 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.840169  8222 tablet_copy_service.cc:140] P d9cd2999aa314b39997b423ec1ae509e: Received BeginTabletCopySession request for tablet dad90abaa8b04508a5b086ff94502e4e from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:55942)
I20260709 13:15:57.840724  8222 tablet_copy_service.cc:161] P d9cd2999aa314b39997b423ec1ae509e: Beginning new tablet copy session on tablet dad90abaa8b04508a5b086ff94502e4e from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:55942: session id = 965e59f272f143449b18f65db4e61d98-dad90abaa8b04508a5b086ff94502e4e
I20260709 13:15:57.841945  6937 tablet_replica.cc:333] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.842854  6937 raft_consensus.cc:2243] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.843431  6937 raft_consensus.cc:2272] T dad90abaa8b04508a5b086ff94502e4e P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.846309 15596 tablet_copy_client.cc:806] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:57.846931 15596 tablet_copy_client.cc:670] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:57.854774  8222 tablet_copy_source_session.cc:215] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:15:57.854996 15596 tablet_copy_client.cc:538] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:57.872460  6937 tablet_replica.cc:333] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.873307  6937 raft_consensus.cc:2243] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.873848  6937 raft_consensus.cc:2272] T ed7f54a64a14435691eeef2ece07e5dc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.876554  6937 tablet_replica.cc:333] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.877315  6937 raft_consensus.cc:2243] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.877859  6937 raft_consensus.cc:2272] T 0e4fa00ecc35497e823a1329fe771042 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.883579 15583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dad90abaa8b04508a5b086ff94502e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:15:57.886248 15596 tablet_bootstrap.cc:492] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:15:57.888322  6937 tablet_replica.cc:333] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.889165  6937 raft_consensus.cc:2243] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.889729  6937 raft_consensus.cc:2272] T 3d785324a51949a79a4de6f0d836be56 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.892571  6937 tablet_replica.cc:333] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.893354  6937 raft_consensus.cc:2243] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.894006  6937 raft_consensus.cc:2272] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.909994  6937 tablet_replica.cc:333] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.918948  6937 raft_consensus.cc:2243] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.919545  6937 raft_consensus.cc:2272] T 8dd7949df539418cbc00001111885883 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.922189  6937 tablet_replica.cc:333] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.922926  6937 raft_consensus.cc:2243] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.923355  6937 pending_rounds.cc:70] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Trying to abort 1 pending ops.
I20260709 13:15:57.923588  6937 pending_rounds.cc:77] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8: Aborting op as it isn't in flight: id { term: 20 index: 13 } timestamp: 7305637709157171200 op_type: NO_OP noop_request { }
I20260709 13:15:57.923985  6937 raft_consensus.cc:2889] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:15:57.924315  6937 raft_consensus.cc:2272] T 62c9c8ea5bb04f99add9743c31017f37 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.944411  6937 tablet_replica.cc:333] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.945255  6937 raft_consensus.cc:2243] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.946286  6937 raft_consensus.cc:2272] T 542d60a75b684950a46a64c1d64b80a5 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.948732  6937 tablet_replica.cc:333] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.949496  6937 raft_consensus.cc:2243] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.950007  6937 raft_consensus.cc:2272] T 18de311592f1492b8b460305c56e6749 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.952440  6937 tablet_replica.cc:333] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.953193  6937 raft_consensus.cc:2243] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:57.953914  6937 raft_consensus.cc:2272] T 5b4358ace86545339036f21610735b47 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.960973 15563 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:15:57.961516 15563 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:57.966905  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:57.967409  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a9aaa6e2a1c42a0a0db69d6406a3762" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:57.970616  8027 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:57.971629 15563 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:57.972479 15413 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:57.980830  6937 tablet_replica.cc:333] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.981690  6937 raft_consensus.cc:2243] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.990590  6937 raft_consensus.cc:2272] T 710721029c8047ff8b9fc6d5920859f4 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:57.993131  6937 tablet_replica.cc:333] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:57.993880  6937 raft_consensus.cc:2243] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:15:57.994908  6937 raft_consensus.cc:2272] T 053e9e9e90fe4d92853de2c0599484aa P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.005004  6937 tablet_replica.cc:333] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.008550  6937 raft_consensus.cc:2243] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:58.009434  6937 raft_consensus.cc:2272] T d4f33fbb7b794128a4668f6fda160937 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.012091  6937 tablet_replica.cc:333] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.025028  6937 raft_consensus.cc:2243] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.025779  6937 raft_consensus.cc:2272] T e529be00baff4bfab3a3d0a45d84c238 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.028473  6937 tablet_replica.cc:333] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.038237 15596 tablet_bootstrap.cc:492] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=3 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:58.039019  6937 raft_consensus.cc:2243] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.050843  6937 raft_consensus.cc:2272] T 51726a5d5cad44fd8e9b6803da998cb2 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.053511  6937 tablet_replica.cc:333] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.054277  6937 raft_consensus.cc:2243] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.054811  6937 raft_consensus.cc:2272] T 481500a64cdb4a7fb94658807295dcbe P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.057767 15596 tablet_bootstrap.cc:492] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:15:58.058218  6937 tablet_replica.cc:333] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.058508 15596 ts_tablet_manager.cc:1403] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.172s	user 0.052s	sys 0.003s
I20260709 13:15:58.058997  6937 raft_consensus.cc:2243] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.059619  6937 raft_consensus.cc:2272] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.067325  6937 tablet_replica.cc:333] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.068090  6937 raft_consensus.cc:2243] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.068744  6937 raft_consensus.cc:2272] T 4489591b74b1469d81269da167e98f71 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.068091 15596 raft_consensus.cc:359] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:58.070691 15596 raft_consensus.cc:740] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:15:58.075306 15596 consensus_queue.cc:260] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:58.079514 15596 ts_tablet_manager.cc:1434] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.001s
I20260709 13:15:58.087973  6937 tablet_replica.cc:333] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.088876  6937 raft_consensus.cc:2243] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:15:58.090168  6937 raft_consensus.cc:2272] T 07fe988cafa34bcaa51af457b7248ccc P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.096390  6937 tablet_replica.cc:333] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.097172  6937 raft_consensus.cc:2243] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.097695  6937 raft_consensus.cc:2272] T b8fabe9526434f818c106b17c57b3b85 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.111433  8222 tablet_copy_service.cc:342] P d9cd2999aa314b39997b423ec1ae509e: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-4489591b74b1469d81269da167e98f71 received from {username='slave'} at 127.0.0.1:55942
I20260709 13:15:58.111901  8222 tablet_copy_service.cc:434] P d9cd2999aa314b39997b423ec1ae509e: ending tablet copy session 965e59f272f143449b18f65db4e61d98-4489591b74b1469d81269da167e98f71 on tablet 4489591b74b1469d81269da167e98f71 with peer 965e59f272f143449b18f65db4e61d98
I20260709 13:15:58.114791 15545 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:15:58.115278 15545 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:58.124075  6937 tablet_replica.cc:333] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.124822  6937 raft_consensus.cc:2243] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:15:58.125734  6937 raft_consensus.cc:2272] T a4992d168cb240009f8bfc8df07668e0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.128033  6937 tablet_replica.cc:333] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.128670  6937 raft_consensus.cc:2243] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.129293  6937 raft_consensus.cc:2272] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.133455  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:58.140157  6937 tablet_replica.cc:333] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.132230 15545 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:58.146804  6937 raft_consensus.cc:2243] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.146870  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97bd47fbf9aa4d1bb8ec05287f69e0b0" candidate_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:58.147545  6937 raft_consensus.cc:2272] T ac625d2d87774bc7be4ff2189dee7eb0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.150211  6937 tablet_replica.cc:333] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.151000  6937 raft_consensus.cc:2243] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.151611  6937 raft_consensus.cc:2272] T cc8454910b36474c90b678c57cea9773 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.151760  8026 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84d775d1767e4cc4aa7b9c627d8f0fb8; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:58.154676 15545 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:58.155786 15553 raft_consensus.cc:1064] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:15:58.158315 15553 consensus_queue.cc:237] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 10, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:58.154181  6937 tablet_replica.cc:333] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.171085  6937 raft_consensus.cc:2243] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.172004  6937 raft_consensus.cc:2272] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.174620  6937 tablet_replica.cc:333] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.175395  6937 raft_consensus.cc:2243] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:15:58.175695 15583 tablet_copy_client.cc:806] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:15:58.176281 15583 tablet_copy_client.cc:670] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:15:58.176404  6937 raft_consensus.cc:2272] T c97c3defa48b473aaf762b3a27c0cba0 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.178236  8726 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [term 20 LEARNER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:15:58.182301  8208 raft_consensus.cc:1275] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:15:58.186473 15497 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:58.189828 15497 consensus_queue.cc:1048] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:58.185760 15583 tablet_copy_client.cc:538] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:15:58.195281  6937 tablet_replica.cc:333] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.196110  6937 raft_consensus.cc:2243] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:15:58.197099  6937 raft_consensus.cc:2272] T a478a4313e6149378cae4b6f833e3c2a P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.199993  6937 tablet_replica.cc:333] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.200726  6937 raft_consensus.cc:2243] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.201327  6937 raft_consensus.cc:2272] T f6b3f42115d64c26b2b412d669ed6010 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.224458  6937 tablet_replica.cc:333] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.225178  6937 raft_consensus.cc:2243] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.225742  6937 raft_consensus.cc:2272] T b186be823643475982e269b73d886a42 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.228325  6937 tablet_replica.cc:333] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.229071  6937 raft_consensus.cc:2243] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:15:58.236063  8724 raft_consensus.cc:1217] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [term 23 LEARNER]: Deduplicated request from leader. Original: 23.14->[23.15-23.15]   Dedup: 23.15->[]
I20260709 13:15:58.243584  6937 raft_consensus.cc:2272] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.248420  6937 tablet_replica.cc:333] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.249017  6937 raft_consensus.cc:2243] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:15:58.254572  6937 raft_consensus.cc:2272] T 1c85566831fd4f4283bc20a68e5ff448 P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:15:58.257969  6937 tablet_replica.cc:333] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8: stopping tablet replica
I20260709 13:15:58.258965  6937 raft_consensus.cc:2243] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:15:58.260277  6937 raft_consensus.cc:2272] T 5d0e2cd74dac4358aba60edf5382befd P 84d775d1767e4cc4aa7b9c627d8f0fb8 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:15:58.267194  8101 consensus_peers.cc:597] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:58.344588 15497 raft_consensus.cc:2955] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index 12 to 13, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:58.354880  8208 raft_consensus.cc:2955] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:58.365119  8726 raft_consensus.cc:2955] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:58.367355 15583 tablet_bootstrap.cc:492] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
I20260709 13:15:58.385607  7960 catalog_manager.cc:5697] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 12 to 13, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:58.532121 15583 tablet_bootstrap.cc:492] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:15:58.533012 15583 tablet_bootstrap.cc:492] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:15:58.533643 15583 ts_tablet_manager.cc:1403] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.167s	user 0.034s	sys 0.013s
I20260709 13:15:58.536257 15583 raft_consensus.cc:359] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [term 13 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:58.536998 15583 raft_consensus.cc:740] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [term 13 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:15:58.537547 15583 consensus_queue.cc:260] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:58.549463 15583 ts_tablet_manager.cc:1434] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:15:58.551360  8222 tablet_copy_service.cc:342] P d9cd2999aa314b39997b423ec1ae509e: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-dad90abaa8b04508a5b086ff94502e4e received from {username='slave'} at 127.0.0.1:55942
I20260709 13:15:58.551821  8222 tablet_copy_service.cc:434] P d9cd2999aa314b39997b423ec1ae509e: ending tablet copy session 965e59f272f143449b18f65db4e61d98-dad90abaa8b04508a5b086ff94502e4e on tablet dad90abaa8b04508a5b086ff94502e4e with peer 965e59f272f143449b18f65db4e61d98
W20260709 13:15:58.614946  8029 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.657714 15570 raft_consensus.cc:1064] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:15:58.660934 15570 consensus_queue.cc:237] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 13, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:58.691581  8134 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:15:58.693033  8726 raft_consensus.cc:1275] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [term 23 LEARNER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:15:58.694798 15584 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:58.699652 15584 consensus_queue.cc:1048] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:15:58.708135  8175 consensus_peers.cc:597] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:58.720863 15570 raft_consensus.cc:2955] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 23 LEADER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:58.730083  8726 raft_consensus.cc:2955] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:58.748605  8134 raft_consensus.cc:2955] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:58.753646 15592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:55848 (request call id 1642) took 1361 ms. Trace:
I20260709 13:15:58.754182 15592 rpcz_store.cc:276] 0709 13:15:57.391842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.391992 (+   150us) service_pool.cc:224] Handling call
0709 13:15:57.473723 (+ 81731us) ts_tablet_manager.cc:934] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919)
0709 13:15:58.753579 (+1279856us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":12726,"tablet-copy.queue_time_us":80663,"thread_start_us":131970,"threads_started":2}
W20260709 13:15:58.757509  8695 connection.cc:466] server connection from 127.0.0.1:55848 torn down before Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:55848 (request call id 1642) could send its response
I20260709 13:15:58.781764  8726 raft_consensus.cc:1217] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [term 13 LEARNER]: Deduplicated request from leader. Original: 13.9->[13.10-13.10]   Dedup: 13.10->[]
I20260709 13:15:58.782769  7957 catalog_manager.cc:5697] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 15 to 16, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:15:58.853111 15603 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:58.853740 15603 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:58.854840 15575 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:58.855413 15575 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:58.857438 15575 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:58.858898  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:58.859562  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:58.867548  8101 leader_election.cc:336] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:58.868045  8101 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:58.874500 15603 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:58.875725 15591 raft_consensus.cc:2749] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:58.881467  8693 leader_election.cc:336] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:58.881982  8693 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:58.882882 15575 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:58.900897 15547 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:58.901476 15547 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:58.909025  8175 leader_election.cc:336] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:58.915015  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:58.916646  8176 leader_election.cc:304] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:58.917577 15584 raft_consensus.cc:2749] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:58.921995 15547 leader_election.cc:290] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:58.949576 15575 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:58.950268 15575 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:58.954080  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:58.956105 15575 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:58.979760 15577 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:58.980400 15577 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:58.998016  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:58.998621  8204 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:15:59.000624  8696 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:59.002743 15577 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:59.005251 15577 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:59.005661 15577 raft_consensus.cc:493] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.006131 15577 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:59.012473 15584 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.013087 15584 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.019526  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:59.020243  8726 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:59.021567  8176 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:59.023334 15547 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:59.023762 15547 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.024274 15547 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:59.030640 15584 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:59.049497 15575 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.050155 15575 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.052752 15575 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:59.061601 15547 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.064114 15575 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.094920 15575 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.097359 15575 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:59.080716 15603 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.098428 15603 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.101265 15603 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:59.105484  8175 leader_election.cc:336] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.131135  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1dfd4edf9eab4bcc8b8aad94587349c3" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:59.132280  8726 raft_consensus.cc:3060] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:59.144287  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:59.145095  8134 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 21.
I20260709 13:15:59.157068 15547 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:59.164842  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:59.165459  8725 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 21.
I20260709 13:15:59.166710  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:59.186210 15570 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.186975 15570 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:59.189222 15570 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:59.122185 15577 raft_consensus.cc:515] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:59.195966  8101 leader_election.cc:336] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:15:59.199117  8175 leader_election.cc:336] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.197398  8726 raft_consensus.cc:2468] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:59.211951  8176 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:59.213177 15547 raft_consensus.cc:2804] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:59.213773 15547 raft_consensus.cc:697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:59.207602 15577 leader_election.cc:290] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:59.206691  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1002e40536a34729929022702d03444c" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:59.216208  8204 raft_consensus.cc:3060] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:59.222661 15570 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.223284 15570 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.225765 15570 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:59.227921  8204 raft_consensus.cc:2468] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:15:59.229182  8696 leader_election.cc:304] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:59.230075 15617 raft_consensus.cc:2804] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:59.231202 15617 raft_consensus.cc:697] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:59.232121 15617 consensus_queue.cc:237] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.234787 15584 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.235340 15584 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.237711 15584 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:59.239553  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:59.240278  8730 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 21.
I20260709 13:15:59.242206  8176 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:59.244199 15616 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.244731 15616 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.250623 15616 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:59.207203  8693 leader_election.cc:336] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:15:59.257053  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 46654us
I20260709 13:15:59.254277  8693 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:59.255685 15584 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:59.215041 15547 consensus_queue.cc:237] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:59.262269  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 55099.4us
I20260709 13:15:59.275241  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:59.260623 15616 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:59.277226  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:59.261651 15584 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:15:59.262930  8693 leader_election.cc:336] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.284389 15584 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:59.287232 15570 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.295598 15570 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.310429 15570 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:59.311472 15621 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.327953  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:59.332317 15626 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.332993 15626 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.336725 15626 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:59.337957  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 51580.6us
I20260709 13:15:59.346268  8693 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 
I20260709 13:15:59.352988  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:59.353830  8134 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:59.364923 15584 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.366928  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:59.369720 15617 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:59.369354  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:59.368296 15584 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:59.370231 15617 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.370651  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fbc44c6e33e41698020173171d4ff21" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:59.372002  8726 raft_consensus.cc:3060] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:59.379535 15617 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:59.383471 15623 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.385530  8726 raft_consensus.cc:2468] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 22.
I20260709 13:15:59.386962  8176 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:59.384071 15623 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.339082 15621 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.396294 15616 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.396857 15616 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.391820 15620 raft_consensus.cc:1064] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
W20260709 13:15:59.419374  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 45745.9us
I20260709 13:15:59.422163 15639 raft_consensus.cc:2804] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:59.422778 15639 raft_consensus.cc:697] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:59.422212 15620 consensus_queue.cc:237] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.423718 15639 consensus_queue.cc:237] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.425704 15618 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.426270 15618 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.402300 15591 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.432004  8102 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:59.418462 15630 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:15:59.440033  8693 leader_election.cc:336] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.441136  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:59.440065 15630 raft_consensus.cc:493] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.442065 15630 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:59.443742 15621 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:59.447168  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:59.450645 15616 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:59.455432 15623 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:59.456142 15642 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.456718 15642 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.457158 15618 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:59.457504 15634 log.cc:927] Time spent T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98: Append to log took a long time: real 0.074s	user 0.002s	sys 0.000s
I20260709 13:15:59.460966 15619 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.461565 15619 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.433640 15591 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.463651 15619 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:59.464762 15591 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:59.475728  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:59.476524  8725 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:15:59.481549  8176 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:59.482402 15618 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:59.482772 15618 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.483083 15618 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:59.486894 15642 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:59.487620 15577 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:59.491438  8102 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 59611.6us
W20260709 13:15:59.492053  8101 leader_election.cc:336] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.493445 15591 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:59.493844 15591 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.494174 15591 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:59.494729 15640 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.495260 15640 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.497460 15640 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:59.500056  8101 leader_election.cc:336] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.478694  7961 catalog_manager.cc:5697] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 18 to 19, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 19 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:15:59.503000  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 62975.9us
I20260709 13:15:59.509171  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:59.509265  8211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:59.509900  8211 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 19.
I20260709 13:15:59.509843  8208 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
W20260709 13:15:59.533865  8101 leader_election.cc:336] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:15:59.534772  8101 leader_election.cc:336] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.535969 15641 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.536466 15641 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.539582 15624 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:15:59.540071  8693 leader_election.cc:336] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.540156 15624 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.540843  8693 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:59.542474 15619 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.543013 15619 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:59.544595  8693 leader_election.cc:336] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.545346 15625 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:59.545910  8130 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:59.546795 15617 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.549880 15641 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:59.553575 15630 raft_consensus.cc:515] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.554780 15617 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:59.555356 15624 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:59.559681 15630 leader_election.cc:290] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:59.563023  8101 leader_election.cc:336] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.544902 15619 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:59.568342  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:59.575457  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:59.576148  8208 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 21.
I20260709 13:15:59.575333  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:59.578281  8730 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:59.528651  8104 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:15:59.577520  8696 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:59.574572  8725 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:59.585285 15616 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:15:59.586122  8104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 57583.8us
I20260709 13:15:59.585697 15616 raft_consensus.cc:493] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.590931  8726 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [term 13 LEARNER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:15:59.588436 15631 raft_consensus.cc:2749] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:15:59.592316  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 225465us
I20260709 13:15:59.588449 15616 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:59.595252  7957 catalog_manager.cc:5697] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 17 to 18, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 18 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.608445 15616 raft_consensus.cc:515] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.610690  8102 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:59.612896 15631 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:59.613317 15631 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.613720 15631 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:59.618383  8104 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:59.619134 15641 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:59.619491 15641 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.619844 15641 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:59.619820  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:59.620290 15618 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.620724  8134 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:59.622372 15618 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:15:59.622476  8693 leader_election.cc:336] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.622949  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:59.623885 15624 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:59.624243  8693 leader_election.cc:336] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.591284 15591 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.630036  8129 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 965e59f272f143449b18f65db4e61d98 in current term 22: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:15:59.630507 15616 leader_election.cc:290] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:59.631053  8101 consensus_peers.cc:597] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:59.632346  8133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe73c60f6d9f4d25a3dffad1e7198694" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:15:59.632920  8133 raft_consensus.cc:3060] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:59.635607  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4f33fbb7b794128a4668f6fda160937" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:59.636122  8208 raft_consensus.cc:3060] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:59.636797  8130 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:15:59.641904  6937 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 8084)
I20260709 13:15:59.646831 15624 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
W20260709 13:15:59.647029  8101 leader_election.cc:336] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.647392 15624 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:59.642274  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 50253us
I20260709 13:15:59.643620  8133 raft_consensus.cc:2468] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:15:59.649658 15624 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:15:59.642150  8693 leader_election.cc:336] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:15:59.650779  8693 leader_election.cc:336] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.653769  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:59.654533  8725 raft_consensus.cc:2393] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 22: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:15:59.654670  8693 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:59.643942  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:59.655788 15640 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:59.656147 15640 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.656497 15640 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:59.656981  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:59.657579  8134 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 21.
I20260709 13:15:59.655200 15641 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.661741 15626 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.662247 15626 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:59.664382  8206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:59.664973  8206 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 11.
I20260709 13:15:59.665292 15641 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:59.666728 15626 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:59.667457  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:15:59.668017  8696 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:59.668028  8203 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 21.
I20260709 13:15:59.668380 15631 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.655524 15624 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:15:59.669085 15638 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:59.669819 15638 raft_consensus.cc:493] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.670187 15638 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:59.670892 15626 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.671412 15626 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.672510 15618 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.672997 15618 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.673710 15626 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:59.675096 15618 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:15:59.676048  8175 leader_election.cc:336] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.676529  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:15:59.676978  8101 leader_election.cc:336] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:15:59.677469  8101 leader_election.cc:336] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.677529  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a478a4313e6149378cae4b6f833e3c2a" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:59.678165  8726 raft_consensus.cc:3060] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:59.678555 15640 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.682430 15591 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:59.683708 15643 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:59.684983 15618 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:15:59.685468  8102 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:59.687707  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "710721029c8047ff8b9fc6d5920859f4" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:59.688236  8203 raft_consensus.cc:3060] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:59.688436 15591 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:15:59.687194  8175 leader_election.cc:336] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.690040  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:15:59.690693  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
W20260709 13:15:59.691037  8175 leader_election.cc:336] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.691283  8204 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
W20260709 13:15:59.692273  8175 leader_election.cc:336] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:15:59.693076  8175 leader_election.cc:336] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.696784  8203 raft_consensus.cc:2468] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 20.
I20260709 13:15:59.697432 15643 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:59.697978 15638 raft_consensus.cc:515] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.699496  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:15:59.700248  8724 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 16.
I20260709 13:15:59.698629  8726 raft_consensus.cc:2468] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 17.
I20260709 13:15:59.701717  8176 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:59.702248  8203 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97c3defa48b473aaf762b3a27c0cba0" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:59.702805  8203 raft_consensus.cc:3060] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:59.702877  8101 leader_election.cc:336] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.703681 15643 raft_consensus.cc:2804] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:59.704223  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:15:59.699395  8693 consensus_peers.cc:597] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:59.704561 15643 raft_consensus.cc:697] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:59.687296 15640 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:59.707275 15638 leader_election.cc:290] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:59.707293 15643 consensus_queue.cc:237] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.710937  8208 raft_consensus.cc:2468] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 22.
I20260709 13:15:59.711566 15640 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:59.711959 15640 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.712071 15631 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:59.712357 15640 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:59.712070  8696 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:59.713517 15638 raft_consensus.cc:2804] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:15:59.714172 15638 raft_consensus.cc:697] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 22 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:59.715018 15638 consensus_queue.cc:237] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.711040  8104 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:15:59.706646  8101 leader_election.cc:336] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.717728  8104 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
W20260709 13:15:59.717957  8101 leader_election.cc:336] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.718501 15629 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:59.718559 15631 raft_consensus.cc:2804] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:59.726560 15629 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.727095 15629 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:59.728260  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "053e9e9e90fe4d92853de2c0599484aa" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:59.728780  8208 raft_consensus.cc:3060] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:15:59.730163  8693 leader_election.cc:336] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.731773  8102 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:59.731920 15631 raft_consensus.cc:697] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:59.732512 15591 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:59.732887 15591 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.733260 15591 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:59.735522  8693 leader_election.cc:336] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.735715  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "542d60a75b684950a46a64c1d64b80a5" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:59.736325  8725 raft_consensus.cc:3060] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:59.738726  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:59.740335 15591 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:59.742986  8693 leader_election.cc:336] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.742782 15631 consensus_queue.cc:237] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:59.743480  8693 consensus_peers.cc:597] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:59.725209  8203 raft_consensus.cc:2468] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 12.
I20260709 13:15:59.744201 15618 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:59.745390  8696 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:59.746446 15624 raft_consensus.cc:2804] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:59.746946 15624 raft_consensus.cc:697] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:15:59.748095  8725 raft_consensus.cc:2468] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:59.747789 15624 consensus_queue.cc:237] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.753392 15640 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:59.758019  8101 leader_election.cc:336] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.758070  8724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0e2cd74dac4358aba60edf5382befd" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:15:59.758793  8724 raft_consensus.cc:3060] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:59.766141  8724 raft_consensus.cc:2468] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 17.
I20260709 13:15:59.742259 15591 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:15:59.748337  8175 leader_election.cc:304] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:59.774104  8102 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:59.774950  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:15:59.776681  8693 consensus_peers.cc:597] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:59.777282 15643 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:59.775465 15618 raft_consensus.cc:2804] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:59.777925 15618 raft_consensus.cc:697] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:15:59.778803 15618 consensus_queue.cc:237] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.779654  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:59.755627 15640 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:59.776129 15626 raft_consensus.cc:2804] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:59.783629 15626 raft_consensus.cc:697] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:59.784595 15626 consensus_queue.cc:237] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:15:59.770577  8101 leader_election.cc:336] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:15:59.794308  8693 consensus_peers.cc:597] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:59.801755  8102 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:59.802045  7960 catalog_manager.cc:5697] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 19 to 20, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 20 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.803053 15641 raft_consensus.cc:2804] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:15:59.803344  8101 consensus_peers.cc:597] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:59.803728 15641 raft_consensus.cc:697] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:59.804613 15641 consensus_queue.cc:237] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:15:59.767817 15629 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:59.808005  8208 raft_consensus.cc:2468] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 21.
I20260709 13:15:59.808997  8104 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:59.809443 15547 raft_consensus.cc:2764] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:15:59.806439  7957 catalog_manager.cc:5697] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 21 to 22, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 22 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.811437  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649" is_pre_election: true
I20260709 13:15:59.812072  8134 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 20.
I20260709 13:15:59.811597  7957 catalog_manager.cc:5697] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 16 to 17, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 17 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:59.814512 15640 raft_consensus.cc:2804] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:59.815439 15640 raft_consensus.cc:697] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:59.816658  7961 catalog_manager.cc:5697] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 21 to 22, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 22 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:59.817129 15640 consensus_queue.cc:237] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.807394 15643 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.818368  7961 catalog_manager.cc:5697] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 11 to 12, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 12 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:15:59.818363 15643 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:15:59.830942  8175 leader_election.cc:336] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.831362  8175 leader_election.cc:304] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
W20260709 13:15:59.832019  8175 leader_election.cc:336] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:15:59.832837  8175 leader_election.cc:336] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:15:59.833340  8175 leader_election.cc:336] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:15:59.833818  8175 leader_election.cc:336] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.834107  8175 leader_election.cc:304] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:15:59.834666 15618 raft_consensus.cc:2749] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:15:59.839877  8175 leader_election.cc:336] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:15:59.840415  8175 consensus_peers.cc:597] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:15:59.841687 15547 raft_consensus.cc:2749] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:59.847911 15629 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:15:59.850708  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:15:59.851269  8208 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:15:59.859678  8101 leader_election.cc:336] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.859737  8208 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 21.
I20260709 13:15:59.861159  8104 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:15:59.862576 15629 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:59.863075 15629 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:15:59.865326 15629 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:15:59.871493  8726 raft_consensus.cc:1275] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:15:59.871847  8725 raft_consensus.cc:1275] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:15:59.873858 15618 consensus_queue.cc:1048] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:15:59.875697 15618 consensus_queue.cc:1048] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:15:59.885391  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:15:59.886436 15639 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:59.886802 15639 raft_consensus.cc:493] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 84d775d1767e4cc4aa7b9c627d8f0fb8)
I20260709 13:15:59.887149 15639 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:15:59.905351  8175 consensus_peers.cc:597] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:15:59.905854  8175 leader_election.cc:336] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:15:59.906513  8175 consensus_peers.cc:597] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:59.907301  7957 catalog_manager.cc:5697] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 17 to 18, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 18 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:59.910813  8693 consensus_peers.cc:597] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:15:59.912966  8207 raft_consensus.cc:1275] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:15:59.925452 15639 raft_consensus.cc:515] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:15:59.927361  8175 consensus_peers.cc:597] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:15:59.929612  8175 leader_election.cc:336] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:15:59.931982 15624 consensus_queue.cc:1048] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:15:59.973248 15643 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:15:59.985853  8204 raft_consensus.cc:2393] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a968cada1e4446ebb2a5e167176e649 in current term 22: Already voted for candidate d9cd2999aa314b39997b423ec1ae509e in this term.
I20260709 13:15:59.986763 15643 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:59.989389  8133 consensus_queue.cc:237] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:16:00.007884  8693 consensus_peers.cc:597] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:00.011807  8693 consensus_peers.cc:597] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:00.025646 15639 leader_election.cc:290] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:15:59.999542 15632 raft_consensus.cc:2955] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:16:00.045470  8101 consensus_peers.cc:597] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:00.049193  8101 consensus_peers.cc:597] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:00.049896  8693 consensus_peers.cc:597] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:00.053876  8724 raft_consensus.cc:2955] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:16:00.072749  7957 catalog_manager.cc:5697] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e reported cstate change: config changed from index 10 to 11, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 13 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:16:00.075950  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 91270.1us
I20260709 13:16:00.080716  8134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:16:00.085568  8134 raft_consensus.cc:2393] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 22: Already voted for candidate 8a968cada1e4446ebb2a5e167176e649 in this term.
I20260709 13:16:00.092928  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9871516a41a2419d8af8e6fe9e0a5fa1" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:16:00.093226  8724 raft_consensus.cc:1275] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:16:00.093490  8129 raft_consensus.cc:3060] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:00.103644  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:00.105526 15640 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:00.099107 15629 consensus_queue.cc:1048] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
W20260709 13:16:00.129006  8101 consensus_peers.cc:597] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:00.129458  8101 consensus_peers.cc:597] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:00.143637  8693 consensus_peers.cc:597] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:00.148773  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 43626.9us
I20260709 13:16:00.162604  8208 raft_consensus.cc:1275] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:16:00.162752  8101 consensus_peers.cc:597] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:00.160631  8129 raft_consensus.cc:2468] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 21.
I20260709 13:16:00.169611  8134 raft_consensus.cc:2955] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } attrs { replace: true } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:16:00.174858 15635 consensus_queue.cc:1048] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:16:00.186252  8175 leader_election.cc:336] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:00.188418 15642 raft_consensus.cc:2955] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index 3 to 11, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:16:00.189611  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:00.196828 15619 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:00.198125  8724 raft_consensus.cc:2955] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index 3 to 11, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:16:00.202904  8101 consensus_peers.cc:597] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:00.197081  8175 consensus_peers.cc:597] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:00.205405  8175 leader_election.cc:336] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:00.212460  8175 consensus_peers.cc:597] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:00.218952  8175 consensus_peers.cc:597] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:00.219735  7945 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a003b4584d3747a381522d67299aaed2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 14)
I20260709 13:16:00.218196  8726 raft_consensus.cc:1275] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
W20260709 13:16:00.226743  8693 consensus_peers.cc:597] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:00.226018  7957 catalog_manager.cc:5697] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 16 to 17, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 17 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.228286  7957 catalog_manager.cc:5697] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 16 to 17, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 17 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.227412 15642 consensus_queue.cc:1048] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:16:00.230280  7957 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 20 to 21, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 21 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.235280  7957 catalog_manager.cc:5697] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 20 to 21, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 21 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:00.255928  8175 leader_election.cc:304] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:16:00.260417 15619 raft_consensus.cc:2804] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:00.261147 15619 raft_consensus.cc:697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 21 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:16:00.261979 15619 consensus_queue.cc:237] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:16:00.268030  8129 raft_consensus.cc:1275] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 13:16:00.275877  8175 consensus_peers.cc:597] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:00.281457  8204 consensus_queue.cc:237] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:00.287123  8208 raft_consensus.cc:1275] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Refusing update from remote peer 965e59f272f143449b18f65db4e61d98: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:16:00.293394 15616 consensus_queue.cc:1048] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:00.295156  7960 catalog_manager.cc:5697] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 reported cstate change: config changed from index 3 to 11, VOTER d9cd2999aa314b39997b423ec1ae509e (127.6.198.67) evicted. New cstate: current_term: 15 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:16:00.305718  8693 consensus_peers.cc:597] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:00.310233  8198 tablet_service.cc:1558] Processing DeleteTablet for tablet a003b4584d3747a381522d67299aaed2 with delete_type TABLET_DATA_TOMBSTONED (TS d9cd2999aa314b39997b423ec1ae509e not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:48166
I20260709 13:16:00.314881 15643 consensus_queue.cc:1048] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:00.315306 15678 tablet_replica.cc:333] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:00.316474 15678 raft_consensus.cc:2243] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:00.322749 15678 raft_consensus.cc:2272] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:00.323328  8208 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:16:00.324833 15642 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:16:00.327729  8175 consensus_peers.cc:597] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:00.328209  8175 consensus_peers.cc:597] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:00.328735  8101 consensus_peers.cc:597] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:00.332122  8101 consensus_peers.cc:597] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:00.340574  8175 consensus_peers.cc:597] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:00.342291  8101 consensus_peers.cc:597] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:00.343237 15678 ts_tablet_manager.cc:1916] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:16:00.369262  8204 raft_consensus.cc:1275] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:00.370000  8208 raft_consensus.cc:1275] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:16:00.375396  8101 consensus_peers.cc:597] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:00.377038  8724 raft_consensus.cc:1275] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:16:00.378504 15629 consensus_queue.cc:1048] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:00.379791 15626 consensus_queue.cc:1048] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:16:00.386821  8175 consensus_peers.cc:597] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:00.395182  8130 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:16:00.396533  8101 consensus_peers.cc:597] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:00.408344 15630 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:00.411891 15678 ts_tablet_manager.cc:1929] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.10
I20260709 13:16:00.404568 15643 consensus_queue.cc:1048] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:16:00.418756  8725 raft_consensus.cc:1275] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:16:00.412451 15678 log.cc:1199] T a003b4584d3747a381522d67299aaed2 P d9cd2999aa314b39997b423ec1ae509e: Deleting WAL directory at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/wals/a003b4584d3747a381522d67299aaed2
I20260709 13:16:00.420249 15668 consensus_queue.cc:1048] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:00.460541  7948 catalog_manager.cc:5028] TS d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): tablet a003b4584d3747a381522d67299aaed2 (table test-workload [id=0c145eb23c244f65946ed4d70486b91d]) successfully deleted
W20260709 13:16:00.490391  8102 consensus_peers.cc:597] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 07fe988cafa34bcaa51af457b7248ccc. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:00.501451 15681 log.cc:927] Time spent T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:16:00.508047  8726 raft_consensus.cc:1275] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:16:00.511600 15640 consensus_queue.cc:1048] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:00.527289  8130 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:36736
I20260709 13:16:00.537055 15547 raft_consensus.cc:2955] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:16:00.559631  8176 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:44573, user_credentials={real_user=slave}} blocked reactor thread for 67387.6us
W20260709 13:16:00.565312  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 68811.2us
W20260709 13:16:00.566217  8175 consensus_peers.cc:597] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:00.570535  8130 raft_consensus.cc:606] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Received request to transfer leadership
I20260709 13:16:00.609632  8129 raft_consensus.cc:2955] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:16:00.617760  7948 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dad90abaa8b04508a5b086ff94502e4e with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:16:00.597255  8726 raft_consensus.cc:2955] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:16:00.644690  6937 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 8084)
W20260709 13:16:00.646332  8693 consensus_peers.cc:597] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:00.661077  8175 consensus_peers.cc:597] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:00.675280  7960 catalog_manager.cc:5697] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 20 to 21, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 21 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } }
W20260709 13:16:00.689169  8101 consensus_peers.cc:597] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:00.718663  8134 raft_consensus.cc:1275] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:16:00.724247  8129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:36736
I20260709 13:16:00.727562  8129 raft_consensus.cc:606] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Received request to transfer leadership
I20260709 13:16:00.740060  7957 catalog_manager.cc:5697] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 11 to 12, VOTER 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) evicted. New cstate: current_term: 13 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:16:00.751168  8175 consensus_peers.cc:597] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:00.760807  8175 consensus_peers.cc:597] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:00.761548  8175 consensus_peers.cc:597] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:00.763793 15547 consensus_queue.cc:1048] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:16:00.801440  7945 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 45359.4us
I20260709 13:16:00.863862 15642 raft_consensus.cc:993] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: : Instructing follower d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:16:00.864578 15629 raft_consensus.cc:1081] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Signalling peer d9cd2999aa314b39997b423ec1ae509e to start an election
I20260709 13:16:00.866640  8208 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc"
dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
 from {username='slave'} at 127.0.0.1:59868
I20260709 13:16:00.867291  8208 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:00.867681  8208 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:00.878221  8129 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc"
dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:36736
I20260709 13:16:00.878978  8129 raft_consensus.cc:606] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Received request to transfer leadership
W20260709 13:16:00.885759  8009 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:16:00.886245  8009 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 07fe988cafa34bcaa51af457b7248ccc already in progress
W20260709 13:16:00.886693  8009 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 07fe988cafa34bcaa51af457b7248ccc already in progress
I20260709 13:16:00.891113  8208 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:16:00.892846  8208 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:16:00.907356  8129 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fe988cafa34bcaa51af457b7248ccc" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: true dest_uuid: "8a968cada1e4446ebb2a5e167176e649"
I20260709 13:16:00.908080  8129 raft_consensus.cc:3055] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:00.908419  8129 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:16:00.909667  8129 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
I20260709 13:16:00.912005  8129 raft_consensus.cc:3060] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:00.938907  7945 catalog_manager.cc:4755] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dad90abaa8b04508a5b086ff94502e4e: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:00.945655  8129 raft_consensus.cc:2468] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 22.
I20260709 13:16:00.956480  8175 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:16:00.958572 15618 raft_consensus.cc:2804] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:00.959126 15618 raft_consensus.cc:697] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:16:00.960129 15618 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
W20260709 13:16:00.958138  8175 leader_election.cc:336] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:00.991252  7959 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 21 to 22, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 22 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:01.326510  8175 consensus_peers.cc:597] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:01.491035  8175 consensus_peers.cc:597] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:01.495605  8176 consensus_peers.cc:597] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e -> Peer 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573): Couldn't send request to peer 965e59f272f143449b18f65db4e61d98. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 07fe988cafa34bcaa51af457b7248ccc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:01.516218  8134 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:16:01.518561 15668 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:16:01.587345 15591 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.587913 15591 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:16:01.603986  8101 leader_election.cc:336] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:01.607861  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:16:01.608496  8726 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 22.
I20260709 13:16:01.609835  8102 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:16:01.610791 15603 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:01.611195 15603 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.611547 15603 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:01.614650 15629 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.615198 15629 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:16:01.618388  8204 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:16:01.619007  8204 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 22.
I20260709 13:16:01.620359  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:16:01.621142 15642 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:01.621232 15629 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:01.621580 15642 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.631896 15642 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:01.621165 15591 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:16:01.645228  6937 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 8084)
I20260709 13:16:01.652793 15642 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:16:01.660282  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a4992d168cb240009f8bfc8df07668e0" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:16:01.660883  8208 raft_consensus.cc:3060] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:16:01.660748  8101 leader_election.cc:336] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:01.663079 15642 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:01.670518 15603 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:01.672525 15603 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 election: Requested vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:16:01.673560  8208 raft_consensus.cc:2468] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 23.
I20260709 13:16:01.675464  8104 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:16:01.676326 15629 raft_consensus.cc:2804] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:01.676821 15629 raft_consensus.cc:697] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 23 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:16:01.677695 15629 consensus_queue.cc:237] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:16:01.691087  8726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e397a319b11487fbbacec77723b5739" candidate_uuid: "8a968cada1e4446ebb2a5e167176e649" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:16:01.691851  8726 raft_consensus.cc:3060] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:16:01.693682  8101 leader_election.cc:336] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:01.696749  8101 leader_election.cc:336] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:01.708483  8726 raft_consensus.cc:2468] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a968cada1e4446ebb2a5e167176e649 in term 23.
I20260709 13:16:01.709609  8102 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8
I20260709 13:16:01.710487 15694 raft_consensus.cc:2804] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:01.710969 15694 raft_consensus.cc:697] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 23 LEADER]: Becoming Leader. State: Replica: 8a968cada1e4446ebb2a5e167176e649, State: Running, Role: LEADER
I20260709 13:16:01.719099 15694 consensus_queue.cc:237] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:01.731554  7959 catalog_manager.cc:5697] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 21 to 23, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 23 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:01.807904  7959 catalog_manager.cc:5697] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: term changed from 21 to 23, leader changed from 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65) to 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66). New cstate: current_term: 23 leader_uuid: "8a968cada1e4446ebb2a5e167176e649" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:01.950418  8693 consensus_peers.cc:597] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:02.002669  8101 consensus_peers.cc:597] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:02.071434 15700 ts_tablet_manager.cc:933] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98: Initiating tablet copy from peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:02.080121 15700 tablet_copy_client.cc:323] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98: tablet copy: Beginning tablet copy session from remote peer at address 127.6.198.67:35745
W20260709 13:16:02.083604  8101 consensus_peers.cc:597] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:02.085255  8222 tablet_copy_service.cc:140] P d9cd2999aa314b39997b423ec1ae509e: Received BeginTabletCopySession request for tablet 07fe988cafa34bcaa51af457b7248ccc from peer 965e59f272f143449b18f65db4e61d98 ({username='slave'} at 127.0.0.1:55942)
I20260709 13:16:02.085855  8222 tablet_copy_service.cc:161] P d9cd2999aa314b39997b423ec1ae509e: Beginning new tablet copy session on tablet 07fe988cafa34bcaa51af457b7248ccc from peer 965e59f272f143449b18f65db4e61d98 at {username='slave'} at 127.0.0.1:55942: session id = 965e59f272f143449b18f65db4e61d98-07fe988cafa34bcaa51af457b7248ccc
I20260709 13:16:02.095621  8222 tablet_copy_source_session.cc:215] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:16:02.099251 15700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07fe988cafa34bcaa51af457b7248ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:02.136168  8726 raft_consensus.cc:1275] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:16:02.137557 15603 consensus_queue.cc:1048] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:16:02.142568 15700 tablet_copy_client.cc:806] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 0 data blocks...
I20260709 13:16:02.143782 15700 tablet_copy_client.cc:670] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98: tablet copy: Starting download of 1 WAL segments...
I20260709 13:16:02.148304 15700 tablet_copy_client.cc:538] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:16:02.167646  7945 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 50516.1us
W20260709 13:16:02.174274  8693 consensus_peers.cc:597] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:02.180434  7945 catalog_manager.cc:4755] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dad90abaa8b04508a5b086ff94502e4e: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:02.188701  8101 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111) [suppressed 423 similar messages]
I20260709 13:16:02.189800 15700 tablet_bootstrap.cc:492] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98: Bootstrap starting.
W20260709 13:16:02.191813  8101 consensus_peers.cc:597] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:02.193773  8693 consensus_peers.cc:597] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:02.213162  8208 raft_consensus.cc:1275] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Refusing update from remote peer 8a968cada1e4446ebb2a5e167176e649: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:16:02.214701 15603 consensus_queue.cc:1048] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:16:02.401049 15700 tablet_bootstrap.cc:492] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=3 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:02.402386 15700 tablet_bootstrap.cc:492] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98: Bootstrap complete.
I20260709 13:16:02.403184 15700 ts_tablet_manager.cc:1403] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98: Time spent bootstrapping tablet: real 0.214s	user 0.058s	sys 0.007s
I20260709 13:16:02.407918 15700 raft_consensus.cc:359] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
W20260709 13:16:02.408463  8175 consensus_peers.cc:597] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:02.408845 15700 raft_consensus.cc:740] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Initialized, Role: LEARNER
I20260709 13:16:02.410398 15700 consensus_queue.cc:260] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: NON_VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: true } }
W20260709 13:16:02.411744  8175 consensus_peers.cc:597] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:02.413337 15700 ts_tablet_manager.cc:1434] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
W20260709 13:16:02.454958  8175 consensus_peers.cc:597] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:02.645731  6937 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 8084)
W20260709 13:16:02.760169  8175 consensus_peers.cc:597] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:02.765027  8175 consensus_peers.cc:597] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:02.771615  8175 consensus_peers.cc:597] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:02.788416  8175 consensus_peers.cc:597] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:02.827239  8726 raft_consensus.cc:1217] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [term 22 LEARNER]: Deduplicated request from leader. Original: 21.15->[22.16-22.16]   Dedup: 22.16->[]
W20260709 13:16:02.880447  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 72351.1us
W20260709 13:16:02.952091  8175 consensus_peers.cc:597] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:02.953073  8175 consensus_peers.cc:597] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:02.954258  8175 consensus_peers.cc:597] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:02.976646  8222 tablet_copy_service.cc:342] P d9cd2999aa314b39997b423ec1ae509e: Request end of tablet copy session 965e59f272f143449b18f65db4e61d98-07fe988cafa34bcaa51af457b7248ccc received from {username='slave'} at 127.0.0.1:55942
I20260709 13:16:02.977119  8222 tablet_copy_service.cc:434] P d9cd2999aa314b39997b423ec1ae509e: ending tablet copy session 965e59f272f143449b18f65db4e61d98-07fe988cafa34bcaa51af457b7248ccc on tablet 07fe988cafa34bcaa51af457b7248ccc with peer 965e59f272f143449b18f65db4e61d98
W20260709 13:16:02.988437  8175 consensus_peers.cc:597] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:02.989542  8175 consensus_peers.cc:597] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:03.047513  8693 consensus_peers.cc:597] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:03.047899  8101 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 75430.3us
W20260709 13:16:03.052657  8101 consensus_peers.cc:597] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:03.053120  8101 consensus_peers.cc:597] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:03.063233  8101 consensus_peers.cc:597] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:03.065485  8101 consensus_peers.cc:597] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:03.065986  8101 consensus_peers.cc:597] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:03.066494  8693 consensus_peers.cc:597] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:03.067067  8693 consensus_peers.cc:597] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:03.067485  8693 consensus_peers.cc:597] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:03.067937  8693 consensus_peers.cc:597] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:03.068454  8693 consensus_peers.cc:597] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:03.068964  8693 consensus_peers.cc:597] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:03.069402  8693 consensus_peers.cc:597] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:03.072095  8101 consensus_peers.cc:597] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:03.120754 15729 raft_consensus.cc:1064] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: attempting to promote NON_VOTER 965e59f272f143449b18f65db4e61d98 to VOTER
I20260709 13:16:03.127761 15729 consensus_queue.cc:237] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 15, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:03.155673  8129 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:16:03.157429 15712 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:16:03.168713  8726 raft_consensus.cc:1275] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [term 22 LEARNER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:16:03.185797 15668 consensus_queue.cc:1048] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:16:03.216977  8175 consensus_peers.cc:597] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:03.225811  8101 consensus_peers.cc:597] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:03.213964 15712 raft_consensus.cc:2955] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Committing config change with OpId 22.17: config changed from index 14 to 17, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:16:03.215798  8724 raft_consensus.cc:2955] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Committing config change with OpId 22.17: config changed from index 14 to 17, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
I20260709 13:16:03.286156  8129 raft_consensus.cc:2955] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Committing config change with OpId 22.17: config changed from index 14 to 17, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:16:03.320631  8175 consensus_peers.cc:597] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:03.325827  7959 catalog_manager.cc:5697] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 reported cstate change: config changed from index 14 to 17, 965e59f272f143449b18f65db4e61d98 (127.6.198.68) changed from NON_VOTER to VOTER. New cstate: current_term: 22 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } } }
W20260709 13:16:03.330694  8101 consensus_peers.cc:597] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:03.345443  8693 consensus_peers.cc:597] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:03.419021  7945 catalog_manager.cc:4755] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dad90abaa8b04508a5b086ff94502e4e: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:03.420429  8175 consensus_peers.cc:597] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:03.426614  8101 consensus_peers.cc:597] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:03.436712  8101 consensus_peers.cc:597] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:03.445122  8101 consensus_peers.cc:597] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:03.450628  8175 consensus_peers.cc:597] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:03.460187  8175 consensus_peers.cc:597] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:03.480329  8751 debug-util.cc:398] Leaking SignalData structure 0x7b080039a300 after lost signal to thread 6940
W20260709 13:16:03.481496  8751 debug-util.cc:398] Leaking SignalData structure 0x7b08002effc0 after lost signal to thread 7995
W20260709 13:16:03.482857  8751 debug-util.cc:398] Leaking SignalData structure 0x7b080030c400 after lost signal to thread 8162
W20260709 13:16:03.484002  8751 debug-util.cc:398] Leaking SignalData structure 0x7b080004a4c0 after lost signal to thread 8236
W20260709 13:16:03.485167  8751 debug-util.cc:398] Leaking SignalData structure 0x7b08001f73c0 after lost signal to thread 8754
W20260709 13:16:03.600564  8101 consensus_peers.cc:597] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:03.646365  6937 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 8084)
W20260709 13:16:03.972472  8175 consensus_peers.cc:597] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:04.646816  6937 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 8084)
W20260709 13:16:04.716665  8101 consensus_peers.cc:597] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:04.822458  8693 consensus_peers.cc:597] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:04.828658  8175 consensus_peers.cc:597] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:04.866187  8101 consensus_peers.cc:597] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:04.897310  8693 consensus_peers.cc:597] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:04.907476  8175 consensus_peers.cc:597] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:04.971747  8101 consensus_peers.cc:597] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:04.982395  8693 consensus_peers.cc:597] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:04.992529  8101 consensus_peers.cc:597] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.046797  8693 consensus_peers.cc:597] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:05.068490  8175 consensus_peers.cc:597] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:05.132411  8693 consensus_peers.cc:597] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.176591  8101 consensus_peers.cc:597] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:05.178143  8693 consensus_peers.cc:597] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.178750  8175 consensus_peers.cc:597] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.263656  8175 consensus_peers.cc:597] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.265688  8175 consensus_peers.cc:597] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.308825  8693 consensus_peers.cc:597] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.339432  8101 consensus_peers.cc:597] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.397672  8693 consensus_peers.cc:597] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.398365  8693 consensus_peers.cc:597] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:05.420497  8101 consensus_peers.cc:597] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:05.422137  8693 consensus_peers.cc:597] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.444180  8693 consensus_peers.cc:597] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.452351  8693 consensus_peers.cc:597] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.453593  8101 consensus_peers.cc:597] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.462031  8101 consensus_peers.cc:597] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.465014  8175 consensus_peers.cc:597] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.474792  8175 consensus_peers.cc:597] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.480774  8175 consensus_peers.cc:597] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.501488  7945 catalog_manager.cc:4755] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dad90abaa8b04508a5b086ff94502e4e: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:05.574481  8175 consensus_peers.cc:597] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:05.621615  8175 consensus_peers.cc:597] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:05.629400  8175 consensus_peers.cc:597] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.632042  8175 consensus_peers.cc:597] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:05.647317  6937 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 8084)
W20260709 13:16:05.688120  8175 consensus_peers.cc:597] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:05.700043  8175 consensus_peers.cc:597] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.780019  8101 consensus_peers.cc:597] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:05.806166  8175 consensus_peers.cc:597] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:05.808923  8101 consensus_peers.cc:597] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:05.968796  8696 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:35745, user_credentials={real_user=slave}} blocked reactor thread for 69089.9us
W20260709 13:16:05.970705  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 70996us
W20260709 13:16:06.057807  8175 consensus_peers.cc:597] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:06.300269  8175 consensus_peers.cc:597] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:06.647825  6937 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 8084)
W20260709 13:16:06.905143  8175 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:45807, user_credentials={real_user=slave}} blocked reactor thread for 54669.6us
W20260709 13:16:06.945283  8101 consensus_peers.cc:597] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:06.945736  8101 consensus_peers.cc:597] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:06.946108  8101 consensus_peers.cc:597] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:06.960899  8101 consensus_peers.cc:597] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:07.184085  8693 consensus_peers.cc:597] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:07.184852  8175 consensus_peers.cc:597] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:07.197314  8101 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111) [suppressed 406 similar messages]
W20260709 13:16:07.326323  8175 consensus_peers.cc:597] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:07.375224  8693 consensus_peers.cc:597] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:07.416496  8175 consensus_peers.cc:597] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:07.471278  8693 consensus_peers.cc:597] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:07.484357  8693 consensus_peers.cc:597] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:07.546531  8693 consensus_peers.cc:597] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:07.640226  8101 consensus_peers.cc:597] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:07.641031  8693 consensus_peers.cc:597] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:07.771582  8159 debug-util.cc:398] Leaking SignalData structure 0x7b08003ee920 after lost signal to thread 6940
W20260709 13:16:07.772673  8159 debug-util.cc:398] Leaking SignalData structure 0x7b08003d78e0 after lost signal to thread 7995
W20260709 13:16:07.773900  8159 debug-util.cc:398] Leaking SignalData structure 0x7b08002b9d00 after lost signal to thread 8162
W20260709 13:16:07.774968  8159 debug-util.cc:398] Leaking SignalData structure 0x7b080027fc60 after lost signal to thread 8236
W20260709 13:16:07.776263  8159 debug-util.cc:398] Leaking SignalData structure 0x7b0800353f00 after lost signal to thread 8754
W20260709 13:16:07.776981  8693 consensus_peers.cc:597] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:07.785866  8175 consensus_peers.cc:597] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:07.808395  8693 consensus_peers.cc:597] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:07.815029  8101 consensus_peers.cc:597] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:07.842522  8175 consensus_peers.cc:597] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:07.872278  8693 consensus_peers.cc:597] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:07.874609  8175 consensus_peers.cc:597] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:07.915047  8175 consensus_peers.cc:597] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:07.951508  8101 consensus_peers.cc:597] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:16:07.977869  8693 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45919, user_credentials={real_user=slave}} blocked reactor thread for 50875.8us
W20260709 13:16:08.056434  8175 consensus_peers.cc:597] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:08.069625  8175 consensus_peers.cc:597] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:08.123567  6937 tablet_server.cc:196] TabletServer@127.6.198.65:0 shutdown complete.
W20260709 13:16:08.138916  8175 consensus_peers.cc:597] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:08.139442  8175 consensus_peers.cc:597] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:08.139994  8175 consensus_peers.cc:597] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:08.151751  8693 consensus_peers.cc:597] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:08.152324  8693 consensus_peers.cc:597] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:08.238437  8175 consensus_peers.cc:597] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:08.241817  8693 consensus_peers.cc:597] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:08.259035  8101 consensus_peers.cc:597] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:08.261224  8101 consensus_peers.cc:597] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:08.294361  6937 tablet_server.cc:179] TabletServer@127.6.198.66:0 shutting down...
W20260709 13:16:08.303640  8175 consensus_peers.cc:597] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:08.308853  8175 consensus_peers.cc:597] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.322369  8175 consensus_peers.cc:597] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:16:08.329977  8101 consensus_peers.cc:597] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:08.377992  8175 consensus_peers.cc:597] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.382216  8693 consensus_peers.cc:597] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.399428  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:08.401396  6937 tablet_replica.cc:333] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.402391  6937 raft_consensus.cc:2243] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.403638  6937 raft_consensus.cc:2272] T b9bd4708595b40438af51aba853a0f94 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.406592  6937 tablet_replica.cc:333] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.407470  6937 raft_consensus.cc:2243] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.408564  6937 raft_consensus.cc:2272] T 6771ba07746943f6a958ca7be1990296 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.411191  6937 tablet_replica.cc:333] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.412056  6937 raft_consensus.cc:2243] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.413213  6937 raft_consensus.cc:2272] T a003b4584d3747a381522d67299aaed2 P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.423251  6937 tablet_replica.cc:333] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.424194  6937 raft_consensus.cc:2243] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.425096  6937 raft_consensus.cc:2272] T bf628848d7464a21aa8f76f7eb56a408 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.427980  6937 tablet_replica.cc:333] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.428906  6937 raft_consensus.cc:2243] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.430460  6937 raft_consensus.cc:2272] T ade1b48685164d64a62772d186ac959f P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:08.433678  8175 consensus_peers.cc:597] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.433714  6937 tablet_replica.cc:333] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.434930  6937 raft_consensus.cc:2243] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.435968  6937 raft_consensus.cc:2272] T 8e397a319b11487fbbacec77723b5739 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.438655  6937 tablet_replica.cc:333] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.439551  6937 raft_consensus.cc:2243] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.440186  6937 raft_consensus.cc:2272] T 1b3101b6cc874539a962e23536fdabc9 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.443275  6937 tablet_replica.cc:333] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.444231  6937 raft_consensus.cc:2243] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.445034  6937 raft_consensus.cc:2272] T 4489591b74b1469d81269da167e98f71 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.447736  6937 tablet_replica.cc:333] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.448638  6937 raft_consensus.cc:2243] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.456097  6937 raft_consensus.cc:2272] T d013f25ff51143beb65d31cc33e5a248 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.459666  6937 tablet_replica.cc:333] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.460559  6937 raft_consensus.cc:2243] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.461266  6937 raft_consensus.cc:2272] T ec8da6c869684688957d5323fb023bc5 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.464389  6937 tablet_replica.cc:333] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.465243  6937 raft_consensus.cc:2243] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.466050  6937 raft_consensus.cc:2272] T 0ee839503bc34e5fb2b1f4e09dfe9784 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.468722  6937 tablet_replica.cc:333] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.469477  6937 raft_consensus.cc:2243] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.470568  6937 raft_consensus.cc:2272] T 0e4fa00ecc35497e823a1329fe771042 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.473246  6937 tablet_replica.cc:333] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.474109  6937 raft_consensus.cc:2243] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.475049  6937 raft_consensus.cc:2272] T dad90abaa8b04508a5b086ff94502e4e P 8a968cada1e4446ebb2a5e167176e649 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:08.484445  8101 consensus_peers.cc:597] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:08.485440  6937 tablet_replica.cc:333] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.487073  6937 raft_consensus.cc:2243] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.488379  6937 raft_consensus.cc:2272] T b578d88c1c1f4ca58b7291fed4909f09 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:08.489902  8175 consensus_peers.cc:597] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:08.491683  6937 tablet_replica.cc:333] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.492559  6937 raft_consensus.cc:2243] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.493187  6937 raft_consensus.cc:2272] T 8dd7949df539418cbc00001111885883 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.496436  6937 tablet_replica.cc:333] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
W20260709 13:16:08.496443  8175 consensus_peers.cc:597] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:08.497632  6937 raft_consensus.cc:2243] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.504657  6937 raft_consensus.cc:2272] T fe73c60f6d9f4d25a3dffad1e7198694 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:08.511121  8101 consensus_peers.cc:597] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:08.513270  8693 consensus_peers.cc:597] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.514415  6937 tablet_replica.cc:333] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.515367  6937 raft_consensus.cc:2243] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.515993  6937 raft_consensus.cc:2272] T d80e5571c873417aa73c79508290e836 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.520875  6937 tablet_replica.cc:333] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
W20260709 13:16:08.520874  8175 consensus_peers.cc:597] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.522184  6937 raft_consensus.cc:2243] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.523345  6937 raft_consensus.cc:2272] T 6cbbc2cb4def4ea09659a9cdc5087f65 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.527335  6937 tablet_replica.cc:333] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.528199  6937 raft_consensus.cc:2243] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.528877  6937 raft_consensus.cc:2272] T 3d785324a51949a79a4de6f0d836be56 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.531407  6937 tablet_replica.cc:333] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.532264  6937 raft_consensus.cc:2243] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.532886  6937 raft_consensus.cc:2272] T a3e2b86010604477955f19a56daf6d04 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.536302  6937 tablet_replica.cc:333] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.537158  6937 raft_consensus.cc:2243] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.537797  6937 raft_consensus.cc:2272] T 62c9c8ea5bb04f99add9743c31017f37 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.540359  6937 tablet_replica.cc:333] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.541399  6937 raft_consensus.cc:2243] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:16:08.548424  8693 consensus_peers.cc:597] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.549618  8175 consensus_peers.cc:597] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.551530  6937 raft_consensus.cc:2272] T f265c49b04e34e9fa83eed8ecdb90607 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.554600  6937 tablet_replica.cc:333] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.555567  6937 raft_consensus.cc:2243] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.556674  6937 raft_consensus.cc:2272] T 542d60a75b684950a46a64c1d64b80a5 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:08.565279  8693 consensus_peers.cc:597] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.579860  6937 tablet_replica.cc:333] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
W20260709 13:16:08.611600  8175 consensus_peers.cc:597] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.602908  8693 consensus_peers.cc:597] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.617324  8693 consensus_peers.cc:597] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.619849  8693 consensus_peers.cc:597] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.622473  8175 consensus_peers.cc:597] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.590512  6937 raft_consensus.cc:2243] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.628717  6937 raft_consensus.cc:2272] T 18de311592f1492b8b460305c56e6749 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.631347  6937 tablet_replica.cc:333] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.640060  6937 raft_consensus.cc:2243] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:16:08.639087  8175 consensus_peers.cc:597] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.642324  8175 consensus_peers.cc:597] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.640772  6937 raft_consensus.cc:2272] T 5b4358ace86545339036f21610735b47 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.646145  6937 tablet_replica.cc:333] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
W20260709 13:16:08.651871  8693 consensus_peers.cc:597] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.655397  8175 consensus_peers.cc:597] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:08.659046  8175 consensus_peers.cc:597] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.661851  6937 raft_consensus.cc:2243] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:16:08.667912  8101 consensus_peers.cc:597] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:16:08.668885  6937 raft_consensus.cc:2272] T e529be00baff4bfab3a3d0a45d84c238 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.671949  6937 tablet_replica.cc:333] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.673038  6937 raft_consensus.cc:2243] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.674355  6937 raft_consensus.cc:2272] T 51726a5d5cad44fd8e9b6803da998cb2 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.677454  6937 tablet_replica.cc:333] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.678330  6937 raft_consensus.cc:2243] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.679059  6937 raft_consensus.cc:2272] T 1f229a5eb4514abc9183538dd97073d2 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.681739  6937 tablet_replica.cc:333] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.682546  6937 raft_consensus.cc:2243] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.683333  6937 raft_consensus.cc:2272] T 481500a64cdb4a7fb94658807295dcbe P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:08.687270  8693 consensus_peers.cc:597] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.689384  6937 tablet_replica.cc:333] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.690233  6937 raft_consensus.cc:2243] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.691030  6937 raft_consensus.cc:2272] T 7c4e1687a1db419c8863789bb37c260a P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.698062  6937 tablet_replica.cc:333] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
W20260709 13:16:08.703251  8175 consensus_peers.cc:597] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.698953  6937 raft_consensus.cc:2243] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:16:08.713999  8693 consensus_peers.cc:597] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.704346  6937 raft_consensus.cc:2272] T bd0294d309cb4f8696e693285a7b5b04 P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.725425  6937 tablet_replica.cc:333] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.726454  6937 raft_consensus.cc:2243] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.727571  6937 raft_consensus.cc:2272] T 710721029c8047ff8b9fc6d5920859f4 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.730257  6937 tablet_replica.cc:333] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.731058  6937 raft_consensus.cc:2243] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.732122  6937 raft_consensus.cc:2272] T 053e9e9e90fe4d92853de2c0599484aa P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.734855  6937 tablet_replica.cc:333] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.735738  6937 raft_consensus.cc:2243] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.736495  6937 raft_consensus.cc:2272] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.739084  6937 tablet_replica.cc:333] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.739914  6937 raft_consensus.cc:2243] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.740710  6937 raft_consensus.cc:2272] T 07fe988cafa34bcaa51af457b7248ccc P 8a968cada1e4446ebb2a5e167176e649 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.743191  6937 tablet_replica.cc:333] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.744100  6937 raft_consensus.cc:2243] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.745365  6937 raft_consensus.cc:2272] T f4539f806feb4e37900025e737458841 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:08.757570  8175 consensus_peers.cc:597] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.758610  6937 tablet_replica.cc:333] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.759486  6937 raft_consensus.cc:2243] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.760921  6937 raft_consensus.cc:2272] T a4992d168cb240009f8bfc8df07668e0 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.771970  6937 tablet_replica.cc:333] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.774957  6937 raft_consensus.cc:2243] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.775566  6937 raft_consensus.cc:2272] T 5124ddc96f984ec2ae0ee97be6732a90 P 8a968cada1e4446ebb2a5e167176e649 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.778247  6937 tablet_replica.cc:333] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.779052  6937 raft_consensus.cc:2243] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.780113  6937 raft_consensus.cc:2272] T cc8454910b36474c90b678c57cea9773 P 8a968cada1e4446ebb2a5e167176e649 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:08.784154  8175 consensus_peers.cc:597] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.786931  6937 tablet_replica.cc:333] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.787757  6937 raft_consensus.cc:2243] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.793242  6937 raft_consensus.cc:2272] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.796063  6937 tablet_replica.cc:333] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.796887  6937 raft_consensus.cc:2243] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.797974  6937 raft_consensus.cc:2272] T f6b3f42115d64c26b2b412d669ed6010 P 8a968cada1e4446ebb2a5e167176e649 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.800599  6937 tablet_replica.cc:333] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.801397  6937 raft_consensus.cc:2243] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.801999  6937 raft_consensus.cc:2272] T b186be823643475982e269b73d886a42 P 8a968cada1e4446ebb2a5e167176e649 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.804661  6937 tablet_replica.cc:333] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.805513  6937 raft_consensus.cc:2243] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:08.806176  6937 raft_consensus.cc:2272] T 9871516a41a2419d8af8e6fe9e0a5fa1 P 8a968cada1e4446ebb2a5e167176e649 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.810756  6937 tablet_replica.cc:333] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.812589  6937 raft_consensus.cc:2243] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.813664  6937 raft_consensus.cc:2272] T 17c830696ad4418b88d85b6a97e41851 P 8a968cada1e4446ebb2a5e167176e649 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.819613  6937 tablet_replica.cc:333] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.821652  6937 raft_consensus.cc:2243] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Raft consensus shutting down.
W20260709 13:16:08.826223  8175 consensus_peers.cc:597] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:08.827441  6937 raft_consensus.cc:2272] T ac625d2d87774bc7be4ff2189dee7eb0 P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:08.830127  6937 tablet_replica.cc:333] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.831663  6937 raft_consensus.cc:2243] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.834373  6937 raft_consensus.cc:2272] T b8b8bb0a8e3545218708596da648262f P 8a968cada1e4446ebb2a5e167176e649 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:08.835827  8175 consensus_peers.cc:597] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:08.838122  6937 tablet_replica.cc:333] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649: stopping tablet replica
I20260709 13:16:08.839128  6937 raft_consensus.cc:2243] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:16:08.840211  6937 raft_consensus.cc:2272] T 5d0e2cd74dac4358aba60edf5382befd P 8a968cada1e4446ebb2a5e167176e649 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:08.865662  8175 consensus_peers.cc:597] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:09.372552 15763 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:09.373114 15763 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:09.376682  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:16:09.378379 15763 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:16:09.383669  8693 leader_election.cc:336] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:09.384150  8693 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:09.384886 15763 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:09.536172 15729 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:09.536695 15729 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:09.539615 15729 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:16:09.540962  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:16:09.542275  8175 leader_election.cc:336] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:09.542869  8176 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 965e59f272f143449b18f65db4e61d98
I20260709 13:16:09.543695 15729 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:09.544112  8175 leader_election.cc:336] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:09.559144 15763 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:09.559697 15763 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:09.561640 15763 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:16:09.568970  8693 leader_election.cc:336] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:09.569749  8693 leader_election.cc:336] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:09.570134  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:09.570803 15775 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:09.601892  8175 consensus_peers.cc:597] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:16:09.612385 15775 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:09.612913 15775 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:09.615960 15775 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:09.616503  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:16:09.616961 15775 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:09.618677 15775 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:16:09.620031  8693 leader_election.cc:336] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:09.620455  8693 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:09.620631 15775 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:16:09.621259 15773 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:16:09.628499  8693 leader_election.cc:336] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:09.629128  8693 leader_election.cc:336] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:09.629495  8693 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:09.630260 15775 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:16:09.651603  7945 catalog_manager.cc:4755] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dad90abaa8b04508a5b086ff94502e4e: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:09.669369  8693 consensus_peers.cc:597] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:16:09.706512 15775 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:09.707055 15775 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:09.709708 15775 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:09.712836  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade1b48685164d64a62772d186ac959f" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:16:09.713514  8209 raft_consensus.cc:2468] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 20.
W20260709 13:16:09.718871  8693 leader_election.cc:336] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:09.719519  8693 leader_election.cc:336] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:09.719954  8693 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:09.720028 15775 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:09.742142 15775 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:09.744515 15775 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:16:09.755461 15773 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:09.784969 15771 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:09.785571 15771 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:09.787679 15771 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:16:09.813920  8693 leader_election.cc:336] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:09.815522  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:16:09.821159  8693 leader_election.cc:336] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:09.827901  8693 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:09.832939 15783 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:09.833551 15729 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:09.835758 15729 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:09.838155 15729 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:16:09.839879  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:16:09.840467  8721 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 21.
I20260709 13:16:09.841473  8176 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:16:09.842253 15729 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:09.842656 15729 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:09.843016 15729 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:09.869707 15773 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:09.870270 15773 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:09.871435 15781 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:09.872694 15781 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:09.874895 15781 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:16:09.881485  8693 consensus_peers.cc:597] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:09.884332 15773 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:09.885851 15729 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:16:09.888752  8693 leader_election.cc:336] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:09.888767  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f265c49b04e34e9fa83eed8ecdb90607" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
W20260709 13:16:09.889622  8693 leader_election.cc:336] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:09.889653  8721 raft_consensus.cc:3060] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:09.890098 15729 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:16:09.891055  8693 consensus_peers.cc:597] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:09.891451  8693 leader_election.cc:336] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:09.892402  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:09.893543 15773 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:09.896833  8721 raft_consensus.cc:2468] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 22.
W20260709 13:16:09.917246  8693 consensus_peers.cc:597] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:16:09.923995  8693 consensus_peers.cc:597] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:16:09.958686  8176 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:16:09.974161 15774 raft_consensus.cc:2804] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:09.975754 15729 raft_consensus.cc:493] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:09.976996 15729 raft_consensus.cc:515] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:16:09.978710 15729 leader_election.cc:290] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:16:09.979210 15774 raft_consensus.cc:697] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:16:09.980151 15774 consensus_queue.cc:237] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:09.983919  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:16:09.984516  8209 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:16:09.985671  8696 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:09.986429 15781 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:09.986789 15781 raft_consensus.cc:493] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:09.987244 15781 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:10.021072 15783 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.021646 15783 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.023692 15783 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:16:10.029006 15771 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.029572 15771 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.030766 15773 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.031562 15771 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:16:10.031322 15773 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.032886 15782 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.033393 15782 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.034320 15773 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:10.034464  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:16:10.035080  8209 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 20.
I20260709 13:16:10.035287 15782 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:16:10.036093  8696 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:16:10.036906 15782 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:10.037297 15782 raft_consensus.cc:493] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.037704 15782 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:10.041505  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:16:10.047340 15771 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.047899 15771 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.049563 15729 raft_consensus.cc:493] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.050024 15729 raft_consensus.cc:515] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:16:10.051137 15771 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:16:10.052407  8723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
W20260709 13:16:10.049152  8175 leader_election.cc:336] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.053051  8723 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 18.
I20260709 13:16:10.054163  8176 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:16:10.053158  8175 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:16:10.055465 15771 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:10.055876 15771 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.051486 15782 raft_consensus.cc:515] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.057015  8721 raft_consensus.cc:2393] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d9cd2999aa314b39997b423ec1ae509e in current term 21: Already voted for candidate 965e59f272f143449b18f65db4e61d98 in this term.
I20260709 13:16:10.056229 15771 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:10.058779  8209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4539f806feb4e37900025e737458841" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:16:10.059333  8209 raft_consensus.cc:3060] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:10.061630 15784 raft_consensus.cc:2749] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:10.062458 15782 leader_election.cc:290] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:10.051761 15729 leader_election.cc:290] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:16:10.065418 15781 raft_consensus.cc:515] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.066874 15783 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.067358 15783 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.069335 15783 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:10.070329  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:16:10.082736 15771 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.084529  8209 raft_consensus.cc:2468] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 21.
I20260709 13:16:10.085186 15771 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:16:10.088454  8696 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:16:10.089357 15782 raft_consensus.cc:2804] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:10.089865 15782 raft_consensus.cc:697] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 21 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:16:10.090752 15782 consensus_queue.cc:237] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.087455  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9bd4708595b40438af51aba853a0f94" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:16:10.088506 15771 raft_consensus.cc:493] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.092568  8721 raft_consensus.cc:3060] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:10.092625 15771 raft_consensus.cc:515] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:16:10.094453 15771 leader_election.cc:290] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:16:10.094870 15771 raft_consensus.cc:493] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.086730  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6771ba07746943f6a958ca7be1990296" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:16:10.095793  8208 raft_consensus.cc:3060] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:16:10.098003  8175 leader_election.cc:336] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:10.098778  8175 leader_election.cc:336] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.095211 15771 raft_consensus.cc:515] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:16:10.103942 15771 leader_election.cc:290] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:16:10.102123  8208 raft_consensus.cc:2468] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 20.
I20260709 13:16:10.105299  8696 leader_election.cc:304] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:16:10.106102  7957 catalog_manager.cc:5697] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 21 to 22, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 22 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:10.108636  8175 consensus_peers.cc:597] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:16:10.109525 15783 raft_consensus.cc:2804] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:10.110317 15783 raft_consensus.cc:697] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 20 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:16:10.111174 15783 consensus_queue.cc:237] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.114614 15784 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.115089 15784 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.116919 15784 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:16:10.117493 15729 raft_consensus.cc:493] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.121349 15729 raft_consensus.cc:515] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:16:10.123066 15729 leader_election.cc:290] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:16:10.119632  8725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:16:10.123739  8725 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 15.
I20260709 13:16:10.124944  8176 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:16:10.127996 15774 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:10.128423 15774 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.133484 15774 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:10.134989 15781 leader_election.cc:290] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:10.136471  8721 raft_consensus.cc:2468] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 19.
I20260709 13:16:10.137646  8176 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
W20260709 13:16:10.131825  8693 leader_election.cc:336] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:10.140231  8693 leader_election.cc:336] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.141755 15785 raft_consensus.cc:2804] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:16:10.141826  8175 consensus_peers.cc:597] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:16:10.142660 15785 raft_consensus.cc:697] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
W20260709 13:16:10.143059  8693 leader_election.cc:336] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.143312  7959 catalog_manager.cc:5697] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 20 to 21, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 21 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:10.143652 15785 consensus_queue.cc:237] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.146167 15774 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.165901  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8b8bb0a8e3545218708596da648262f" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98"
I20260709 13:16:10.166445  8721 raft_consensus.cc:3060] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:10.172533  8721 raft_consensus.cc:2468] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9cd2999aa314b39997b423ec1ae509e in term 16.
I20260709 13:16:10.173453  8176 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:16:10.175034 15729 raft_consensus.cc:2804] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:10.175500 15729 raft_consensus.cc:697] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Becoming Leader. State: Replica: d9cd2999aa314b39997b423ec1ae509e, State: Running, Role: LEADER
I20260709 13:16:10.176209 15729 consensus_queue.cc:237] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:16:10.143486  8175 leader_election.cc:336] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:10.182499  8175 leader_election.cc:336] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:10.184435  8693 leader_election.cc:336] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:10.184811  8693 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:16:10.185701  8693 consensus_peers.cc:597] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:16:10.182991  8175 consensus_peers.cc:597] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:10.187177 15783 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:10.192767  7959 catalog_manager.cc:5697] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 19 to 20, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 20 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:16:10.195590  7957 catalog_manager.cc:5697] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 18 to 19, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 19 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:10.200867 15774 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
W20260709 13:16:10.186088  8693 leader_election.cc:336] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:10.205837  8175 leader_election.cc:336] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.206451  8175 leader_election.cc:304] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:10.207196 15785 raft_consensus.cc:2749] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:10.204041  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:10.208986 15781 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:16:10.207511  8175 leader_election.cc:336] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.212113  8175 leader_election.cc:304] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
I20260709 13:16:10.213781 15774 raft_consensus.cc:2749] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:10.215945  8693 leader_election.cc:336] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:10.216889  8175 leader_election.cc:336] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:10.217948  8175 leader_election.cc:336] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:10.218550  8693 leader_election.cc:336] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.219666  8693 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:10.218955  8175 leader_election.cc:304] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:10.218953  7959 catalog_manager.cc:5697] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e reported cstate change: term changed from 15 to 16, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to d9cd2999aa314b39997b423ec1ae509e (127.6.198.67). New cstate: current_term: 16 leader_uuid: "d9cd2999aa314b39997b423ec1ae509e" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:10.220821 15781 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:10.221639 15774 raft_consensus.cc:2749] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:10.223073  8693 leader_election.cc:336] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:10.223845  8175 leader_election.cc:336] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.240499 15774 raft_consensus.cc:493] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.241859 15774 raft_consensus.cc:515] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
I20260709 13:16:10.243763 15774 leader_election.cc:290] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:16:10.246116  8175 leader_election.cc:336] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:10.246644  8175 leader_election.cc:336] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:10.247578  8175 leader_election.cc:336] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:10.248867  8175 leader_election.cc:336] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.249272  8175 leader_election.cc:304] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
W20260709 13:16:10.250043  8175 leader_election.cc:336] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.250377 15774 raft_consensus.cc:493] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.250589  8175 leader_election.cc:304] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:10.250907 15774 raft_consensus.cc:515] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } }
W20260709 13:16:10.251662  8175 leader_election.cc:336] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:10.252369  8175 leader_election.cc:336] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.252696  8175 leader_election.cc:304] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:10.252940 15774 leader_election.cc:290] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:16:10.253809 15771 raft_consensus.cc:2749] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:10.253859 15784 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.254535 15784 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.256542 15784 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), 965e59f272f143449b18f65db4e61d98 (127.6.198.68:44573)
I20260709 13:16:10.253474 15774 raft_consensus.cc:2749] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:10.260375  8721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "d9cd2999aa314b39997b423ec1ae509e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "965e59f272f143449b18f65db4e61d98" is_pre_election: true
I20260709 13:16:10.250043 15795 raft_consensus.cc:2749] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:16:10.269904  8175 consensus_peers.cc:597] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:16:10.271374  8693 consensus_peers.cc:597] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:16:10.275338  8175 leader_election.cc:336] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:10.275821  8175 leader_election.cc:336] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:10.276278  8175 leader_election.cc:336] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.276608  8175 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649, 965e59f272f143449b18f65db4e61d98
W20260709 13:16:10.277441  8175 leader_election.cc:336] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.279066 15795 raft_consensus.cc:2749] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:10.284977  8175 leader_election.cc:336] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:10.285349  8175 leader_election.cc:304] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:10.286934 15795 raft_consensus.cc:2749] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:10.287829  8175 consensus_peers.cc:597] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:16:10.288208  8175 leader_election.cc:336] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:10.288534  8175 leader_election.cc:304] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9cd2999aa314b39997b423ec1ae509e; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:10.289148 15774 raft_consensus.cc:2749] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:10.294911  6937 tablet_server.cc:196] TabletServer@127.6.198.66:0 shutdown complete.
W20260709 13:16:10.305770  8693 consensus_peers.cc:597] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:16:10.312350 15783 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.312945 15783 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.314663 15783 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:10.316074  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e" is_pre_election: true
I20260709 13:16:10.316658  8208 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 18.
I20260709 13:16:10.317620  8696 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 
I20260709 13:16:10.318360 15783 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:10.318742 15783 raft_consensus.cc:493] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.319139 15783 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:10.320333  8693 leader_election.cc:336] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.323666 15781 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:10.324265 15781 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.326562 15781 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:16:10.326658 15783 raft_consensus.cc:515] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.330910 15783 leader_election.cc:290] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Requested vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:10.331936  8208 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c830696ad4418b88d85b6a97e41851" candidate_uuid: "965e59f272f143449b18f65db4e61d98" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "d9cd2999aa314b39997b423ec1ae509e"
I20260709 13:16:10.335075  8208 raft_consensus.cc:3060] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:10.338582  8693 leader_election.cc:336] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:10.339294  8693 leader_election.cc:336] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.339668  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:10.340394 15781 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:16:10.341382  8693 leader_election.cc:336] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:10.342182  8208 raft_consensus.cc:2468] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 965e59f272f143449b18f65db4e61d98 in term 19.
I20260709 13:16:10.343232  8696 leader_election.cc:304] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 965e59f272f143449b18f65db4e61d98, d9cd2999aa314b39997b423ec1ae509e; no voters: 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:10.343952 15783 raft_consensus.cc:2804] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:10.344518 15783 raft_consensus.cc:697] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Becoming Leader. State: Replica: 965e59f272f143449b18f65db4e61d98, State: Running, Role: LEADER
I20260709 13:16:10.345534 15783 consensus_queue.cc:237] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:10.365291  7959 catalog_manager.cc:5697] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 reported cstate change: term changed from 18 to 19, leader changed from 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66) to 965e59f272f143449b18f65db4e61d98 (127.6.198.68). New cstate: current_term: 19 leader_uuid: "965e59f272f143449b18f65db4e61d98" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:16:10.367969  8693 consensus_peers.cc:597] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:16:10.399521  8175 consensus_peers.cc:597] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:16:10.403949  8175 consensus_peers.cc:597] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:10.424674  6937 tablet_server.cc:179] TabletServer@127.6.198.67:0 shutting down...
W20260709 13:16:10.440882  8696 consensus_peers.cc:597] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.444635  8693 consensus_peers.cc:597] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:16:10.452105  8696 consensus_peers.cc:597] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.473438  8175 consensus_peers.cc:597] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.477456  8696 consensus_peers.cc:597] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.482635  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:10.483577  6937 tablet_replica.cc:333] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.484509  6937 raft_consensus.cc:2243] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.485210  6937 raft_consensus.cc:2272] T ade1b48685164d64a62772d186ac959f P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.487308  6937 tablet_replica.cc:333] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.487856  6937 raft_consensus.cc:2243] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.488571  6937 raft_consensus.cc:2272] T 1b3101b6cc874539a962e23536fdabc9 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.490589  6937 tablet_replica.cc:333] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.491303  6937 raft_consensus.cc:2243] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.491722  6937 raft_consensus.cc:2272] T 6771ba07746943f6a958ca7be1990296 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.494032  6937 tablet_replica.cc:333] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.494746  6937 raft_consensus.cc:2243] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.495151  6937 raft_consensus.cc:2272] T 1002e40536a34729929022702d03444c P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.497027  6937 tablet_replica.cc:333] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.497568  6937 raft_consensus.cc:2243] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.498276  6937 pending_rounds.cc:70] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Trying to abort 1 pending ops.
I20260709 13:16:10.498539  6937 pending_rounds.cc:77] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e: Aborting op as it isn't in flight: id { term: 19 index: 17 } timestamp: 7305637765794107392 op_type: NO_OP noop_request { }
I20260709 13:16:10.498953  6937 raft_consensus.cc:2889] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:16:10.499258  6937 raft_consensus.cc:2272] T b9bd4708595b40438af51aba853a0f94 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.499926  8693 consensus_peers.cc:597] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.502481  6937 tablet_replica.cc:333] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.503319  6937 raft_consensus.cc:2243] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.503769  6937 raft_consensus.cc:2272] T f7fb8453eda243f9856f489af50e67a2 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.505808  6937 tablet_replica.cc:333] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.506467  6937 raft_consensus.cc:2243] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.507481  6937 raft_consensus.cc:2272] T 07fe988cafa34bcaa51af457b7248ccc P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.509099  6937 tablet_replica.cc:333] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.509599  6937 raft_consensus.cc:2243] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.510365  6937 raft_consensus.cc:2272] T 0ee839503bc34e5fb2b1f4e09dfe9784 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.513527  6937 tablet_replica.cc:333] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.514178  6937 raft_consensus.cc:2243] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.515163  6937 raft_consensus.cc:2272] T ec8da6c869684688957d5323fb023bc5 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.517176  6937 tablet_replica.cc:333] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.517746  6937 raft_consensus.cc:2243] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.518605  6937 raft_consensus.cc:2272] T b578d88c1c1f4ca58b7291fed4909f09 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.520246  6937 tablet_replica.cc:333] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.520789  6937 raft_consensus.cc:2243] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.521464  6937 raft_consensus.cc:2272] T dad90abaa8b04508a5b086ff94502e4e P d9cd2999aa314b39997b423ec1ae509e [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.523097  6937 tablet_replica.cc:333] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.523586  6937 raft_consensus.cc:2243] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.524293  6937 raft_consensus.cc:2272] T ed7f54a64a14435691eeef2ece07e5dc P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.560771  6937 tablet_replica.cc:333] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.564148  6937 raft_consensus.cc:2243] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.566284  6937 raft_consensus.cc:2272] T 17c830696ad4418b88d85b6a97e41851 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.570631  6937 tablet_replica.cc:333] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.572268  6937 raft_consensus.cc:2243] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.574445  6937 raft_consensus.cc:2272] T a3e2b86010604477955f19a56daf6d04 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.578246  6937 tablet_replica.cc:333] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.579879  6937 raft_consensus.cc:2243] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.582971  6937 raft_consensus.cc:2272] T 62c9c8ea5bb04f99add9743c31017f37 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.585703  8721 raft_consensus.cc:1275] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:16:10.586665  6937 tablet_replica.cc:333] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.587857  6937 raft_consensus.cc:2243] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.588111 15785 consensus_queue.cc:1048] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:16:10.590642  6937 raft_consensus.cc:2272] T 3d785324a51949a79a4de6f0d836be56 P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.596475  6937 tablet_replica.cc:333] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.598138  6937 raft_consensus.cc:2243] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 13:16:10.598488  8696 consensus_peers.cc:597] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.599370  8696 consensus_peers.cc:597] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.599782  6937 raft_consensus.cc:2272] T 6cbbc2cb4def4ea09659a9cdc5087f65 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.603050  6937 tablet_replica.cc:333] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.604383  6937 raft_consensus.cc:2243] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.607465  6937 raft_consensus.cc:2272] T fe73c60f6d9f4d25a3dffad1e7198694 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.609479  8696 consensus_peers.cc:597] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.611079  6937 tablet_replica.cc:333] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.612541  6937 raft_consensus.cc:2243] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.614871  6937 raft_consensus.cc:2272] T d80e5571c873417aa73c79508290e836 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.618024  8721 raft_consensus.cc:1275] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Refusing update from remote peer d9cd2999aa314b39997b423ec1ae509e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:16:10.618546  6937 tablet_replica.cc:333] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.619712 15774 consensus_queue.cc:1048] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [LEADER]: Connected to new peer: Peer: permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:10.619711  6937 raft_consensus.cc:2243] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.624078  6937 raft_consensus.cc:2272] T 1dfd4edf9eab4bcc8b8aad94587349c3 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.625813  8693 consensus_peers.cc:597] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.629384  6937 tablet_replica.cc:333] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.630528  6937 raft_consensus.cc:2243] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.632925  6937 raft_consensus.cc:2272] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P d9cd2999aa314b39997b423ec1ae509e [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.636651  6937 tablet_replica.cc:333] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.637624  6937 raft_consensus.cc:2243] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.638597  6937 raft_consensus.cc:2272] T 1f229a5eb4514abc9183538dd97073d2 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.642184  6937 tablet_replica.cc:333] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.643092  6937 raft_consensus.cc:2243] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.644945  6937 raft_consensus.cc:2272] T 7c4e1687a1db419c8863789bb37c260a P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.647806  6937 tablet_replica.cc:333] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.648638  6937 raft_consensus.cc:2243] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 18 LEADER]: Raft consensus shutting down.
W20260709 13:16:10.652032  8693 consensus_peers.cc:597] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:16:10.659044  6937 raft_consensus.cc:2272] T 18de311592f1492b8b460305c56e6749 P d9cd2999aa314b39997b423ec1ae509e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.662700  6937 tablet_replica.cc:333] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.663622  6937 raft_consensus.cc:2243] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.665690  6937 raft_consensus.cc:2272] T 5b4358ace86545339036f21610735b47 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.668784  6937 tablet_replica.cc:333] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
W20260709 13:16:10.665786  8175 consensus_peers.cc:597] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.674029  6937 raft_consensus.cc:2243] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.677547  6937 raft_consensus.cc:2272] T e529be00baff4bfab3a3d0a45d84c238 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.680554  6937 tablet_replica.cc:333] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.681387  6937 raft_consensus.cc:2243] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.682283  6937 raft_consensus.cc:2272] T 710721029c8047ff8b9fc6d5920859f4 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.685464  6937 tablet_replica.cc:333] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.686720  6937 raft_consensus.cc:2243] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.687559  6937 raft_consensus.cc:2272] T bd0294d309cb4f8696e693285a7b5b04 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.688362  8696 consensus_peers.cc:597] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:10.690451  6937 tablet_replica.cc:333] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.691381  6937 raft_consensus.cc:2243] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.692250  6937 raft_consensus.cc:2272] T 053e9e9e90fe4d92853de2c0599484aa P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.693048  8175 consensus_peers.cc:597] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:16:10.695565  6937 tablet_replica.cc:333] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.701684  6937 raft_consensus.cc:2243] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.704753  6937 raft_consensus.cc:2272] T d4f33fbb7b794128a4668f6fda160937 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.708168  6937 tablet_replica.cc:333] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.708976  6937 raft_consensus.cc:2243] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.710812  6937 raft_consensus.cc:2272] T a478a4313e6149378cae4b6f833e3c2a P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.713205  6937 tablet_replica.cc:333] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.713821  6937 raft_consensus.cc:2243] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.715217  6937 raft_consensus.cc:2272] T b8b8bb0a8e3545218708596da648262f P d9cd2999aa314b39997b423ec1ae509e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.717607  6937 tablet_replica.cc:333] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.718403  6937 raft_consensus.cc:2243] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.719084  6937 raft_consensus.cc:2272] T cc8454910b36474c90b678c57cea9773 P d9cd2999aa314b39997b423ec1ae509e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.721860  6937 tablet_replica.cc:333] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.722802  6937 raft_consensus.cc:2243] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.724027  6937 raft_consensus.cc:2272] T c97c3defa48b473aaf762b3a27c0cba0 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.727149  6937 tablet_replica.cc:333] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.727980  6937 raft_consensus.cc:2243] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.729532  6937 raft_consensus.cc:2272] T ac625d2d87774bc7be4ff2189dee7eb0 P d9cd2999aa314b39997b423ec1ae509e [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.729532  8693 consensus_peers.cc:597] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:16:10.732512  6937 tablet_replica.cc:333] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.733220  6937 raft_consensus.cc:2243] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.734858  6937 raft_consensus.cc:2272] T 9871516a41a2419d8af8e6fe9e0a5fa1 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.737443  6937 tablet_replica.cc:333] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.738060  6937 raft_consensus.cc:2243] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.739137  6937 raft_consensus.cc:2272] T f6b3f42115d64c26b2b412d669ed6010 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.741482  6937 tablet_replica.cc:333] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.742030  6937 raft_consensus.cc:2243] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.743744  6937 raft_consensus.cc:2272] T b8fabe9526434f818c106b17c57b3b85 P d9cd2999aa314b39997b423ec1ae509e [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.745774  6937 tablet_replica.cc:333] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.746309  6937 raft_consensus.cc:2243] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.746965  6937 raft_consensus.cc:2272] T f4539f806feb4e37900025e737458841 P d9cd2999aa314b39997b423ec1ae509e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.749238  6937 tablet_replica.cc:333] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.749763  6937 raft_consensus.cc:2243] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.751726  6937 raft_consensus.cc:2272] T 4489591b74b1469d81269da167e98f71 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.753628  6937 tablet_replica.cc:333] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.754164  6937 raft_consensus.cc:2243] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.754966  6937 raft_consensus.cc:2272] T a4992d168cb240009f8bfc8df07668e0 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.757321  6937 tablet_replica.cc:333] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.757860  6937 raft_consensus.cc:2243] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.759176  6937 raft_consensus.cc:2272] T 1c85566831fd4f4283bc20a68e5ff448 P d9cd2999aa314b39997b423ec1ae509e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.761904  6937 tablet_replica.cc:333] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.762553  6937 raft_consensus.cc:2243] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.764768  6937 raft_consensus.cc:2272] T bc4cbdf31b174089be94b5f87a22f629 P d9cd2999aa314b39997b423ec1ae509e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.767691  6937 tablet_replica.cc:333] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.768254  6937 raft_consensus.cc:2243] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.769596  6937 raft_consensus.cc:2272] T 9fbc44c6e33e41698020173171d4ff21 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.772133  6937 tablet_replica.cc:333] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.772789  6937 raft_consensus.cc:2243] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:10.773587  6937 raft_consensus.cc:2272] T 5124ddc96f984ec2ae0ee97be6732a90 P d9cd2999aa314b39997b423ec1ae509e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:10.776600  6937 tablet_replica.cc:333] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e: stopping tablet replica
I20260709 13:16:10.777050  6937 raft_consensus.cc:2243] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:16:10.778604  6937 raft_consensus.cc:2272] T f265c49b04e34e9fa83eed8ecdb90607 P d9cd2999aa314b39997b423ec1ae509e [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:10.786188  8696 consensus_peers.cc:597] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.800241  8696 consensus_peers.cc:597] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.831476  8693 consensus_peers.cc:597] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.850455  8696 consensus_peers.cc:597] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.851737  8693 consensus_peers.cc:597] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:10.862165  8693 consensus_peers.cc:597] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:10.870682  8696 consensus_peers.cc:597] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.910862  8696 consensus_peers.cc:597] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:10.930212  8233 debug-util.cc:398] Leaking SignalData structure 0x7b0800380c20 after lost signal to thread 6940
W20260709 13:16:10.931979  8233 debug-util.cc:398] Leaking SignalData structure 0x7b0800353ea0 after lost signal to thread 7995
W20260709 13:16:10.933679  8233 debug-util.cc:398] Leaking SignalData structure 0x7b080017f300 after lost signal to thread 8236
W20260709 13:16:10.935499  8233 debug-util.cc:398] Leaking SignalData structure 0x7b08001b6f40 after lost signal to thread 8754
W20260709 13:16:10.936296  8693 consensus_peers.cc:597] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:10.960943  8696 consensus_peers.cc:597] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:11.040642  8693 consensus_peers.cc:597] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:11.041184  8693 consensus_peers.cc:597] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:11.275920 15781 raft_consensus.cc:493] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:11.276628 15781 raft_consensus.cc:515] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:11.279475 15781 leader_election.cc:290] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:16:11.284487  8693 consensus_peers.cc:597] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:11.284899  8693 leader_election.cc:336] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:11.285524  8693 leader_election.cc:336] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:11.285892  8693 leader_election.cc:304] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:11.286701 15781 raft_consensus.cc:2749] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:11.290287  8693 consensus_peers.cc:597] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:11.306768  8693 consensus_peers.cc:597] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:11.318923  8693 consensus_peers.cc:597] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:11.329139  8693 consensus_peers.cc:597] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 -> Peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Couldn't send request to peer 8a968cada1e4446ebb2a5e167176e649. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:11.382153 15781 raft_consensus.cc:493] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:11.382769 15781 raft_consensus.cc:515] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:11.384631 15781 leader_election.cc:290] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:16:11.389528  8693 leader_election.cc:336] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:11.390060  8693 leader_election.cc:336] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:11.390405  8693 leader_election.cc:304] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:11.391072 15781 raft_consensus.cc:2749] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:11.545465 15781 raft_consensus.cc:493] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:11.546145 15781 raft_consensus.cc:515] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } attrs { replace: true } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:11.549117 15781 leader_election.cc:290] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:16:11.553349  8696 leader_election.cc:336] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
W20260709 13:16:11.567132  8693 leader_election.cc:336] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:11.567886  8693 leader_election.cc:304] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:11.568609 15781 raft_consensus.cc:2749] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:11.569676  8693 leader_election.cc:336] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:11.615289 15781 raft_consensus.cc:493] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:16:11.615995 15781 raft_consensus.cc:515] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:11.618209 15781 leader_election.cc:290] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:16:11.623207  8696 leader_election.cc:336] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
W20260709 13:16:11.624281  8693 leader_election.cc:336] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:11.624708  8693 leader_election.cc:304] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:11.625425 15781 raft_consensus.cc:2749] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:11.654383 15781 raft_consensus.cc:493] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:16:11.655019 15781 raft_consensus.cc:515] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:11.657189 15781 leader_election.cc:290] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:16:11.663084  8693 leader_election.cc:336] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:11.663543  8696 leader_election.cc:336] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
I20260709 13:16:11.664620  8696 leader_election.cc:304] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:11.665388 15781 raft_consensus.cc:2749] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:11.711164 15783 raft_consensus.cc:493] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:16:11.711819 15783 raft_consensus.cc:515] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:11.714699 15783 leader_election.cc:290] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:16:11.721323  8693 leader_election.cc:336] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:11.721969  8693 leader_election.cc:336] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:11.722404  8693 leader_election.cc:304] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:11.723217 15783 raft_consensus.cc:2749] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:16:11.726768  8696 leader_election.cc:336] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
I20260709 13:16:11.787670 15781 raft_consensus.cc:493] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:16:11.788254 15781 raft_consensus.cc:515] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:11.790365 15781 leader_election.cc:290] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:16:11.793292  8693 leader_election.cc:336] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:11.796114  8696 leader_election.cc:336] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
I20260709 13:16:11.796504  8696 leader_election.cc:304] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:11.797164 15781 raft_consensus.cc:2749] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:11.811623 15781 raft_consensus.cc:493] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:11.812228 15781 raft_consensus.cc:515] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:11.814316 15781 leader_election.cc:290] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:16:11.818444  8693 leader_election.cc:336] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:11.819161  8693 leader_election.cc:336] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:11.819561  8693 leader_election.cc:304] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:11.820286 15781 raft_consensus.cc:2749] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:11.857379 15781 raft_consensus.cc:493] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:16:11.858093 15781 raft_consensus.cc:515] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:11.860440 15781 leader_election.cc:290] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:11.866304 15781 raft_consensus.cc:493] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
W20260709 13:16:11.867085  8696 leader_election.cc:336] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
I20260709 13:16:11.866878 15781 raft_consensus.cc:515] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
W20260709 13:16:11.870442  8693 leader_election.cc:336] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:11.870898  8693 leader_election.cc:304] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:11.870997 15781 leader_election.cc:290] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:11.871620 15783 raft_consensus.cc:2749] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:16:11.876753  8696 leader_election.cc:336] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
W20260709 13:16:11.877231  8693 leader_election.cc:336] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:11.877636  8693 leader_election.cc:304] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:11.879235 15783 raft_consensus.cc:2749] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:11.887763 15781 raft_consensus.cc:493] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:11.888362 15781 raft_consensus.cc:515] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:11.891165 15781 leader_election.cc:290] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:16:11.896178  6937 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 8233)
W20260709 13:16:11.897392  8693 leader_election.cc:336] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:11.898875  8693 leader_election.cc:336] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:11.899287  8693 leader_election.cc:304] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:11.899955 15781 raft_consensus.cc:2749] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:11.963599 15781 raft_consensus.cc:493] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:16:11.964234 15781 raft_consensus.cc:515] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:11.966668 15781 leader_election.cc:290] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
I20260709 13:16:11.967299 15783 raft_consensus.cc:493] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:16:11.968021 15783 raft_consensus.cc:515] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:11.970026 15783 leader_election.cc:290] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:16:11.972018  8693 leader_election.cc:336] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:11.975102  8693 leader_election.cc:336] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:11.976737  8696 leader_election.cc:336] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
I20260709 13:16:11.977231  8696 leader_election.cc:304] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:11.978024 15781 raft_consensus.cc:2749] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:16:11.980005  8696 leader_election.cc:336] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
I20260709 13:16:11.981164  8696 leader_election.cc:304] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:11.981814 15781 raft_consensus.cc:2749] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:12.013224 15781 raft_consensus.cc:493] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:16:12.013888 15781 raft_consensus.cc:515] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:12.016062 15781 leader_election.cc:290] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:16:12.019843  8693 leader_election.cc:336] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:12.022243  8696 leader_election.cc:336] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
I20260709 13:16:12.022677  8696 leader_election.cc:304] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:12.023326 15781 raft_consensus.cc:2749] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:12.060561 15781 raft_consensus.cc:493] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:12.061163 15781 raft_consensus.cc:515] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:12.063237 15781 leader_election.cc:290] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:16:12.068895  8693 leader_election.cc:336] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:12.070547  8693 leader_election.cc:336] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:12.070914  8693 leader_election.cc:304] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:12.073168 15781 raft_consensus.cc:2749] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:12.077632 15781 raft_consensus.cc:493] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:16:12.078147 15781 raft_consensus.cc:515] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:12.080325 15781 leader_election.cc:290] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:16:12.084545  8693 leader_election.cc:336] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:12.086131  8696 leader_election.cc:336] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
I20260709 13:16:12.086571  8696 leader_election.cc:304] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:12.087277 15781 raft_consensus.cc:2749] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:12.100023 15781 raft_consensus.cc:493] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:16:12.100514 15781 raft_consensus.cc:515] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:12.102638 15781 leader_election.cc:290] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:16:12.108220  8696 leader_election.cc:336] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
W20260709 13:16:12.112171  8693 leader_election.cc:336] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:12.112738  8693 leader_election.cc:304] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:12.113655 15781 raft_consensus.cc:2749] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:12.116639  8693 consensus_peers.cc:597] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:16:12.154078 15781 raft_consensus.cc:493] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:16:12.154685 15781 raft_consensus.cc:515] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:12.157298 15781 leader_election.cc:290] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:16:12.160401  8696 leader_election.cc:336] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
W20260709 13:16:12.162297  8693 leader_election.cc:336] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:12.162709  8693 leader_election.cc:304] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:12.163341 15781 raft_consensus.cc:2749] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:12.180299 15781 raft_consensus.cc:493] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:16:12.180789 15781 raft_consensus.cc:515] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:12.182862 15781 leader_election.cc:290] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:16:12.186833  8693 leader_election.cc:336] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:12.188011  8696 leader_election.cc:336] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
I20260709 13:16:12.188391  8696 leader_election.cc:304] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:12.189047 15781 raft_consensus.cc:2749] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:16:12.200958  8693 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111) [suppressed 578 similar messages]
W20260709 13:16:12.203290  8693 consensus_peers.cc:597] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:16:12.206735 15781 raft_consensus.cc:493] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader d9cd2999aa314b39997b423ec1ae509e)
I20260709 13:16:12.207299 15781 raft_consensus.cc:515] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } attrs { replace: true } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:12.212195 15781 leader_election.cc:290] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:16:12.214170  8696 leader_election.cc:336] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
W20260709 13:16:12.216810  8693 leader_election.cc:336] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:12.217212  8693 leader_election.cc:304] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:12.217860 15781 raft_consensus.cc:2749] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:16:12.218245  8693 leader_election.cc:336] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
W20260709 13:16:12.229290  8693 consensus_peers.cc:597] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:16:12.371492 15783 raft_consensus.cc:493] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
I20260709 13:16:12.372156 15783 raft_consensus.cc:515] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:12.374277 15783 leader_election.cc:290] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
W20260709 13:16:12.379794  8693 leader_election.cc:336] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:12.380455  8693 leader_election.cc:336] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:12.380810  8693 leader_election.cc:304] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:12.381458 15783 raft_consensus.cc:2749] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:12.472908 15781 raft_consensus.cc:493] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.473497 15781 raft_consensus.cc:515] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "d9cd2999aa314b39997b423ec1ae509e" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 35745 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:12.475495 15781 leader_election.cc:290] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807), d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745)
W20260709 13:16:12.478871  8693 leader_election.cc:336] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:12.479801 15781 raft_consensus.cc:493] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a968cada1e4446ebb2a5e167176e649)
W20260709 13:16:12.480517  8696 leader_election.cc:336] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111)
I20260709 13:16:12.480355 15781 raft_consensus.cc:515] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "84d775d1767e4cc4aa7b9c627d8f0fb8" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45919 } } peers { permanent_uuid: "8a968cada1e4446ebb2a5e167176e649" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 45807 } } peers { permanent_uuid: "965e59f272f143449b18f65db4e61d98" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 44573 } attrs { promote: false } }
I20260709 13:16:12.481047  8696 leader_election.cc:304] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 8a968cada1e4446ebb2a5e167176e649, d9cd2999aa314b39997b423ec1ae509e
I20260709 13:16:12.482411 15781 leader_election.cc:290] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919), 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807)
I20260709 13:16:12.483414 15781 raft_consensus.cc:2749] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:16:12.488011  8693 leader_election.cc:336] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
W20260709 13:16:12.488708  8693 leader_election.cc:336] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8a968cada1e4446ebb2a5e167176e649 (127.6.198.66:45807): Network error: Client connection negotiation failed: client connection to 127.6.198.66:45807: connect: Connection refused (error 111)
I20260709 13:16:12.489068  8693 leader_election.cc:304] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 965e59f272f143449b18f65db4e61d98; no voters: 84d775d1767e4cc4aa7b9c627d8f0fb8, 8a968cada1e4446ebb2a5e167176e649
I20260709 13:16:12.489696 15781 raft_consensus.cc:2749] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:16:12.593787  8693 consensus_peers.cc:597] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:16:12.626891  8693 consensus_peers.cc:597] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:16:12.774811  6937 tablet_server.cc:196] TabletServer@127.6.198.67:0 shutdown complete.
W20260709 13:16:12.784093  8693 consensus_peers.cc:597] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:16:12.790906  8693 consensus_peers.cc:597] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:16:12.820905  8693 consensus_peers.cc:597] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:16:12.823640  8693 consensus_peers.cc:597] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:16:12.855054  8696 consensus_peers.cc:597] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:12.864756  8696 consensus_peers.cc:597] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:12.866114  8693 consensus_peers.cc:597] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:16:12.869975  8693 consensus_peers.cc:597] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 -> Peer 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): Couldn't send request to peer 84d775d1767e4cc4aa7b9c627d8f0fb8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:16:12.897614  8696 consensus_peers.cc:597] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:12.910212  6937 tablet_server.cc:179] TabletServer@127.6.198.68:0 shutting down...
W20260709 13:16:12.960601  8696 consensus_peers.cc:597] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:12.962790  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:12.963793  6937 tablet_replica.cc:333] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:12.965111  6937 raft_consensus.cc:2243] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:16:12.967092  6937 raft_consensus.cc:2272] T 1002e40536a34729929022702d03444c P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:12.970443  6937 tablet_replica.cc:333] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:12.971304  6937 raft_consensus.cc:2243] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:12.972098  6937 raft_consensus.cc:2272] T 0e4fa00ecc35497e823a1329fe771042 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:12.975411  6937 tablet_replica.cc:333] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:12.976346  6937 raft_consensus.cc:2243] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:12.977365  6937 raft_consensus.cc:2272] T ec8da6c869684688957d5323fb023bc5 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:12.980540  6937 tablet_replica.cc:333] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:12.981279  6937 raft_consensus.cc:2243] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:16:12.982477  6937 pending_rounds.cc:70] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98: Trying to abort 1 pending ops.
I20260709 13:16:12.982717  6937 pending_rounds.cc:77] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98: Aborting op as it isn't in flight: id { term: 19 index: 14 } timestamp: 7305637766546026496 op_type: NO_OP noop_request { }
I20260709 13:16:12.983140  6937 raft_consensus.cc:2889] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:16:12.983511  6937 raft_consensus.cc:2272] T 17c830696ad4418b88d85b6a97e41851 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:12.986573  6937 tablet_replica.cc:333] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:12.987440  6937 raft_consensus.cc:2243] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:12.988523  6937 raft_consensus.cc:2272] T 8e397a319b11487fbbacec77723b5739 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:12.991729  6937 tablet_replica.cc:333] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:12.992635  6937 raft_consensus.cc:2243] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:12.993326  6937 raft_consensus.cc:2272] T b8b8bb0a8e3545218708596da648262f P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:12.996166  6937 tablet_replica.cc:333] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:12.996958  6937 raft_consensus.cc:2243] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:12.997622  6937 raft_consensus.cc:2272] T a478a4313e6149378cae4b6f833e3c2a P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.000527  6937 tablet_replica.cc:333] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.001296  6937 raft_consensus.cc:2243] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.001873  6937 raft_consensus.cc:2272] T b578d88c1c1f4ca58b7291fed4909f09 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.004686  6937 tablet_replica.cc:333] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.005465  6937 raft_consensus.cc:2243] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.006683  6937 raft_consensus.cc:2272] T f7fb8453eda243f9856f489af50e67a2 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.009577  6937 tablet_replica.cc:333] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.010322  6937 raft_consensus.cc:2243] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.011427  6937 raft_consensus.cc:2272] T b186be823643475982e269b73d886a42 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.014211  6937 tablet_replica.cc:333] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.014957  6937 raft_consensus.cc:2243] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.015962  6937 raft_consensus.cc:2272] T 5124ddc96f984ec2ae0ee97be6732a90 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.018663  6937 tablet_replica.cc:333] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.019222  6937 raft_consensus.cc:2243] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.020061  6937 raft_consensus.cc:2272] T 7c4e1687a1db419c8863789bb37c260a P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.022331  6937 tablet_replica.cc:333] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.022998  6937 raft_consensus.cc:2243] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.023622  6937 raft_consensus.cc:2272] T ade1b48685164d64a62772d186ac959f P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.025806  6937 tablet_replica.cc:333] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.026458  6937 raft_consensus.cc:2243] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.027124  6937 raft_consensus.cc:2272] T 5d0e2cd74dac4358aba60edf5382befd P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.029668  6937 tablet_replica.cc:333] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.030931  6937 raft_consensus.cc:2243] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.032047  6937 raft_consensus.cc:2272] T 1c85566831fd4f4283bc20a68e5ff448 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.034796  6937 tablet_replica.cc:333] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.035348  6937 raft_consensus.cc:2243] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.036201  6937 raft_consensus.cc:2272] T 3ca98b6f039a45fb80c6e78ff2a0ae32 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.038264  6937 tablet_replica.cc:333] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.038883  6937 raft_consensus.cc:2243] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.039558  6937 raft_consensus.cc:2272] T 07fe988cafa34bcaa51af457b7248ccc P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.041908  6937 tablet_replica.cc:333] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.042711  6937 raft_consensus.cc:2243] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.043274  6937 raft_consensus.cc:2272] T 4489591b74b1469d81269da167e98f71 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.045858  6937 tablet_replica.cc:333] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.046379  6937 raft_consensus.cc:2243] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.047345  6937 pending_rounds.cc:70] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Trying to abort 1 pending ops.
I20260709 13:16:13.047564  6937 pending_rounds.cc:77] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98: Aborting op as it isn't in flight: id { term: 21 index: 15 } timestamp: 7305637765602279424 op_type: NO_OP noop_request { }
I20260709 13:16:13.047825  6937 raft_consensus.cc:2889] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 21 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:16:13.048127  6937 raft_consensus.cc:2272] T f4539f806feb4e37900025e737458841 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.049943  6937 tablet_replica.cc:333] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.050499  6937 raft_consensus.cc:2243] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.051051  6937 raft_consensus.cc:2272] T 7a9aaa6e2a1c42a0a0db69d6406a3762 P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.053063  6937 tablet_replica.cc:333] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.053574  6937 raft_consensus.cc:2243] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.054059  6937 raft_consensus.cc:2272] T 542d60a75b684950a46a64c1d64b80a5 P 965e59f272f143449b18f65db4e61d98 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.056120  6937 tablet_replica.cc:333] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.056608  6937 raft_consensus.cc:2243] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.057711  6937 raft_consensus.cc:2272] T 8dd7949df539418cbc00001111885883 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.060153  6937 tablet_replica.cc:333] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.060743  6937 raft_consensus.cc:2243] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.061185  6937 raft_consensus.cc:2272] T ed7f54a64a14435691eeef2ece07e5dc P 965e59f272f143449b18f65db4e61d98 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.063117  6937 tablet_replica.cc:333] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.063664  6937 raft_consensus.cc:2243] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.064088  6937 raft_consensus.cc:2272] T f265c49b04e34e9fa83eed8ecdb90607 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.066449  6937 tablet_replica.cc:333] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.066969  6937 raft_consensus.cc:2243] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.067761  6937 raft_consensus.cc:2272] T b8fabe9526434f818c106b17c57b3b85 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.069648  6937 tablet_replica.cc:333] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.070180  6937 raft_consensus.cc:2243] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.070706  6937 raft_consensus.cc:2272] T d013f25ff51143beb65d31cc33e5a248 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.072638  6937 tablet_replica.cc:333] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.073266  6937 raft_consensus.cc:2243] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.074108  6937 raft_consensus.cc:2272] T 51726a5d5cad44fd8e9b6803da998cb2 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.076052  6937 tablet_replica.cc:333] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.076574  6937 raft_consensus.cc:2243] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.077122  6937 raft_consensus.cc:2272] T bc4cbdf31b174089be94b5f87a22f629 P 965e59f272f143449b18f65db4e61d98 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.079020  6937 tablet_replica.cc:333] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.079634  6937 raft_consensus.cc:2243] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.080154  6937 raft_consensus.cc:2272] T a003b4584d3747a381522d67299aaed2 P 965e59f272f143449b18f65db4e61d98 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.082136  6937 tablet_replica.cc:333] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.082631  6937 raft_consensus.cc:2243] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.083079  6937 raft_consensus.cc:2272] T 1dfd4edf9eab4bcc8b8aad94587349c3 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.084735  6937 tablet_replica.cc:333] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.085253  6937 raft_consensus.cc:2243] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.086210  6937 raft_consensus.cc:2272] T dad90abaa8b04508a5b086ff94502e4e P 965e59f272f143449b18f65db4e61d98 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.089263  6937 tablet_replica.cc:333] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.089993  6937 raft_consensus.cc:2243] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:16:13.090003  8696 consensus_peers.cc:597] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 -> Peer d9cd2999aa314b39997b423ec1ae509e (127.6.198.67:35745): Couldn't send request to peer d9cd2999aa314b39997b423ec1ae509e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.67:35745: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:13.090934  6937 raft_consensus.cc:2272] T 1f229a5eb4514abc9183538dd97073d2 P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.092880  6937 tablet_replica.cc:333] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.093430  6937 raft_consensus.cc:2243] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.094609  6937 raft_consensus.cc:2272] T d4f33fbb7b794128a4668f6fda160937 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.097074  6937 tablet_replica.cc:333] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.097555  6937 raft_consensus.cc:2243] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.098304  6937 raft_consensus.cc:2272] T bd0294d309cb4f8696e693285a7b5b04 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.100067  6937 tablet_replica.cc:333] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.100584  6937 raft_consensus.cc:2243] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.101239  6937 raft_consensus.cc:2272] T 97bd47fbf9aa4d1bb8ec05287f69e0b0 P 965e59f272f143449b18f65db4e61d98 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.103436  6937 tablet_replica.cc:333] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.103937  6937 raft_consensus.cc:2243] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.104743  6937 raft_consensus.cc:2272] T bf628848d7464a21aa8f76f7eb56a408 P 965e59f272f143449b18f65db4e61d98 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.106720  6937 tablet_replica.cc:333] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.107199  6937 raft_consensus.cc:2243] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.108124  6937 raft_consensus.cc:2272] T c97c3defa48b473aaf762b3a27c0cba0 P 965e59f272f143449b18f65db4e61d98 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.110321  6937 tablet_replica.cc:333] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.110962  6937 raft_consensus.cc:2243] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.112099  6937 pending_rounds.cc:70] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: Trying to abort 1 pending ops.
I20260709 13:16:13.112375  6937 pending_rounds.cc:77] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98: Aborting op as it isn't in flight: id { term: 20 index: 17 } timestamp: 7305637765585457152 op_type: NO_OP noop_request { }
I20260709 13:16:13.112655  6937 raft_consensus.cc:2889] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 20 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:16:13.112967  6937 raft_consensus.cc:2272] T 6771ba07746943f6a958ca7be1990296 P 965e59f272f143449b18f65db4e61d98 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.114902  6937 tablet_replica.cc:333] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.115383  6937 raft_consensus.cc:2243] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.115859  6937 raft_consensus.cc:2272] T a3e2b86010604477955f19a56daf6d04 P 965e59f272f143449b18f65db4e61d98 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.117674  6937 tablet_replica.cc:333] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.118140  6937 raft_consensus.cc:2243] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.118575  6937 raft_consensus.cc:2272] T b9bd4708595b40438af51aba853a0f94 P 965e59f272f143449b18f65db4e61d98 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.120432  6937 tablet_replica.cc:333] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.120887  6937 raft_consensus.cc:2243] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:16:13.121724  6937 raft_consensus.cc:2272] T 481500a64cdb4a7fb94658807295dcbe P 965e59f272f143449b18f65db4e61d98 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.123603  6937 tablet_replica.cc:333] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98: stopping tablet replica
I20260709 13:16:13.124054  6937 raft_consensus.cc:2243] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:13.124593  6937 raft_consensus.cc:2272] T 9fbc44c6e33e41698020173171d4ff21 P 965e59f272f143449b18f65db4e61d98 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:13.586241  6937 tablet_server.cc:196] TabletServer@127.6.198.68:0 shutdown complete.
I20260709 13:16:13.669394  6937 master.cc:562] Master@127.6.198.126:38543 shutting down...
W20260709 13:16:13.718475  8009 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:16:13.718747  8009 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:16:13.718966  8009 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:16:17.429358  7992 debug-util.cc:398] Leaking SignalData structure 0x7b08003160e0 after lost signal to thread 6940
W20260709 13:16:17.429811  7992 debug-util.cc:398] Leaking SignalData structure 0x7b08002bd940 after lost signal to thread 7995
W20260709 13:16:17.887306  7945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111) [suppressed 78 similar messages]
W20260709 13:16:17.890280  7945 catalog_manager.cc:4755] TS 84d775d1767e4cc4aa7b9c627d8f0fb8 (127.6.198.65:45919): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet dad90abaa8b04508a5b086ff94502e4e: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45919: connect: Connection refused (error 111)
I20260709 13:16:18.219870  6937 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:18.221664  6937 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:18.222033  6937 tablet_replica.cc:333] T 00000000000000000000000000000000 P e3b771f7515a4bb7beb66bc429c57201: stopping tablet replica
I20260709 13:16:18.357857  6937 master.cc:584] Master@127.6.198.126:38543 shutdown complete.
I20260709 13:16:18.393935  6937 test_util.cc:182] -----------------------------------------------
I20260709 13:16:18.394151  6937 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602718985547-6937-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (235608 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:16:18.397063  6937 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.198.126:41247
I20260709 13:16:18.398092  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:18.403745 15806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:18.404176 15807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:18.404615 15809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:18.405134  6937 server_base.cc:1061] running on GCE node
I20260709 13:16:18.408351  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:18.408540  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:18.408658  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602978408641 us; error 0 us; skew 500 ppm
I20260709 13:16:18.409142  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:18.411504  6937 webserver.cc:533] Webserver started at http://127.6.198.126:36335/ using document root <none> and password file <none>
I20260709 13:16:18.411965  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:18.412127  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:18.412331  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:18.413439  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/master-0-root/instance:
uuid: "6fd52bd8ed974e4ca4e0ee6a84db3ebc"
format_stamp: "Formatted at 2026-07-09 13:16:18 on dist-test-slave-tqzw"
I20260709 13:16:18.417557  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:16:18.420717 15814 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.421424  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:16:18.421701  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/master-0-root
uuid: "6fd52bd8ed974e4ca4e0ee6a84db3ebc"
format_stamp: "Formatted at 2026-07-09 13:16:18 on dist-test-slave-tqzw"
I20260709 13:16:18.421988  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:18.453992  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:18.455331  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:18.493261  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.126:41247
I20260709 13:16:18.493358 15865 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.126:41247 every 8 connection(s)
I20260709 13:16:18.497215 15866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:18.507579 15866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc: Bootstrap starting.
I20260709 13:16:18.512392 15866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:18.516305 15866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc: No bootstrap required, opened a new log
I20260709 13:16:18.518298 15866 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fd52bd8ed974e4ca4e0ee6a84db3ebc" member_type: VOTER }
I20260709 13:16:18.518704 15866 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:18.518884 15866 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fd52bd8ed974e4ca4e0ee6a84db3ebc, State: Initialized, Role: FOLLOWER
I20260709 13:16:18.519366 15866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fd52bd8ed974e4ca4e0ee6a84db3ebc" member_type: VOTER }
I20260709 13:16:18.519737 15866 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:16:18.519919 15866 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:16:18.520124 15866 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:18.524248 15866 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fd52bd8ed974e4ca4e0ee6a84db3ebc" member_type: VOTER }
I20260709 13:16:18.524669 15866 leader_election.cc:304] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fd52bd8ed974e4ca4e0ee6a84db3ebc; no voters: 
I20260709 13:16:18.525651 15866 leader_election.cc:290] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:16:18.526039 15869 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:18.527359 15869 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [term 1 LEADER]: Becoming Leader. State: Replica: 6fd52bd8ed974e4ca4e0ee6a84db3ebc, State: Running, Role: LEADER
I20260709 13:16:18.527968 15869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fd52bd8ed974e4ca4e0ee6a84db3ebc" member_type: VOTER }
I20260709 13:16:18.528585 15866 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:16:18.532656 15870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fd52bd8ed974e4ca4e0ee6a84db3ebc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fd52bd8ed974e4ca4e0ee6a84db3ebc" member_type: VOTER } }
I20260709 13:16:18.533404 15870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:18.533394 15871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fd52bd8ed974e4ca4e0ee6a84db3ebc. Latest consensus state: current_term: 1 leader_uuid: "6fd52bd8ed974e4ca4e0ee6a84db3ebc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fd52bd8ed974e4ca4e0ee6a84db3ebc" member_type: VOTER } }
I20260709 13:16:18.534428 15871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:18.536471 15876 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:16:18.542050 15876 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:16:18.547653  6937 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:16:18.551708 15876 catalog_manager.cc:1383] Generated new cluster ID: 317c163d379a49168c4a7160e6300d14
I20260709 13:16:18.552037 15876 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:16:18.572039 15876 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:16:18.573316 15876 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:16:18.591013 15876 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc: Generated new TSK 0
I20260709 13:16:18.591584 15876 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:16:18.614332  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:18.620260 15887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:18.621661 15888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:18.624697 15890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:18.625659  6937 server_base.cc:1061] running on GCE node
I20260709 13:16:18.627610  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:18.627785  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:18.627979  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602978627945 us; error 0 us; skew 500 ppm
I20260709 13:16:18.628485  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:18.630841  6937 webserver.cc:533] Webserver started at http://127.6.198.65:43789/ using document root <none> and password file <none>
I20260709 13:16:18.631311  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:18.631490  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:18.631778  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:18.632881  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/instance:
uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
format_stamp: "Formatted at 2026-07-09 13:16:18 on dist-test-slave-tqzw"
I20260709 13:16:18.637158  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:16:18.640340 15895 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.641060  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:16:18.641350  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-0-root
uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
format_stamp: "Formatted at 2026-07-09 13:16:18 on dist-test-slave-tqzw"
I20260709 13:16:18.641620  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:18.659816  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:18.660936  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:18.662405  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:18.664604  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:18.664829  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.665059  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:18.665239  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.704918  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.65:45841
I20260709 13:16:18.705029 15957 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.65:45841 every 8 connection(s)
I20260709 13:16:18.709867  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:18.717348 15962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:18.721122 15963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:18.725028 15965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:18.725454  6937 server_base.cc:1061] running on GCE node
I20260709 13:16:18.725672 15958 heartbeater.cc:344] Connected to a master server at 127.6.198.126:41247
I20260709 13:16:18.726385 15958 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:18.727582 15958 heartbeater.cc:507] Master 127.6.198.126:41247 requested a full tablet report, sending...
I20260709 13:16:18.728487  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:18.728792  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:18.729007  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602978728986 us; error 0 us; skew 500 ppm
I20260709 13:16:18.729626  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:18.729771 15831 ts_manager.cc:194] Registered new tserver with Master: bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:18.732287  6937 webserver.cc:533] Webserver started at http://127.6.198.66:44023/ using document root <none> and password file <none>
I20260709 13:16:18.732425 15831 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36906
I20260709 13:16:18.732990  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:18.733250  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:18.733570  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:18.735149  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/instance:
uuid: "903b3435239241eab01ea54ffb015ee3"
format_stamp: "Formatted at 2026-07-09 13:16:18 on dist-test-slave-tqzw"
I20260709 13:16:18.739320  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:16:18.742451 15970 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.743201  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:16:18.743564  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-1-root
uuid: "903b3435239241eab01ea54ffb015ee3"
format_stamp: "Formatted at 2026-07-09 13:16:18 on dist-test-slave-tqzw"
I20260709 13:16:18.743882  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:18.776252  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:18.777384  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:18.778738  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:18.780830  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:18.781157  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.781443  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:18.781600  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.821650  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.66:32879
I20260709 13:16:18.821753 16032 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.66:32879 every 8 connection(s)
I20260709 13:16:18.826599  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:18.834030 16036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:18.834883 16037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:18.838163  6937 server_base.cc:1061] running on GCE node
W20260709 13:16:18.838093 16039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:18.839495 16033 heartbeater.cc:344] Connected to a master server at 127.6.198.126:41247
I20260709 13:16:18.839744  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:16:18.839870 16033 heartbeater.cc:461] Registering TS with master...
W20260709 13:16:18.840010  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:18.840261  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602978840241 us; error 0 us; skew 500 ppm
I20260709 13:16:18.840762 16033 heartbeater.cc:507] Master 127.6.198.126:41247 requested a full tablet report, sending...
I20260709 13:16:18.840835  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:18.843046 15831 ts_manager.cc:194] Registered new tserver with Master: 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:18.844403 15831 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36910
I20260709 13:16:18.845886  6937 webserver.cc:533] Webserver started at http://127.6.198.67:43733/ using document root <none> and password file <none>
I20260709 13:16:18.846469  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:18.846676  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:18.846947  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:18.848107  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/instance:
uuid: "d4d6e13a8f4a473098351ada96fd7dab"
format_stamp: "Formatted at 2026-07-09 13:16:18 on dist-test-slave-tqzw"
I20260709 13:16:18.852325  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 13:16:18.855471 16044 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.856187  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:16:18.856479  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-2-root
uuid: "d4d6e13a8f4a473098351ada96fd7dab"
format_stamp: "Formatted at 2026-07-09 13:16:18 on dist-test-slave-tqzw"
I20260709 13:16:18.856760  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:18.882869  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:18.884093  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:18.885553  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:18.887761  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:18.887990  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.888235  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:18.888409  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.929000  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.67:32795
I20260709 13:16:18.929098 16106 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.67:32795 every 8 connection(s)
I20260709 13:16:18.933836  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:18.940110 16110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:18.941700 16111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:18.942795  6937 server_base.cc:1061] running on GCE node
W20260709 13:16:18.944896 16113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:18.946027  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:18.946267  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:18.946520  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602978946499 us; error 0 us; skew 500 ppm
I20260709 13:16:18.946707 16107 heartbeater.cc:344] Connected to a master server at 127.6.198.126:41247
I20260709 13:16:18.947121 16107 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:18.947180  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:18.947945 16107 heartbeater.cc:507] Master 127.6.198.126:41247 requested a full tablet report, sending...
I20260709 13:16:18.949671  6937 webserver.cc:533] Webserver started at http://127.6.198.68:38021/ using document root <none> and password file <none>
I20260709 13:16:18.950183  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:18.950129 15831 ts_manager.cc:194] Registered new tserver with Master: d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:18.950603  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:18.950934  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:18.951774 15831 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36924
I20260709 13:16:18.951999  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-3-root/instance:
uuid: "c95f0816c41142699d80a2e326e17ab8"
format_stamp: "Formatted at 2026-07-09 13:16:18 on dist-test-slave-tqzw"
I20260709 13:16:18.956470  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:16:18.959715 16118 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.960446  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:16:18.960741  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-3-root
uuid: "c95f0816c41142699d80a2e326e17ab8"
format_stamp: "Formatted at 2026-07-09 13:16:18 on dist-test-slave-tqzw"
I20260709 13:16:18.961007  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:18.975430  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:18.976611  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:18.978044  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:18.980175  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:18.980369  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:18.980626  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:18.980784  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:19.020831  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.68:36069
I20260709 13:16:19.020921 16180 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.68:36069 every 8 connection(s)
I20260709 13:16:19.034298 16181 heartbeater.cc:344] Connected to a master server at 127.6.198.126:41247
I20260709 13:16:19.034673 16181 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:19.035374 16181 heartbeater.cc:507] Master 127.6.198.126:41247 requested a full tablet report, sending...
I20260709 13:16:19.037137 15831 ts_manager.cc:194] Registered new tserver with Master: c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:19.037710  6937 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.013470456s
I20260709 13:16:19.038576 15831 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36928
I20260709 13:16:19.062227 15831 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36938:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:16:19.308218 15998 tablet_service.cc:1511] Processing CreateTablet for tablet ce23588dc52d46338893551f44214a28 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:16:19.309756 15998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce23588dc52d46338893551f44214a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.311409 15997 tablet_service.cc:1511] Processing CreateTablet for tablet f7473dd845e447a0b5a453e01dbb4b6b (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:16:19.312352 16146 tablet_service.cc:1511] Processing CreateTablet for tablet f7473dd845e447a0b5a453e01dbb4b6b (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:16:19.312816 15997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7473dd845e447a0b5a453e01dbb4b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.313774 16146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7473dd845e447a0b5a453e01dbb4b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.313529 16144 tablet_service.cc:1511] Processing CreateTablet for tablet ab6469b36c7444ba906df4c1f29f6f0e (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:16:19.314860 16144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab6469b36c7444ba906df4c1f29f6f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.315555 15996 tablet_service.cc:1511] Processing CreateTablet for tablet ab6469b36c7444ba906df4c1f29f6f0e (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:16:19.317123 15996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab6469b36c7444ba906df4c1f29f6f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.319578 16143 tablet_service.cc:1511] Processing CreateTablet for tablet c895cd2e2a144ff692fcb615c095e82a (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:16:19.320971 16143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c895cd2e2a144ff692fcb615c095e82a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.321162 15995 tablet_service.cc:1511] Processing CreateTablet for tablet 6aa884c0889444ea8aa23dbe7bcfe9de (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:16:19.322520 15995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6aa884c0889444ea8aa23dbe7bcfe9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.315601 16145 tablet_service.cc:1511] Processing CreateTablet for tablet 1708b4e2385e408abc600d9b2f6d42a1 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:16:19.327243 16145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1708b4e2385e408abc600d9b2f6d42a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.328701 15994 tablet_service.cc:1511] Processing CreateTablet for tablet c895cd2e2a144ff692fcb615c095e82a (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:16:19.330027 15994 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c895cd2e2a144ff692fcb615c095e82a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.347935 16142 tablet_service.cc:1511] Processing CreateTablet for tablet 7cca9877952a413a9112ea61481bc582 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:16:19.349309 16142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cca9877952a413a9112ea61481bc582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.359377 15993 tablet_service.cc:1511] Processing CreateTablet for tablet ef6f890d60f14faead1e5986904c458a (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:16:19.361088 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef6f890d60f14faead1e5986904c458a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.361270 16072 tablet_service.cc:1511] Processing CreateTablet for tablet ce23588dc52d46338893551f44214a28 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:16:19.362546 16072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce23588dc52d46338893551f44214a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.365522 16141 tablet_service.cc:1511] Processing CreateTablet for tablet ef6f890d60f14faead1e5986904c458a (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:16:19.366847 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef6f890d60f14faead1e5986904c458a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.369864 15992 tablet_service.cc:1511] Processing CreateTablet for tablet e08b5d04b39149c187ddb882b1b5d2de (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:16:19.371274 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e08b5d04b39149c187ddb882b1b5d2de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.371928 16071 tablet_service.cc:1511] Processing CreateTablet for tablet 1708b4e2385e408abc600d9b2f6d42a1 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:16:19.373626 16071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1708b4e2385e408abc600d9b2f6d42a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.380203 16070 tablet_service.cc:1511] Processing CreateTablet for tablet ab6469b36c7444ba906df4c1f29f6f0e (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:16:19.382037 16070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab6469b36c7444ba906df4c1f29f6f0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.389532 16144 tablet_service.cc:1511] Processing CreateTablet for tablet c9df103185604f988d3abe94ae731141 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:16:19.390901 16144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9df103185604f988d3abe94ae731141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.392081 15993 tablet_service.cc:1511] Processing CreateTablet for tablet c9df103185604f988d3abe94ae731141 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:16:19.406574 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9df103185604f988d3abe94ae731141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.420358 16069 tablet_service.cc:1511] Processing CreateTablet for tablet 6aa884c0889444ea8aa23dbe7bcfe9de (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:16:19.422263 16069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6aa884c0889444ea8aa23dbe7bcfe9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.432094 16068 tablet_service.cc:1511] Processing CreateTablet for tablet 7cca9877952a413a9112ea61481bc582 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:16:19.434006 16068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cca9877952a413a9112ea61481bc582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.440614 16067 tablet_service.cc:1511] Processing CreateTablet for tablet ef6f890d60f14faead1e5986904c458a (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:16:19.442510 16067 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef6f890d60f14faead1e5986904c458a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.462920 16202 tablet_bootstrap.cc:492] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:19.468796 16066 tablet_service.cc:1511] Processing CreateTablet for tablet e08b5d04b39149c187ddb882b1b5d2de (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:16:19.470319 16201 tablet_bootstrap.cc:492] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:19.470840 16066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e08b5d04b39149c187ddb882b1b5d2de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.474980 16065 tablet_service.cc:1511] Processing CreateTablet for tablet 2c2dba979b4b4cfdb4eb1159d36ae158 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:16:19.476696 16065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c2dba979b4b4cfdb4eb1159d36ae158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.485975 16202 tablet_bootstrap.cc:654] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.492476 15923 tablet_service.cc:1511] Processing CreateTablet for tablet ce23588dc52d46338893551f44214a28 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:16:19.501784 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce23588dc52d46338893551f44214a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.492794 15922 tablet_service.cc:1511] Processing CreateTablet for tablet f7473dd845e447a0b5a453e01dbb4b6b (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:16:19.507265 15922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7473dd845e447a0b5a453e01dbb4b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.494403 15918 tablet_service.cc:1511] Processing CreateTablet for tablet 7cca9877952a413a9112ea61481bc582 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:16:19.515255 15918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cca9877952a413a9112ea61481bc582. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.507265 16145 tablet_service.cc:1511] Processing CreateTablet for tablet 2c2dba979b4b4cfdb4eb1159d36ae158 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:16:19.517405 16145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c2dba979b4b4cfdb4eb1159d36ae158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.494552 15921 tablet_service.cc:1511] Processing CreateTablet for tablet 1708b4e2385e408abc600d9b2f6d42a1 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:16:19.495101 15920 tablet_service.cc:1511] Processing CreateTablet for tablet 6aa884c0889444ea8aa23dbe7bcfe9de (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:16:19.508775 16072 tablet_service.cc:1511] Processing CreateTablet for tablet 7454f1cf21c0400c8a3a8290d5d148dd (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:16:19.496289 15919 tablet_service.cc:1511] Processing CreateTablet for tablet c895cd2e2a144ff692fcb615c095e82a (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:16:19.497493 15917 tablet_service.cc:1511] Processing CreateTablet for tablet e08b5d04b39149c187ddb882b1b5d2de (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:16:19.519316 15921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1708b4e2385e408abc600d9b2f6d42a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.520164 15917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e08b5d04b39149c187ddb882b1b5d2de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.520800 15920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6aa884c0889444ea8aa23dbe7bcfe9de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.499117 15916 tablet_service.cc:1511] Processing CreateTablet for tablet c9df103185604f988d3abe94ae731141 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:16:19.499501 15915 tablet_service.cc:1511] Processing CreateTablet for tablet 2c2dba979b4b4cfdb4eb1159d36ae158 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:16:19.511297 15995 tablet_service.cc:1511] Processing CreateTablet for tablet 7454f1cf21c0400c8a3a8290d5d148dd (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:16:19.521842 15916 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9df103185604f988d3abe94ae731141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.519313 16072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7454f1cf21c0400c8a3a8290d5d148dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.522151 15995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7454f1cf21c0400c8a3a8290d5d148dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.522547 15919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c895cd2e2a144ff692fcb615c095e82a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.525347 15915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c2dba979b4b4cfdb4eb1159d36ae158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.536024 16143 tablet_service.cc:1511] Processing CreateTablet for tablet 7454f1cf21c0400c8a3a8290d5d148dd (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:16:19.537310 16143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7454f1cf21c0400c8a3a8290d5d148dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.537155 15914 tablet_service.cc:1511] Processing CreateTablet for tablet cb4ef2e4f6be44efbf83fb70838b4ae7 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:16:19.538434 15914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb4ef2e4f6be44efbf83fb70838b4ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.539628 16203 tablet_bootstrap.cc:492] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
W20260709 13:16:19.543596 15879 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:19.543391 15993 tablet_service.cc:1511] Processing CreateTablet for tablet cb4ef2e4f6be44efbf83fb70838b4ae7 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:16:19.544734 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb4ef2e4f6be44efbf83fb70838b4ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.559365 16201 tablet_bootstrap.cc:654] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.564400 16071 tablet_service.cc:1511] Processing CreateTablet for tablet cb4ef2e4f6be44efbf83fb70838b4ae7 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:16:19.565596 16203 tablet_bootstrap.cc:654] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.582264 16144 tablet_service.cc:1511] Processing CreateTablet for tablet 718e00ba6fa34542a71fc6d5758e92a1 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:16:19.589087 16144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 718e00ba6fa34542a71fc6d5758e92a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.593362 15915 tablet_service.cc:1511] Processing CreateTablet for tablet 718e00ba6fa34542a71fc6d5758e92a1 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:16:19.595384 15915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 718e00ba6fa34542a71fc6d5758e92a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.565732 16071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb4ef2e4f6be44efbf83fb70838b4ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.607066 16202 tablet_bootstrap.cc:492] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:19.607625 16202 ts_tablet_manager.cc:1403] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.145s	user 0.015s	sys 0.000s
I20260709 13:16:19.608958 16070 tablet_service.cc:1511] Processing CreateTablet for tablet 3a39139d4f2b48d09ba9eb320dfcf265 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:16:19.611569 16070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a39139d4f2b48d09ba9eb320dfcf265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.610246 15995 tablet_service.cc:1511] Processing CreateTablet for tablet 718e00ba6fa34542a71fc6d5758e92a1 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:16:19.616715 15995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 718e00ba6fa34542a71fc6d5758e92a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.610047 16143 tablet_service.cc:1511] Processing CreateTablet for tablet 3a39139d4f2b48d09ba9eb320dfcf265 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:16:19.618228 16143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a39139d4f2b48d09ba9eb320dfcf265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.621371 15923 tablet_service.cc:1511] Processing CreateTablet for tablet 3a39139d4f2b48d09ba9eb320dfcf265 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:16:19.623749 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a39139d4f2b48d09ba9eb320dfcf265. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.624207 16204 tablet_bootstrap.cc:492] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:19.635373 16202 raft_consensus.cc:359] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:19.635782 15996 tablet_service.cc:1511] Processing CreateTablet for tablet 68a3e44fcd5b438486e1ec0241ef1e8d (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:16:19.636466 16201 tablet_bootstrap.cc:492] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:19.636945 16201 ts_tablet_manager.cc:1403] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.167s	user 0.011s	sys 0.006s
I20260709 13:16:19.637163 15996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a3e44fcd5b438486e1ec0241ef1e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.636178 16202 raft_consensus.cc:385] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.639155 16202 raft_consensus.cc:740] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.639990 16204 tablet_bootstrap.cc:654] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.640141 16202 consensus_queue.cc:260] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:19.640478 16201 raft_consensus.cc:359] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:19.641294 16201 raft_consensus.cc:385] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.641716 16201 raft_consensus.cc:740] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.642515 16201 consensus_queue.cc:260] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:19.645538 16181 heartbeater.cc:499] Master 127.6.198.126:41247 was elected leader, sending a full tablet report...
I20260709 13:16:19.647787 16145 tablet_service.cc:1511] Processing CreateTablet for tablet 68a3e44fcd5b438486e1ec0241ef1e8d (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:16:19.649091 16145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a3e44fcd5b438486e1ec0241ef1e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.655814 16071 tablet_service.cc:1511] Processing CreateTablet for tablet 68a3e44fcd5b438486e1ec0241ef1e8d (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:16:19.657166 16071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a3e44fcd5b438486e1ec0241ef1e8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.664469 16202 ts_tablet_manager.cc:1434] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.004s
I20260709 13:16:19.665488 16202 tablet_bootstrap.cc:492] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:19.667100 16203 tablet_bootstrap.cc:492] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:19.667730 16203 ts_tablet_manager.cc:1403] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.128s	user 0.016s	sys 0.000s
I20260709 13:16:19.684762 16201 ts_tablet_manager.cc:1434] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.048s	user 0.015s	sys 0.025s
I20260709 13:16:19.685670 16033 heartbeater.cc:499] Master 127.6.198.126:41247 was elected leader, sending a full tablet report...
I20260709 13:16:19.685992 16201 tablet_bootstrap.cc:492] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:19.692973 16201 tablet_bootstrap.cc:654] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.684463 16203 raft_consensus.cc:359] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:19.695783 16204 tablet_bootstrap.cc:492] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:19.698589 15915 tablet_service.cc:1511] Processing CreateTablet for tablet c6319ed0d71e4b4e9608ad673766fb5e (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:16:19.702535 16203 raft_consensus.cc:385] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.703729 16203 raft_consensus.cc:740] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.703930 16204 ts_tablet_manager.cc:1403] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.080s	user 0.006s	sys 0.010s
I20260709 13:16:19.704656 16203 consensus_queue.cc:260] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:19.705137 16202 tablet_bootstrap.cc:654] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.706921 15915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6319ed0d71e4b4e9608ad673766fb5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.707623 16204 raft_consensus.cc:359] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:19.709287 16204 raft_consensus.cc:385] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.713145 16204 raft_consensus.cc:740] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.715191 16204 consensus_queue.cc:260] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:19.717118 15996 tablet_service.cc:1511] Processing CreateTablet for tablet c6319ed0d71e4b4e9608ad673766fb5e (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:16:19.717942 16203 ts_tablet_manager.cc:1434] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.006s
I20260709 13:16:19.718572 15996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6319ed0d71e4b4e9608ad673766fb5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.718801 16107 heartbeater.cc:499] Master 127.6.198.126:41247 was elected leader, sending a full tablet report...
I20260709 13:16:19.724588 16203 tablet_bootstrap.cc:492] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:19.727850 16145 tablet_service.cc:1511] Processing CreateTablet for tablet 4633df69636042288f27156e6084dce9 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:16:19.730756 16145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4633df69636042288f27156e6084dce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.735366 15958 heartbeater.cc:499] Master 127.6.198.126:41247 was elected leader, sending a full tablet report...
I20260709 13:16:19.742902 16071 tablet_service.cc:1511] Processing CreateTablet for tablet c6319ed0d71e4b4e9608ad673766fb5e (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:16:19.744378 16071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6319ed0d71e4b4e9608ad673766fb5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.754730 15923 tablet_service.cc:1511] Processing CreateTablet for tablet 4633df69636042288f27156e6084dce9 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:16:19.756891 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4633df69636042288f27156e6084dce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.765858 16202 tablet_bootstrap.cc:492] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:19.766696 16202 ts_tablet_manager.cc:1403] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.101s	user 0.020s	sys 0.011s
I20260709 13:16:19.767021 16070 tablet_service.cc:1511] Processing CreateTablet for tablet 034918f94907492ca9a964c053edd50e (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:16:19.768321 16070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034918f94907492ca9a964c053edd50e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.768031 16143 tablet_service.cc:1511] Processing CreateTablet for tablet 034918f94907492ca9a964c053edd50e (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:16:19.769261 16143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034918f94907492ca9a964c053edd50e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.770953 16202 raft_consensus.cc:359] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:19.772040 16202 raft_consensus.cc:385] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.772504 16202 raft_consensus.cc:740] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.771302 15995 tablet_service.cc:1511] Processing CreateTablet for tablet 4633df69636042288f27156e6084dce9 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:16:19.773551 16202 consensus_queue.cc:260] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
W20260709 13:16:19.775364 16182 tablet.cc:2406] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:16:19.776132 16202 ts_tablet_manager.cc:1434] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:16:19.783380 16202 tablet_bootstrap.cc:492] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:19.774407 15995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4633df69636042288f27156e6084dce9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.790838 16204 ts_tablet_manager.cc:1434] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.086s	user 0.022s	sys 0.026s
I20260709 13:16:19.791862 16204 tablet_bootstrap.cc:492] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:19.791649 15993 tablet_service.cc:1511] Processing CreateTablet for tablet 07ae89d92f6942d196dcc8d146b8f459 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:16:19.792194 16201 tablet_bootstrap.cc:492] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:19.792783 16201 ts_tablet_manager.cc:1403] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.107s	user 0.022s	sys 0.003s
I20260709 13:16:19.792929 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07ae89d92f6942d196dcc8d146b8f459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.795905 16201 raft_consensus.cc:359] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:19.798130 16204 tablet_bootstrap.cc:654] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.802057 16203 tablet_bootstrap.cc:654] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.805529 16201 raft_consensus.cc:385] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.807606 16201 raft_consensus.cc:740] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.809269 16201 consensus_queue.cc:260] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:19.810740 16208 raft_consensus.cc:493] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.811302 16201 ts_tablet_manager.cc:1434] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:16:19.811261 16208 raft_consensus.cc:515] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:19.812476 16201 tablet_bootstrap.cc:492] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:19.810168 16071 tablet_service.cc:1511] Processing CreateTablet for tablet 07ae89d92f6942d196dcc8d146b8f459 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:16:19.817220 16071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07ae89d92f6942d196dcc8d146b8f459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.817704 16144 tablet_service.cc:1511] Processing CreateTablet for tablet 07ae89d92f6942d196dcc8d146b8f459 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:16:19.820538 16144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07ae89d92f6942d196dcc8d146b8f459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.823776 16208 leader_election.cc:290] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:19.837235 16211 raft_consensus.cc:493] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.837760 16211 raft_consensus.cc:515] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:19.818102 15915 tablet_service.cc:1511] Processing CreateTablet for tablet 034918f94907492ca9a964c053edd50e (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:16:19.838562 15921 tablet_service.cc:1511] Processing CreateTablet for tablet e38e5c73946b4d14825079217a21a21a (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:16:19.839205 15915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034918f94907492ca9a964c053edd50e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.841379 16202 tablet_bootstrap.cc:654] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.842878 15997 tablet_service.cc:1511] Processing CreateTablet for tablet e38e5c73946b4d14825079217a21a21a (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:16:19.844172 15997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38e5c73946b4d14825079217a21a21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.844599 16070 tablet_service.cc:1511] Processing CreateTablet for tablet e38e5c73946b4d14825079217a21a21a (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:16:19.845793 16070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38e5c73946b4d14825079217a21a21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.848168 16211 leader_election.cc:290] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:19.856015 16208 raft_consensus.cc:493] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.867015 16208 raft_consensus.cc:515] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:19.869391 15992 tablet_service.cc:1511] Processing CreateTablet for tablet f46c65175b29436c8f0704ffa98d4eb7 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:16:19.871412 16208 leader_election.cc:290] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:19.880354 16145 tablet_service.cc:1511] Processing CreateTablet for tablet f46c65175b29436c8f0704ffa98d4eb7 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:16:19.881644 16145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f46c65175b29436c8f0704ffa98d4eb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.881878 15917 tablet_service.cc:1511] Processing CreateTablet for tablet f46c65175b29436c8f0704ffa98d4eb7 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:16:19.859962 15921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38e5c73946b4d14825079217a21a21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.872920 16201 tablet_bootstrap.cc:654] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:19.901393 16072 tablet_service.cc:1511] Processing CreateTablet for tablet bf30209cb5544714bc183c72359c9656 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:16:19.903365 16072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf30209cb5544714bc183c72359c9656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.907478 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f46c65175b29436c8f0704ffa98d4eb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.911695 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c895cd2e2a144ff692fcb615c095e82a" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:19.913192 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7473dd845e447a0b5a453e01dbb4b6b" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
W20260709 13:16:19.914741 16119 leader_election.cc:343] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:19.915370 16119 leader_election.cc:343] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:19.932821 15917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f46c65175b29436c8f0704ffa98d4eb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.935726 16213 raft_consensus.cc:493] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.936244 16213 raft_consensus.cc:515] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:19.937435 16145 tablet_service.cc:1511] Processing CreateTablet for tablet bf30209cb5544714bc183c72359c9656 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:16:19.938678 16145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf30209cb5544714bc183c72359c9656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.938843 16213 leader_election.cc:290] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:19.953749 15914 tablet_service.cc:1511] Processing CreateTablet for tablet bf30209cb5544714bc183c72359c9656 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:16:19.954561 16203 tablet_bootstrap.cc:492] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:19.955068 15914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf30209cb5544714bc183c72359c9656. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:19.959685 16201 tablet_bootstrap.cc:492] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:19.960175 16201 ts_tablet_manager.cc:1403] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.148s	user 0.010s	sys 0.007s
I20260709 13:16:19.955111 16203 ts_tablet_manager.cc:1403] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.231s	user 0.014s	sys 0.015s
I20260709 13:16:19.964521 16201 raft_consensus.cc:359] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:19.965242 16201 raft_consensus.cc:385] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.965544 16201 raft_consensus.cc:740] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.966398 16201 consensus_queue.cc:260] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:19.966279 16203 raft_consensus.cc:359] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:19.971391 16203 raft_consensus.cc:385] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.971745 16203 raft_consensus.cc:740] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.972381 16201 ts_tablet_manager.cc:1434] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:16:19.972638 16203 consensus_queue.cc:260] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:19.973443 16201 tablet_bootstrap.cc:492] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:19.980752 16202 tablet_bootstrap.cc:492] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:19.981266 16202 ts_tablet_manager.cc:1403] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.198s	user 0.030s	sys 0.037s
I20260709 13:16:19.991950 16204 tablet_bootstrap.cc:492] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:19.994524 16204 ts_tablet_manager.cc:1403] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.203s	user 0.010s	sys 0.005s
I20260709 13:16:19.996340 16202 raft_consensus.cc:359] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:19.997087 16202 raft_consensus.cc:385] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:19.997387 16202 raft_consensus.cc:740] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:19.997602 16204 raft_consensus.cc:359] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:20.001395 16071 tablet_service.cc:1511] Processing CreateTablet for tablet eebbb30a54a04a4aa567c6e16581ad1a (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:16:20.002619 16201 tablet_bootstrap.cc:654] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.004971 16204 raft_consensus.cc:385] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.008432 16204 raft_consensus.cc:740] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.009176 16204 consensus_queue.cc:260] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:20.011206 16204 ts_tablet_manager.cc:1434] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:16:20.012156 16204 tablet_bootstrap.cc:492] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:20.019336 16071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eebbb30a54a04a4aa567c6e16581ad1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.022111 15993 tablet_service.cc:1511] Processing CreateTablet for tablet eebbb30a54a04a4aa567c6e16581ad1a (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:16:20.023564 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eebbb30a54a04a4aa567c6e16581ad1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.014773 16143 tablet_service.cc:1511] Processing CreateTablet for tablet eebbb30a54a04a4aa567c6e16581ad1a (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:16:20.039855 16143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eebbb30a54a04a4aa567c6e16581ad1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.041678 15997 tablet_service.cc:1511] Processing CreateTablet for tablet 5922376de62747aabde22ad134c86f36 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:16:20.042943 15997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5922376de62747aabde22ad134c86f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.043536 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c895cd2e2a144ff692fcb615c095e82a" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:20.044266 16008 raft_consensus.cc:2468] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 0.
I20260709 13:16:20.045794 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7473dd845e447a0b5a453e01dbb4b6b" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:20.061698 16121 leader_election.cc:304] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
W20260709 13:16:20.063069 16121 leader_election.cc:343] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.063553 16121 leader_election.cc:304] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:20.064641 16208 raft_consensus.cc:2804] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.065048 16208 raft_consensus.cc:493] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.065399 16208 raft_consensus.cc:3060] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.067950 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce23588dc52d46338893551f44214a28" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:20.068881 16008 raft_consensus.cc:2468] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:20.070322 16047 leader_election.cc:304] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:20.071252 16211 raft_consensus.cc:2804] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.071614 16211 raft_consensus.cc:493] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.071936 16211 raft_consensus.cc:3060] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.077845 16070 tablet_service.cc:1511] Processing CreateTablet for tablet 5922376de62747aabde22ad134c86f36 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:16:20.080008 16070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5922376de62747aabde22ad134c86f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.085202 15914 tablet_service.cc:1511] Processing CreateTablet for tablet 8f15b348b66b4bffae9c7f9eed8207e2 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:16:20.082850 16202 consensus_queue.cc:260] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:20.086472 15914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f15b348b66b4bffae9c7f9eed8207e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.088543 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce23588dc52d46338893551f44214a28" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:20.095997 15932 raft_consensus.cc:2468] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:20.097246 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce23588dc52d46338893551f44214a28" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:20.097872 16008 raft_consensus.cc:2468] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:20.098129 15915 tablet_service.cc:1511] Processing CreateTablet for tablet 5922376de62747aabde22ad134c86f36 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:16:20.091687 16211 raft_consensus.cc:515] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:20.092471 16204 tablet_bootstrap.cc:654] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.099308 15898 leader_election.cc:304] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:20.103360 16213 raft_consensus.cc:2804] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.103709 16213 raft_consensus.cc:493] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.104077 16213 raft_consensus.cc:3060] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.095278 16203 ts_tablet_manager.cc:1434] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.133s	user 0.007s	sys 0.000s
I20260709 13:16:20.117056 16208 raft_consensus.cc:515] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.117444 16144 tablet_service.cc:1511] Processing CreateTablet for tablet cc17a40e973a46a18d21e55a0a939417 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:16:20.118693 16144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc17a40e973a46a18d21e55a0a939417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.113678 16072 tablet_service.cc:1511] Processing CreateTablet for tablet cc17a40e973a46a18d21e55a0a939417 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:16:20.123224 16203 tablet_bootstrap.cc:492] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:20.126931 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce23588dc52d46338893551f44214a28" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:20.127234 15917 tablet_service.cc:1511] Processing CreateTablet for tablet cc17a40e973a46a18d21e55a0a939417 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
W20260709 13:16:20.127753 16121 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:32879, user_credentials={real_user=slave}} blocked reactor thread for 64725.6us
I20260709 13:16:20.110996 15915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5922376de62747aabde22ad134c86f36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.131846 16231 raft_consensus.cc:2749] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.133881 16141 tablet_service.cc:1511] Processing CreateTablet for tablet aaa175423efe491da6747a37df086375 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:16:20.135164 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaa175423efe491da6747a37df086375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.134866 16066 tablet_service.cc:1511] Processing CreateTablet for tablet aaa175423efe491da6747a37df086375 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:16:20.135808 16082 raft_consensus.cc:2393] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bdeb23ab98c34a269f5b2e71b03c5e4e in current term 1: Already voted for candidate d4d6e13a8f4a473098351ada96fd7dab in this term.
I20260709 13:16:20.149849 16066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaa175423efe491da6747a37df086375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.150378 16202 ts_tablet_manager.cc:1434] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.169s	user 0.008s	sys 0.007s
I20260709 13:16:20.150779 16072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc17a40e973a46a18d21e55a0a939417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.151551 16202 tablet_bootstrap.cc:492] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:20.147506 15917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc17a40e973a46a18d21e55a0a939417. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.153550 16145 tablet_service.cc:1511] Processing CreateTablet for tablet 8f15b348b66b4bffae9c7f9eed8207e2 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:16:20.160243 15992 tablet_service.cc:1511] Processing CreateTablet for tablet aaa175423efe491da6747a37df086375 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:16:20.161952 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aaa175423efe491da6747a37df086375. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.167913 16145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f15b348b66b4bffae9c7f9eed8207e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.168967 16202 tablet_bootstrap.cc:654] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.171481 15996 tablet_service.cc:1511] Processing CreateTablet for tablet 8f15b348b66b4bffae9c7f9eed8207e2 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:16:20.172478 16201 tablet_bootstrap.cc:492] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:20.173062 16201 ts_tablet_manager.cc:1403] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.200s	user 0.016s	sys 0.002s
I20260709 13:16:20.174460 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c895cd2e2a144ff692fcb615c095e82a" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:20.177627 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c895cd2e2a144ff692fcb615c095e82a" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
W20260709 13:16:20.180797 16119 leader_election.cc:343] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.178194 16008 raft_consensus.cc:3060] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.173782 16208 leader_election.cc:290] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:20.192900 16201 raft_consensus.cc:359] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.193500 16201 raft_consensus.cc:385] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.193768 16201 raft_consensus.cc:740] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.196252 15923 tablet_service.cc:1511] Processing CreateTablet for tablet f5565bfb243b4eeab5cbb5549d639378 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:16:20.197595 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5565bfb243b4eeab5cbb5549d639378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.198559 16071 tablet_service.cc:1511] Processing CreateTablet for tablet f5565bfb243b4eeab5cbb5549d639378 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:16:20.199786 16071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5565bfb243b4eeab5cbb5549d639378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.202090 15997 tablet_service.cc:1511] Processing CreateTablet for tablet f5565bfb243b4eeab5cbb5549d639378 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:16:20.202693 16213 raft_consensus.cc:515] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:20.204056 16142 tablet_service.cc:1511] Processing CreateTablet for tablet 31217a26afbc45258ea518a1623dbb01 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:16:20.205278 15921 tablet_service.cc:1511] Processing CreateTablet for tablet 31217a26afbc45258ea518a1623dbb01 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:16:20.205946 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce23588dc52d46338893551f44214a28" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:20.206609 15921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31217a26afbc45258ea518a1623dbb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.206745 16082 raft_consensus.cc:2393] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bdeb23ab98c34a269f5b2e71b03c5e4e in current term 1: Already voted for candidate d4d6e13a8f4a473098351ada96fd7dab in this term.
I20260709 13:16:20.207064 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce23588dc52d46338893551f44214a28" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:20.207614 16068 tablet_service.cc:1511] Processing CreateTablet for tablet d6e83ffadf17413a95b361a2cf2b1b7f (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:16:20.207929 16007 raft_consensus.cc:3060] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.205237 16142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31217a26afbc45258ea518a1623dbb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.210510 16201 consensus_queue.cc:260] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.212256 15995 tablet_service.cc:1511] Processing CreateTablet for tablet 31217a26afbc45258ea518a1623dbb01 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:16:20.212847 16201 ts_tablet_manager.cc:1434] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.000s
I20260709 13:16:20.213836 16201 tablet_bootstrap.cc:492] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:20.215515 16211 leader_election.cc:290] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:20.215845 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce23588dc52d46338893551f44214a28" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:20.216158 15996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f15b348b66b4bffae9c7f9eed8207e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.216809 15997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5565bfb243b4eeab5cbb5549d639378. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.216750 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce23588dc52d46338893551f44214a28" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:20.217662 15932 raft_consensus.cc:2393] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d4d6e13a8f4a473098351ada96fd7dab in current term 1: Already voted for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in this term.
I20260709 13:16:20.218817 16045 leader_election.cc:304] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:20.219389 16007 raft_consensus.cc:2468] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:20.219607 16008 raft_consensus.cc:2468] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 1.
I20260709 13:16:20.220806 16121 leader_election.cc:304] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:20.222035 15898 leader_election.cc:304] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:20.222817 15995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31217a26afbc45258ea518a1623dbb01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.223423 16211 raft_consensus.cc:2749] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.208097 16213 leader_election.cc:290] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:20.234910 16213 raft_consensus.cc:2804] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:20.232039 16230 raft_consensus.cc:2804] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:20.235375 16213 raft_consensus.cc:697] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:20.236006 16230 raft_consensus.cc:697] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
I20260709 13:16:20.236078 16068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6e83ffadf17413a95b361a2cf2b1b7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.236953 16230 consensus_queue.cc:237] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.240911 16202 tablet_bootstrap.cc:492] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:20.241637 16202 ts_tablet_manager.cc:1403] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.090s	user 0.003s	sys 0.021s
I20260709 13:16:20.190220 16203 tablet_bootstrap.cc:654] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.245383 16202 raft_consensus.cc:359] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:20.246271 16202 raft_consensus.cc:385] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.246662 16202 raft_consensus.cc:740] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.261432 16143 tablet_service.cc:1511] Processing CreateTablet for tablet d6e83ffadf17413a95b361a2cf2b1b7f (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:16:20.262760 16202 consensus_queue.cc:260] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:20.265095 16202 ts_tablet_manager.cc:1434] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.003s
I20260709 13:16:20.266085 16202 tablet_bootstrap.cc:492] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:20.265671 15828 catalog_manager.cc:5697] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 1 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:20.275952 16209 raft_consensus.cc:493] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.276425 16209 raft_consensus.cc:515] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:20.279250 16209 leader_election.cc:290] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:20.236279 16213 consensus_queue.cc:237] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:20.309478 15998 tablet_service.cc:1511] Processing CreateTablet for tablet bd2e0e0244294ec18481adf638d57ff0 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:16:20.310248 16203 tablet_bootstrap.cc:492] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:20.310781 15998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2e0e0244294ec18481adf638d57ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.310824 16203 ts_tablet_manager.cc:1403] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.188s	user 0.007s	sys 0.005s
W20260709 13:16:20.304392 15898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:32879, user_credentials={real_user=slave}} blocked reactor thread for 82461.4us
I20260709 13:16:20.314510 16065 tablet_service.cc:1511] Processing CreateTablet for tablet bd2e0e0244294ec18481adf638d57ff0 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:16:20.315973 16065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2e0e0244294ec18481adf638d57ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.322425 16143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6e83ffadf17413a95b361a2cf2b1b7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.333405 15993 tablet_service.cc:1511] Processing CreateTablet for tablet 20762eaf5e494270a0e86a0b95c1437f (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:16:20.334759 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20762eaf5e494270a0e86a0b95c1437f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.341611 16201 tablet_bootstrap.cc:654] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.342396 16211 raft_consensus.cc:493] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.342901 16211 raft_consensus.cc:515] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:20.345029 16211 leader_election.cc:290] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:20.345719 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6aa884c0889444ea8aa23dbe7bcfe9de" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:20.346604 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6aa884c0889444ea8aa23dbe7bcfe9de" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:20.347172 16045 leader_election.cc:343] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.314996 16203 raft_consensus.cc:359] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:20.347877 16203 raft_consensus.cc:385] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:16:20.347967 16047 leader_election.cc:343] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.348253 16203 raft_consensus.cc:740] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.348475 16047 leader_election.cc:304] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:20.349555 16211 raft_consensus.cc:2749] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.349232 16203 consensus_queue.cc:260] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:20.352079 16202 tablet_bootstrap.cc:654] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.352203 16203 ts_tablet_manager.cc:1434] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.000s
I20260709 13:16:20.370623 16230 raft_consensus.cc:493] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.371142 16230 raft_consensus.cc:515] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:20.371236 16068 tablet_service.cc:1511] Processing CreateTablet for tablet 20762eaf5e494270a0e86a0b95c1437f (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:16:20.373483 16230 leader_election.cc:290] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:20.385774 16068 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20762eaf5e494270a0e86a0b95c1437f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.386897 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6469b36c7444ba906df4c1f29f6f0e" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:20.388438 16121 leader_election.cc:343] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.394699 16065 tablet_service.cc:1511] Processing CreateTablet for tablet 8b94b09aec6746f69853f14b417f18e3 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:16:20.395977 16065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b94b09aec6746f69853f14b417f18e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.402503 15831 catalog_manager.cc:5697] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:20.407006 16144 tablet_service.cc:1511] Processing CreateTablet for tablet 62ac7645b9954621a685e4fc9ddde0dd (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:16:20.407770 16141 tablet_service.cc:1511] Processing CreateTablet for tablet 8b94b09aec6746f69853f14b417f18e3 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:16:20.408434 15917 tablet_service.cc:1511] Processing CreateTablet for tablet 62ac7645b9954621a685e4fc9ddde0dd (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:16:20.408934 16237 raft_consensus.cc:493] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.409690 15917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62ac7645b9954621a685e4fc9ddde0dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.409395 15923 tablet_service.cc:1511] Processing CreateTablet for tablet 20762eaf5e494270a0e86a0b95c1437f (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:16:20.410684 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20762eaf5e494270a0e86a0b95c1437f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.409457 16237 raft_consensus.cc:515] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:20.410995 16143 tablet_service.cc:1511] Processing CreateTablet for tablet bd2e0e0244294ec18481adf638d57ff0 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:16:20.414011 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2dba979b4b4cfdb4eb1159d36ae158" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:20.408270 16144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62ac7645b9954621a685e4fc9ddde0dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.417797 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b94b09aec6746f69853f14b417f18e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.415864 16237 leader_election.cc:290] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
W20260709 13:16:20.422317 15897 leader_election.cc:343] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.423202 16143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd2e0e0244294ec18481adf638d57ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.424345 16201 tablet_bootstrap.cc:492] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:20.424888 16201 ts_tablet_manager.cc:1403] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.211s	user 0.013s	sys 0.014s
I20260709 13:16:20.418413 16203 tablet_bootstrap.cc:492] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:20.428056 16201 raft_consensus.cc:359] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:20.428820 16201 raft_consensus.cc:385] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.429118 16201 raft_consensus.cc:740] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.429864 16201 consensus_queue.cc:260] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:20.432057 16204 tablet_bootstrap.cc:492] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:20.432976 16204 ts_tablet_manager.cc:1403] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.421s	user 0.022s	sys 0.020s
I20260709 13:16:20.408973 15921 tablet_service.cc:1511] Processing CreateTablet for tablet d6e83ffadf17413a95b361a2cf2b1b7f (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:16:20.435592 15921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6e83ffadf17413a95b361a2cf2b1b7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.436154 16204 raft_consensus.cc:359] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:20.436970 16204 raft_consensus.cc:385] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.437321 16204 raft_consensus.cc:740] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.438197 16204 consensus_queue.cc:260] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:20.445356 15915 tablet_service.cc:1511] Processing CreateTablet for tablet 8b94b09aec6746f69853f14b417f18e3 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:16:20.447252 15915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b94b09aec6746f69853f14b417f18e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.459641 16142 tablet_service.cc:1511] Processing CreateTablet for tablet dadf33b9db824fdcbb63864c9e975c6b (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:16:20.461107 16142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dadf33b9db824fdcbb63864c9e975c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.462952 15914 tablet_service.cc:1511] Processing CreateTablet for tablet 0a574a47fef045e183c154c25260da37 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:16:20.454267 16203 tablet_bootstrap.cc:654] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.470913 16201 ts_tablet_manager.cc:1434] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.001s
I20260709 13:16:20.472018 16201 tablet_bootstrap.cc:492] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:20.472297 15914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a574a47fef045e183c154c25260da37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.480917 16066 tablet_service.cc:1511] Processing CreateTablet for tablet 0a574a47fef045e183c154c25260da37 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:16:20.481879 16145 tablet_service.cc:1511] Processing CreateTablet for tablet 1a21a302bc17473392974aeea5a31d60 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:16:20.483175 16145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a21a302bc17473392974aeea5a31d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.483240 15996 tablet_service.cc:1511] Processing CreateTablet for tablet 0a574a47fef045e183c154c25260da37 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:16:20.482267 16066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a574a47fef045e183c154c25260da37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.486481 16071 tablet_service.cc:1511] Processing CreateTablet for tablet dadf33b9db824fdcbb63864c9e975c6b (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:16:20.498652 15996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a574a47fef045e183c154c25260da37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.499579 16204 ts_tablet_manager.cc:1434] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.000s
I20260709 13:16:20.504829 16204 tablet_bootstrap.cc:492] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:20.511140 16204 tablet_bootstrap.cc:654] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.510864 15992 tablet_service.cc:1511] Processing CreateTablet for tablet 62ac7645b9954621a685e4fc9ddde0dd (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:16:20.512280 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62ac7645b9954621a685e4fc9ddde0dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.514050 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6f890d60f14faead1e5986904c458a" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:20.514756 16082 raft_consensus.cc:2468] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:20.516220 15972 leader_election.cc:304] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:20.517102 16209 raft_consensus.cc:2804] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.517590 16209 raft_consensus.cc:493] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.518008 16209 raft_consensus.cc:3060] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.521209 16071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dadf33b9db824fdcbb63864c9e975c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.458310 15993 tablet_service.cc:1511] Processing CreateTablet for tablet dadf33b9db824fdcbb63864c9e975c6b (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:16:20.522203 15993 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dadf33b9db824fdcbb63864c9e975c6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.527577 16230 raft_consensus.cc:493] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.528115 16230 raft_consensus.cc:515] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:20.530216 16230 leader_election.cc:290] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:20.531651 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cca9877952a413a9112ea61481bc582" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
W20260709 13:16:20.533582 16119 leader_election.cc:343] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.534971 16209 raft_consensus.cc:515] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:20.553043 16202 tablet_bootstrap.cc:492] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:20.553607 16202 ts_tablet_manager.cc:1403] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.288s	user 0.011s	sys 0.023s
I20260709 13:16:20.560374 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6f890d60f14faead1e5986904c458a" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:20.561035 16082 raft_consensus.cc:3060] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.564885 16202 raft_consensus.cc:359] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:20.565692 16202 raft_consensus.cc:385] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.566027 16202 raft_consensus.cc:740] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.566987 16202 consensus_queue.cc:260] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:20.569550 16202 ts_tablet_manager.cc:1434] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 13:16:20.570711 16202 tablet_bootstrap.cc:492] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:20.570775 16082 raft_consensus.cc:2468] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:20.572036 15972 leader_election.cc:304] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:20.579207 16251 raft_consensus.cc:493] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.579722 16251 raft_consensus.cc:515] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.582033 15997 tablet_service.cc:1511] Processing CreateTablet for tablet 1a21a302bc17473392974aeea5a31d60 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:16:20.583433 15997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a21a302bc17473392974aeea5a31d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.588315 16209 leader_election.cc:290] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:20.591682 16237 raft_consensus.cc:493] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.591960 16209 raft_consensus.cc:2804] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:20.592522 16209 raft_consensus.cc:697] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:20.592211 16237 raft_consensus.cc:515] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:20.593760 16202 tablet_bootstrap.cc:654] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.593379 16209 consensus_queue.cc:237] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:20.561043 15923 tablet_service.cc:1511] Processing CreateTablet for tablet 1a21a302bc17473392974aeea5a31d60 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:16:20.596921 16237 leader_election.cc:290] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:20.599996 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6aa884c0889444ea8aa23dbe7bcfe9de" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:20.601173 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6aa884c0889444ea8aa23dbe7bcfe9de" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:20.601760 16082 raft_consensus.cc:2468] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:20.603247 15897 leader_election.cc:304] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
W20260709 13:16:20.604576 15898 leader_election.cc:343] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.605566 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2dba979b4b4cfdb4eb1159d36ae158" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:20.610277 16081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6469b36c7444ba906df4c1f29f6f0e" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:20.610498 16237 raft_consensus.cc:2804] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.610932 16237 raft_consensus.cc:493] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.611253 16237 raft_consensus.cc:3060] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.611601 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cca9877952a413a9112ea61481bc582" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:20.613557 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a21a302bc17473392974aeea5a31d60. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:20.615826 15899 leader_election.cc:343] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.616561 16204 tablet_bootstrap.cc:492] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:20.616451 15899 leader_election.cc:304] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:20.616819 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6f890d60f14faead1e5986904c458a" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:20.617501 16156 raft_consensus.cc:2468] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:20.617964 16251 leader_election.cc:290] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:20.617871 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef6f890d60f14faead1e5986904c458a" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:20.617046 16204 ts_tablet_manager.cc:1403] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.117s	user 0.022s	sys 0.018s
I20260709 13:16:20.618624 16155 raft_consensus.cc:3060] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.618832 16154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7473dd845e447a0b5a453e01dbb4b6b" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:20.619541 16154 raft_consensus.cc:2468] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:20.620321 16213 raft_consensus.cc:2749] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:20.615908 16120 leader_election.cc:343] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.622587 16120 leader_election.cc:304] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:20.621400 16204 raft_consensus.cc:359] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.634791 16204 raft_consensus.cc:385] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.635104 16203 tablet_bootstrap.cc:492] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:20.635129 16204 raft_consensus.cc:740] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.635756 16203 ts_tablet_manager.cc:1403] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.233s	user 0.020s	sys 0.011s
I20260709 13:16:20.636078 16204 consensus_queue.cc:260] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.638159 16204 ts_tablet_manager.cc:1434] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.019s	user 0.002s	sys 0.002s
I20260709 13:16:20.638752 16203 raft_consensus.cc:359] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:20.639190 16204 tablet_bootstrap.cc:492] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:20.639519 16203 raft_consensus.cc:385] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.639832 16203 raft_consensus.cc:740] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.640533 16155 raft_consensus.cc:2468] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:20.640525 16203 consensus_queue.cc:260] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:20.643553 16203 ts_tablet_manager.cc:1434] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:20.644133 16237 raft_consensus.cc:515] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:20.644614 16203 tablet_bootstrap.cc:492] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:20.645498 16201 tablet_bootstrap.cc:654] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.645954 16204 tablet_bootstrap.cc:654] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.646231 16237 leader_election.cc:290] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:20.695094 15974 leader_election.cc:304] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
W20260709 13:16:20.697434 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:36069, user_credentials={real_user=slave}} blocked reactor thread for 70394.6us
I20260709 13:16:20.699472 15917 tablet_service.cc:1511] Processing CreateTablet for tablet ccc0c303e2dd466caf0e684c6ea83284 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:16:20.699816 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6aa884c0889444ea8aa23dbe7bcfe9de" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
W20260709 13:16:20.704511 15898 leader_election.cc:343] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.705004 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6aa884c0889444ea8aa23dbe7bcfe9de" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:20.705928 15917 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc0c303e2dd466caf0e684c6ea83284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.707640 16071 tablet_service.cc:1511] Processing CreateTablet for tablet ccc0c303e2dd466caf0e684c6ea83284 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:16:20.709649 16071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc0c303e2dd466caf0e684c6ea83284. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:20.709812 16120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:32795, user_credentials={real_user=slave}} blocked reactor thread for 93981.5us
W20260709 13:16:20.711110 16120 leader_election.cc:343] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.711735 16120 leader_election.cc:304] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:20.728029 16208 raft_consensus.cc:2749] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.705924 16082 raft_consensus.cc:3060] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.728624 16230 raft_consensus.cc:2749] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.728248 16251 raft_consensus.cc:2804] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.732677 16251 raft_consensus.cc:493] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.733155 16251 raft_consensus.cc:3060] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.734066 16202 tablet_bootstrap.cc:492] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:20.735098 16202 ts_tablet_manager.cc:1403] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.165s	user 0.011s	sys 0.004s
I20260709 13:16:20.740526 16230 consensus_queue.cc:1048] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:16:20.743233 16119 consensus_peers.cc:597] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:20.732868 16065 tablet_service.cc:1511] Processing CreateTablet for tablet de7a15b080c54551b2c51bee7af64600 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:16:20.726087 15992 tablet_service.cc:1511] Processing CreateTablet for tablet de7a15b080c54551b2c51bee7af64600 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:16:20.746532 16065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de7a15b080c54551b2c51bee7af64600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.747732 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de7a15b080c54551b2c51bee7af64600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.748586 16203 tablet_bootstrap.cc:654] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.756503 16204 tablet_bootstrap.cc:492] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:20.759671 16202 raft_consensus.cc:359] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.760687 16202 raft_consensus.cc:385] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.761068 16202 raft_consensus.cc:740] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.771677 16237 raft_consensus.cc:493] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:20.772549 15879 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:20.772431 16237 raft_consensus.cc:515] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.775592 16209 raft_consensus.cc:493] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.776275 16209 raft_consensus.cc:515] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:20.777405 15829 catalog_manager.cc:5697] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:20.782007 16204 ts_tablet_manager.cc:1403] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.143s	user 0.010s	sys 0.009s
W20260709 13:16:20.782526 15818 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.6.198.68:36069, user_credentials={real_user=slave}} blocked reactor thread for 93254.7us
I20260709 13:16:20.782819 16237 leader_election.cc:290] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:20.784117 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7473dd845e447a0b5a453e01dbb4b6b" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:20.787170 16209 leader_election.cc:290] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:20.787853 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7473dd845e447a0b5a453e01dbb4b6b" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:20.788403 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08b5d04b39149c187ddb882b1b5d2de" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:20.788564 15932 raft_consensus.cc:2468] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:20.790611 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08b5d04b39149c187ddb882b1b5d2de" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:20.792502 16204 raft_consensus.cc:359] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.793253 16204 raft_consensus.cc:385] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.793579 16204 raft_consensus.cc:740] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.795547 15914 tablet_service.cc:1511] Processing CreateTablet for tablet 3c53d28c984a4016b8461698b47d3300 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:16:20.797099 15914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c53d28c984a4016b8461698b47d3300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.796988 16066 tablet_service.cc:1511] Processing CreateTablet for tablet 3c53d28c984a4016b8461698b47d3300 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:16:20.798448 16066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c53d28c984a4016b8461698b47d3300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.801520 16204 consensus_queue.cc:260] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.801684 15996 tablet_service.cc:1511] Processing CreateTablet for tablet 3c53d28c984a4016b8461698b47d3300 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:16:20.807221 15996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c53d28c984a4016b8461698b47d3300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.807710 15915 tablet_service.cc:1511] Processing CreateTablet for tablet 28cbf604c51f48da99cb7e3089651236 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:16:20.808530 16145 tablet_service.cc:1511] Processing CreateTablet for tablet 28cbf604c51f48da99cb7e3089651236 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:16:20.809067 15915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28cbf604c51f48da99cb7e3089651236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.809257 16144 tablet_service.cc:1511] Processing CreateTablet for tablet de7a15b080c54551b2c51bee7af64600 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:16:20.809900 16145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28cbf604c51f48da99cb7e3089651236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.803710 16204 ts_tablet_manager.cc:1434] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
W20260709 13:16:20.792805 15972 leader_election.cc:343] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.762065 16202 consensus_queue.cc:260] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.816051 16237 raft_consensus.cc:493] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.821338 16237 raft_consensus.cc:515] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.824506 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7473dd845e447a0b5a453e01dbb4b6b" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:20.825735 16155 raft_consensus.cc:2468] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:20.827140 16237 leader_election.cc:290] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:20.827775 16144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de7a15b080c54551b2c51bee7af64600. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.828620 15899 leader_election.cc:304] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:20.828811 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c895cd2e2a144ff692fcb615c095e82a" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:20.831636 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c895cd2e2a144ff692fcb615c095e82a" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
W20260709 13:16:20.832266 15971 leader_election.cc:343] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.833438 15899 leader_election.cc:304] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:20.835309 15971 leader_election.cc:304] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:20.836443 16209 raft_consensus.cc:2749] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:20.837409 15972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.67:32795, user_credentials={real_user=slave}} blocked reactor thread for 44748.9us
I20260709 13:16:20.838594 16213 raft_consensus.cc:2804] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:20.840004 16237 raft_consensus.cc:2749] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.840102 16213 raft_consensus.cc:493] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.840656 16213 raft_consensus.cc:3060] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.853691 15995 tablet_service.cc:1511] Processing CreateTablet for tablet 28cbf604c51f48da99cb7e3089651236 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:16:20.856757 15995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28cbf604c51f48da99cb7e3089651236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.860117 16082 raft_consensus.cc:2468] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:20.860466 15918 tablet_service.cc:1511] Processing CreateTablet for tablet f72ec4c0d31742baa2d0d9d0aad7caed (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:16:20.861806 15918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f72ec4c0d31742baa2d0d9d0aad7caed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.864460 16251 raft_consensus.cc:515] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.865676 16008 raft_consensus.cc:2393] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bdeb23ab98c34a269f5b2e71b03c5e4e in current term 1: Already voted for candidate 903b3435239241eab01ea54ffb015ee3 in this term.
I20260709 13:16:20.867120 15897 leader_election.cc:304] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3
I20260709 13:16:20.866127 16071 tablet_service.cc:1511] Processing CreateTablet for tablet f72ec4c0d31742baa2d0d9d0aad7caed (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:16:20.871569 16071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f72ec4c0d31742baa2d0d9d0aad7caed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.870090 16142 tablet_service.cc:1511] Processing CreateTablet for tablet ccc0c303e2dd466caf0e684c6ea83284 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:16:20.873734 16230 raft_consensus.cc:493] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.881717 16142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc0c303e2dd466caf0e684c6ea83284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.878160 16237 raft_consensus.cc:2804] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:20.886605 16251 leader_election.cc:290] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:20.887014 16204 tablet_bootstrap.cc:492] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:20.888203 16143 tablet_service.cc:1511] Processing CreateTablet for tablet f72ec4c0d31742baa2d0d9d0aad7caed (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:16:20.885803 16070 tablet_service.cc:1511] Processing CreateTablet for tablet ad97b67157f44c618ddfebbc1c25588b (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:16:20.889782 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7473dd845e447a0b5a453e01dbb4b6b" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:20.891702 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7473dd845e447a0b5a453e01dbb4b6b" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:20.892092 16234 consensus_queue.cc:1048] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:20.929634 16237 raft_consensus.cc:697] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:20.930140 16141 tablet_service.cc:1511] Processing CreateTablet for tablet ad97b67157f44c618ddfebbc1c25588b (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:16:20.930752 16237 consensus_queue.cc:237] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:20.932077 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad97b67157f44c618ddfebbc1c25588b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.934266 16155 raft_consensus.cc:3060] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:20.893332 15997 tablet_service.cc:1511] Processing CreateTablet for tablet ad97b67157f44c618ddfebbc1c25588b (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:16:20.938597 15997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad97b67157f44c618ddfebbc1c25588b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.940845 16213 raft_consensus.cc:515] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.942327 16070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad97b67157f44c618ddfebbc1c25588b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.896067 16201 tablet_bootstrap.cc:492] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:20.943554 16201 ts_tablet_manager.cc:1403] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.472s	user 0.008s	sys 0.023s
I20260709 13:16:20.944216 16154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7473dd845e447a0b5a453e01dbb4b6b" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:20.946786 16213 leader_election.cc:290] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:20.947909 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7473dd845e447a0b5a453e01dbb4b6b" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:20.948963 16008 raft_consensus.cc:2393] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bdeb23ab98c34a269f5b2e71b03c5e4e in current term 1: Already voted for candidate 903b3435239241eab01ea54ffb015ee3 in this term.
I20260709 13:16:20.950220 15933 raft_consensus.cc:2393] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 903b3435239241eab01ea54ffb015ee3 in current term 1: Already voted for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in this term.
I20260709 13:16:20.904354 16202 ts_tablet_manager.cc:1434] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.169s	user 0.006s	sys 0.004s
I20260709 13:16:20.952046 16143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f72ec4c0d31742baa2d0d9d0aad7caed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:20.953145 16202 tablet_bootstrap.cc:492] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:20.913700 16230 raft_consensus.cc:515] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:20.956261 16230 leader_election.cc:290] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:20.917699 16204 tablet_bootstrap.cc:654] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.958356 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9df103185604f988d3abe94ae731141" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:20.961921 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9df103185604f988d3abe94ae731141" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:20.958365 15898 leader_election.cc:304] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8
W20260709 13:16:20.963510 16121 leader_election.cc:343] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.964707 16234 raft_consensus.cc:2749] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:16:20.965166 16119 leader_election.cc:343] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:20.965610 16119 leader_election.cc:304] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:20.967469 16201 raft_consensus.cc:359] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:20.968181 16201 raft_consensus.cc:385] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:20.968324 16230 raft_consensus.cc:2749] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:20.968451 16201 raft_consensus.cc:740] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:20.969313 16201 consensus_queue.cc:260] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:20.987155 16202 tablet_bootstrap.cc:654] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:20.991767 16201 ts_tablet_manager.cc:1434] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.048s	user 0.004s	sys 0.003s
I20260709 13:16:20.993016 16201 tablet_bootstrap.cc:492] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:20.978021 16155 raft_consensus.cc:2468] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:21.007371 15974 leader_election.cc:304] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:21.011793 16251 raft_consensus.cc:2804] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:21.015933 16251 raft_consensus.cc:697] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:21.016850 16251 consensus_queue.cc:237] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:21.017371 15923 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd99621a84f4fe994996601dae8988c (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:16:21.018790 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd99621a84f4fe994996601dae8988c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.019941 16071 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd99621a84f4fe994996601dae8988c (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:16:21.021171 16071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd99621a84f4fe994996601dae8988c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.022087 15829 catalog_manager.cc:5697] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.027287 16268 consensus_queue.cc:1048] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:21.029310 16145 tablet_service.cc:1511] Processing CreateTablet for tablet b5e6891e9cab4a7298664c3c431ce36d (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:16:21.030575 16145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e6891e9cab4a7298664c3c431ce36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.033133 16201 tablet_bootstrap.cc:654] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.037344 15914 tablet_service.cc:1511] Processing CreateTablet for tablet b5e6891e9cab4a7298664c3c431ce36d (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:16:21.051811 16211 raft_consensus.cc:493] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.052421 15914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e6891e9cab4a7298664c3c431ce36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.052188 15996 tablet_service.cc:1511] Processing CreateTablet for tablet 5bd99621a84f4fe994996601dae8988c (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:16:21.052340 16211 raft_consensus.cc:515] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.053438 15996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bd99621a84f4fe994996601dae8988c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.057351 16066 tablet_service.cc:1511] Processing CreateTablet for tablet c6a3eeeeab11406e90e1fd10d63228e8 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:16:21.057775 16211 leader_election.cc:290] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:21.060614 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2dba979b4b4cfdb4eb1159d36ae158" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:21.061412 15933 raft_consensus.cc:2468] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:21.061770 16066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6a3eeeeab11406e90e1fd10d63228e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.062994 16045 leader_election.cc:304] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:21.064240 16211 raft_consensus.cc:2804] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:21.064635 16211 raft_consensus.cc:493] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.065029 16211 raft_consensus.cc:3060] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.075946 15992 tablet_service.cc:1511] Processing CreateTablet for tablet b5e6891e9cab4a7298664c3c431ce36d (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:16:21.086287 16203 tablet_bootstrap.cc:492] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:21.088593 16203 ts_tablet_manager.cc:1403] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.444s	user 0.026s	sys 0.023s
I20260709 13:16:21.095234 16204 tablet_bootstrap.cc:492] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:21.095754 16204 ts_tablet_manager.cc:1403] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.209s	user 0.010s	sys 0.007s
I20260709 13:16:21.097606 16211 raft_consensus.cc:515] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.099164 15829 catalog_manager.cc:5697] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:21.100467 16211 leader_election.cc:290] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:21.101573 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2dba979b4b4cfdb4eb1159d36ae158" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:21.102139 15933 raft_consensus.cc:3060] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.105969 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e6891e9cab4a7298664c3c431ce36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.111249 16204 raft_consensus.cc:359] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:21.111492 15933 raft_consensus.cc:2468] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:21.112039 16204 raft_consensus.cc:385] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.112416 16204 raft_consensus.cc:740] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.113173 16045 leader_election.cc:304] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:21.113183 16204 consensus_queue.cc:260] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:21.116863 16204 ts_tablet_manager.cc:1434] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:16:21.118010 16204 tablet_bootstrap.cc:492] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:21.123848 16141 tablet_service.cc:1511] Processing CreateTablet for tablet c6a3eeeeab11406e90e1fd10d63228e8 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:16:21.125463 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6a3eeeeab11406e90e1fd10d63228e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.128041 16204 tablet_bootstrap.cc:654] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.128448 16211 raft_consensus.cc:2804] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:21.129737 16211 raft_consensus.cc:697] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:21.131075 16211 consensus_queue.cc:237] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.148502 15915 tablet_service.cc:1511] Processing CreateTablet for tablet c6a3eeeeab11406e90e1fd10d63228e8 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:16:21.150125 15915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6a3eeeeab11406e90e1fd10d63228e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.151062 16251 consensus_queue.cc:1048] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:21.153249 15997 tablet_service.cc:1511] Processing CreateTablet for tablet d7ddd2107d2e4e73a47f9e4f6bdc73fb (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:16:21.154892 15997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7ddd2107d2e4e73a47f9e4f6bdc73fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.165845 16251 raft_consensus.cc:493] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.166427 16251 raft_consensus.cc:515] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.166888 16203 raft_consensus.cc:359] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.167819 16203 raft_consensus.cc:385] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.168169 16203 raft_consensus.cc:740] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.169240 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6aa884c0889444ea8aa23dbe7bcfe9de" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:21.169061 16203 consensus_queue.cc:260] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.172343 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6aa884c0889444ea8aa23dbe7bcfe9de" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:21.172858 16203 ts_tablet_manager.cc:1434] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.001s
I20260709 13:16:21.173107 16080 raft_consensus.cc:2393] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 903b3435239241eab01ea54ffb015ee3 in current term 1: Already voted for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in this term.
I20260709 13:16:21.174149 16203 tablet_bootstrap.cc:492] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:21.178476 16248 consensus_queue.cc:1048] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:16:21.179895 16251 leader_election.cc:290] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:21.189733 16203 tablet_bootstrap.cc:654] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.185729 15971 leader_election.cc:304] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:21.207917 16204 tablet_bootstrap.cc:492] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:21.208675 16204 ts_tablet_manager.cc:1403] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.091s	user 0.009s	sys 0.007s
I20260709 13:16:21.199870 16251 raft_consensus.cc:3060] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.216050 16070 tablet_service.cc:1511] Processing CreateTablet for tablet d7ddd2107d2e4e73a47f9e4f6bdc73fb (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:16:21.217367 16070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7ddd2107d2e4e73a47f9e4f6bdc73fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.217613 16201 tablet_bootstrap.cc:492] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:21.218299 16201 ts_tablet_manager.cc:1403] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.226s	user 0.012s	sys 0.010s
I20260709 13:16:21.237300 15933 raft_consensus.cc:3060] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.249168 16202 tablet_bootstrap.cc:492] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:21.249727 16202 ts_tablet_manager.cc:1403] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.297s	user 0.026s	sys 0.006s
I20260709 13:16:21.252311 16202 raft_consensus.cc:359] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.252842 16202 raft_consensus.cc:385] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.253132 16202 raft_consensus.cc:740] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.253993 16202 consensus_queue.cc:260] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.254670 16201 raft_consensus.cc:359] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:21.258972 16202 ts_tablet_manager.cc:1434] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.005s
I20260709 13:16:21.279981 16202 tablet_bootstrap.cc:492] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:21.280287 16201 raft_consensus.cc:385] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.269718 16145 tablet_service.cc:1511] Processing CreateTablet for tablet d7ddd2107d2e4e73a47f9e4f6bdc73fb (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:16:21.285113 16145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7ddd2107d2e4e73a47f9e4f6bdc73fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.271771 16066 tablet_service.cc:1511] Processing CreateTablet for tablet 9ca16f452c2a4b19a53a12119268b92d (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:16:21.231256 16204 raft_consensus.cc:359] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.256834 15828 catalog_manager.cc:5697] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.276427 15992 tablet_service.cc:1511] Processing CreateTablet for tablet 9ca16f452c2a4b19a53a12119268b92d (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:16:21.292414 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ca16f452c2a4b19a53a12119268b92d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.297968 16203 tablet_bootstrap.cc:492] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:21.298494 16203 ts_tablet_manager.cc:1403] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.125s	user 0.033s	sys 0.017s
I20260709 13:16:21.280632 15914 tablet_service.cc:1511] Processing CreateTablet for tablet 9ca16f452c2a4b19a53a12119268b92d (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:16:21.300098 15914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ca16f452c2a4b19a53a12119268b92d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.301470 16203 raft_consensus.cc:359] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.283129 16154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2dba979b4b4cfdb4eb1159d36ae158" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
W20260709 13:16:21.303745 16048 leader_election.cc:343] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.305301 16143 tablet_service.cc:1511] Processing CreateTablet for tablet 69d7d4e0bc194613bb410e793fd7bd39 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:16:21.306665 16143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69d7d4e0bc194613bb410e793fd7bd39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.306887 15915 tablet_service.cc:1511] Processing CreateTablet for tablet 69d7d4e0bc194613bb410e793fd7bd39 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:16:21.308003 15915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69d7d4e0bc194613bb410e793fd7bd39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.283939 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2dba979b4b4cfdb4eb1159d36ae158" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:21.262611 16251 raft_consensus.cc:2749] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:21.321713 16048 leader_election.cc:343] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.286939 16202 tablet_bootstrap.cc:654] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.287904 16066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ca16f452c2a4b19a53a12119268b92d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.290467 16204 raft_consensus.cc:385] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.326097 16204 raft_consensus.cc:740] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.327258 16204 consensus_queue.cc:260] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.286825 16201 raft_consensus.cc:740] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.331787 16201 consensus_queue.cc:260] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:21.336099 16070 tablet_service.cc:1511] Processing CreateTablet for tablet 3132182819ce40fdb691736db3dd1061 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:16:21.337898 16070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3132182819ce40fdb691736db3dd1061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.302215 16203 raft_consensus.cc:385] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.338505 16203 raft_consensus.cc:740] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.339299 16203 consensus_queue.cc:260] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.340265 16201 ts_tablet_manager.cc:1434] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.122s	user 0.003s	sys 0.005s
I20260709 13:16:21.341372 16201 tablet_bootstrap.cc:492] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:21.342396 16203 ts_tablet_manager.cc:1434] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 13:16:21.348270 16204 ts_tablet_manager.cc:1434] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.121s	user 0.007s	sys 0.003s
I20260709 13:16:21.349542 16204 tablet_bootstrap.cc:492] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:21.350206 16203 tablet_bootstrap.cc:492] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:21.355541 16141 tablet_service.cc:1511] Processing CreateTablet for tablet 3132182819ce40fdb691736db3dd1061 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:16:21.356653 16202 tablet_bootstrap.cc:492] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:21.356818 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3132182819ce40fdb691736db3dd1061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.357125 16202 ts_tablet_manager.cc:1403] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.077s	user 0.012s	sys 0.007s
I20260709 13:16:21.358995 16203 tablet_bootstrap.cc:654] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.360078 16202 raft_consensus.cc:359] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.360756 16202 raft_consensus.cc:385] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.361021 16202 raft_consensus.cc:740] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.361701 16202 consensus_queue.cc:260] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.363785 16202 ts_tablet_manager.cc:1434] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:16:21.365012 16202 tablet_bootstrap.cc:492] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:21.376420 15996 tablet_service.cc:1511] Processing CreateTablet for tablet 0b83d43d98074cb09deac765b8b035e6 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:16:21.392087 16202 tablet_bootstrap.cc:654] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.392942 15996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b83d43d98074cb09deac765b8b035e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.383841 15918 tablet_service.cc:1511] Processing CreateTablet for tablet d6cabb92769049d088d88c54ec482c85 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:16:21.393991 16066 tablet_service.cc:1511] Processing CreateTablet for tablet 0b83d43d98074cb09deac765b8b035e6 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:16:21.394578 15918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6cabb92769049d088d88c54ec482c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.395308 16066 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b83d43d98074cb09deac765b8b035e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.369516 15914 tablet_service.cc:1511] Processing CreateTablet for tablet 3132182819ce40fdb691736db3dd1061 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:16:21.397905 15914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3132182819ce40fdb691736db3dd1061. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.381348 16237 consensus_queue.cc:1048] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:16:21.365612 15992 tablet_service.cc:1511] Processing CreateTablet for tablet 69d7d4e0bc194613bb410e793fd7bd39 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:16:21.406905 16204 tablet_bootstrap.cc:654] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.407969 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69d7d4e0bc194613bb410e793fd7bd39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.392947 16201 tablet_bootstrap.cc:654] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.381760 16142 tablet_service.cc:1511] Processing CreateTablet for tablet 0b83d43d98074cb09deac765b8b035e6 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:16:21.411849 16142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b83d43d98074cb09deac765b8b035e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.421674 16203 tablet_bootstrap.cc:492] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:21.414578 16234 consensus_queue.cc:1048] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:16:21.422199 16203 ts_tablet_manager.cc:1403] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.072s	user 0.010s	sys 0.000s
I20260709 13:16:21.427662 16202 tablet_bootstrap.cc:492] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:21.428167 16202 ts_tablet_manager.cc:1403] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.063s	user 0.014s	sys 0.025s
I20260709 13:16:21.433511 16203 raft_consensus.cc:359] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.435313 16203 raft_consensus.cc:385] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.435983 16203 raft_consensus.cc:740] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.440822 16251 raft_consensus.cc:493] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.440927 16071 tablet_service.cc:1511] Processing CreateTablet for tablet d6cabb92769049d088d88c54ec482c85 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:16:21.441396 16251 raft_consensus.cc:515] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:21.442140 16071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6cabb92769049d088d88c54ec482c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.445906 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9df103185604f988d3abe94ae731141" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:21.447494 16065 tablet_service.cc:1511] Processing CreateTablet for tablet e58eccbbe71e465188784170045cdb34 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:16:21.448329 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9df103185604f988d3abe94ae731141" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:21.448786 16065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e58eccbbe71e465188784170045cdb34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.448948 15933 raft_consensus.cc:2468] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:21.449349 16141 tablet_service.cc:1511] Processing CreateTablet for tablet 4332ee7a936341baad672bfc16703cc7 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:16:21.450167 16155 raft_consensus.cc:2468] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:21.450605 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4332ee7a936341baad672bfc16703cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.452297 16143 tablet_service.cc:1511] Processing CreateTablet for tablet e58eccbbe71e465188784170045cdb34 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:16:21.453357 16143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e58eccbbe71e465188784170045cdb34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.454648 15974 leader_election.cc:304] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:21.455498 16248 raft_consensus.cc:2804] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:21.455843 16248 raft_consensus.cc:493] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.456156 16248 raft_consensus.cc:3060] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.456557 15996 tablet_service.cc:1511] Processing CreateTablet for tablet d6cabb92769049d088d88c54ec482c85 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:16:21.457669 15996 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6cabb92769049d088d88c54ec482c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.443636 16251 leader_election.cc:290] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:21.459885 16202 raft_consensus.cc:359] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:21.462826 16202 raft_consensus.cc:385] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.464888 15918 tablet_service.cc:1511] Processing CreateTablet for tablet 4332ee7a936341baad672bfc16703cc7 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:16:21.466192 15918 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4332ee7a936341baad672bfc16703cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.465901 15997 tablet_service.cc:1511] Processing CreateTablet for tablet 3424ea4fd86e4453a5d9fde62fc43e7f (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:16:21.467227 15997 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3424ea4fd86e4453a5d9fde62fc43e7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.471879 16230 raft_consensus.cc:493] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.472584 16230 raft_consensus.cc:515] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.475040 15992 tablet_service.cc:1511] Processing CreateTablet for tablet 4332ee7a936341baad672bfc16703cc7 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:16:21.476148 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4332ee7a936341baad672bfc16703cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.477007 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1708b4e2385e408abc600d9b2f6d42a1" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
W20260709 13:16:21.478610 16119 leader_election.cc:343] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.480319 16145 tablet_service.cc:1511] Processing CreateTablet for tablet 3424ea4fd86e4453a5d9fde62fc43e7f (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:16:21.481380 16145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3424ea4fd86e4453a5d9fde62fc43e7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.476864 16070 tablet_service.cc:1511] Processing CreateTablet for tablet 3424ea4fd86e4453a5d9fde62fc43e7f (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:16:21.482157 15915 tablet_service.cc:1511] Processing CreateTablet for tablet 9719c47150e74128b537e918110116e0 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:16:21.483047 16070 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3424ea4fd86e4453a5d9fde62fc43e7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.483253 15915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9719c47150e74128b537e918110116e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.483444 16071 tablet_service.cc:1511] Processing CreateTablet for tablet 9719c47150e74128b537e918110116e0 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:16:21.484509 16071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9719c47150e74128b537e918110116e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.484218 15914 tablet_service.cc:1511] Processing CreateTablet for tablet e58eccbbe71e465188784170045cdb34 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:16:21.485301 15914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e58eccbbe71e465188784170045cdb34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.484887 16203 consensus_queue.cc:260] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.486598 15995 tablet_service.cc:1511] Processing CreateTablet for tablet 9719c47150e74128b537e918110116e0 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:16:21.487546 16203 ts_tablet_manager.cc:1434] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.060s	user 0.005s	sys 0.001s
I20260709 13:16:21.487785 16144 tablet_service.cc:1511] Processing CreateTablet for tablet 4712e5b3e765415b916c9bf89f5b0d74 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:16:21.488654 16203 tablet_bootstrap.cc:492] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:21.488976 16144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4712e5b3e765415b916c9bf89f5b0d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.487820 15995 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9719c47150e74128b537e918110116e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.496425 16201 tablet_bootstrap.cc:492] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:21.497020 16201 ts_tablet_manager.cc:1403] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.156s	user 0.010s	sys 0.004s
I20260709 13:16:21.494045 16230 leader_election.cc:290] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:21.474086 16248 raft_consensus.cc:515] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:21.512255 16248 leader_election.cc:290] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:21.499739 16201 raft_consensus.cc:359] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.513186 16201 raft_consensus.cc:385] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.513465 16201 raft_consensus.cc:740] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.514698 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9df103185604f988d3abe94ae731141" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:21.515514 16155 raft_consensus.cc:3060] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.514755 15923 tablet_service.cc:1511] Processing CreateTablet for tablet 4712e5b3e765415b916c9bf89f5b0d74 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:16:21.517104 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4712e5b3e765415b916c9bf89f5b0d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.518779 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1708b4e2385e408abc600d9b2f6d42a1" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:21.520632 16082 raft_consensus.cc:2468] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 0.
I20260709 13:16:21.519835 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9df103185604f988d3abe94ae731141" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:21.521445 15933 raft_consensus.cc:3060] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.523640 16120 leader_election.cc:304] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:21.525199 16230 raft_consensus.cc:2804] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:21.525748 16230 raft_consensus.cc:493] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.527397 16230 raft_consensus.cc:3060] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.463177 16202 raft_consensus.cc:740] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.535588 16202 consensus_queue.cc:260] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:21.549182 16155 raft_consensus.cc:2468] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:21.550633 16211 raft_consensus.cc:493] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.556658 16201 consensus_queue.cc:260] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.564682 15933 raft_consensus.cc:2468] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:21.566006 16251 consensus_queue.cc:1048] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:21.581508 16248 consensus_queue.cc:1048] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:21.583549 15974 leader_election.cc:304] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:21.584731 16248 raft_consensus.cc:2804] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:21.585198 16248 raft_consensus.cc:697] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:21.585942 16248 consensus_queue.cc:237] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:21.577118 16204 tablet_bootstrap.cc:492] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:21.590358 16204 ts_tablet_manager.cc:1403] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.241s	user 0.013s	sys 0.030s
I20260709 13:16:21.577478 16211 raft_consensus.cc:515] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.604642 16203 tablet_bootstrap.cc:654] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.611523 16211 leader_election.cc:290] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:21.614208 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cca9877952a413a9112ea61481bc582" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:21.614938 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cca9877952a413a9112ea61481bc582" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:21.624526 16155 raft_consensus.cc:2468] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:21.624573 15933 raft_consensus.cc:2468] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:21.625397 15923 tablet_service.cc:1511] Processing CreateTablet for tablet 8a7a7d029fb040b881d5165c98065a12 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:16:21.625931 16045 leader_election.cc:304] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:21.626842 16275 raft_consensus.cc:2804] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:21.626668 16204 raft_consensus.cc:359] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.627031 15923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a7a7d029fb040b881d5165c98065a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.627260 16275 raft_consensus.cc:493] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.628512 16275 raft_consensus.cc:3060] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.621829 15992 tablet_service.cc:1511] Processing CreateTablet for tablet 4712e5b3e765415b916c9bf89f5b0d74 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:16:21.630398 16204 raft_consensus.cc:385] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.630707 16204 raft_consensus.cc:740] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.630892 15992 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4712e5b3e765415b916c9bf89f5b0d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.574280 16230 raft_consensus.cc:515] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.632354 16201 ts_tablet_manager.cc:1434] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.135s	user 0.004s	sys 0.004s
I20260709 13:16:21.634125 16071 tablet_service.cc:1511] Processing CreateTablet for tablet 8a7a7d029fb040b881d5165c98065a12 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:16:21.635149 16071 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a7a7d029fb040b881d5165c98065a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.637034 16204 consensus_queue.cc:260] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.641988 16211 raft_consensus.cc:493] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.642475 16230 leader_election.cc:290] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:21.643220 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1708b4e2385e408abc600d9b2f6d42a1" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:21.643853 16082 raft_consensus.cc:3060] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.646389 16201 tablet_bootstrap.cc:492] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:21.642506 16211 raft_consensus.cc:515] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.649720 16211 leader_election.cc:290] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:21.650481 16294 raft_consensus.cc:493] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.653203 16201 tablet_bootstrap.cc:654] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.656518 16202 ts_tablet_manager.cc:1434] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.228s	user 0.026s	sys 0.034s
I20260709 13:16:21.657604 16202 tablet_bootstrap.cc:492] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:21.658919 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6469b36c7444ba906df4c1f29f6f0e" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:21.660761 16155 raft_consensus.cc:2468] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:21.651136 16294 raft_consensus.cc:515] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.659651 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1708b4e2385e408abc600d9b2f6d42a1" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:21.662562 16048 leader_election.cc:304] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
W20260709 13:16:21.664284 16119 leader_election.cc:343] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.667095 16082 raft_consensus.cc:2468] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 1.
I20260709 13:16:21.668221 16120 leader_election.cc:304] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:21.669449 16293 raft_consensus.cc:2804] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:21.669943 16293 raft_consensus.cc:697] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
I20260709 13:16:21.670727 16202 tablet_bootstrap.cc:654] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.674013 16204 ts_tablet_manager.cc:1434] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.000s
I20260709 13:16:21.636641 16145 tablet_service.cc:1511] Processing CreateTablet for tablet 8a7a7d029fb040b881d5165c98065a12 (DEFAULT_TABLE table=test-workload [id=b9098ad012f340d78e51bb21f19aef1d]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:16:21.676307 16145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a7a7d029fb040b881d5165c98065a12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:21.674494 16251 raft_consensus.cc:493] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.674603 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6469b36c7444ba906df4c1f29f6f0e" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:21.678915 16251 raft_consensus.cc:515] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.674852 15829 catalog_manager.cc:5697] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:21.679790 16211 consensus_queue.cc:1048] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:21.680756 16007 raft_consensus.cc:2468] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:21.681109 16251 leader_election.cc:290] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:21.685209 16204 tablet_bootstrap.cc:492] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:21.685220 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6469b36c7444ba906df4c1f29f6f0e" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:21.686021 16155 raft_consensus.cc:2468] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:21.686946 16275 raft_consensus.cc:515] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.687299 15974 leader_election.cc:304] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:21.688129 16251 raft_consensus.cc:2804] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:21.688485 16251 raft_consensus.cc:493] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.688802 16251 raft_consensus.cc:3060] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.689483 16275 leader_election.cc:290] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:21.691812 16204 tablet_bootstrap.cc:654] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.695444 16201 tablet_bootstrap.cc:492] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:21.695861 16201 ts_tablet_manager.cc:1403] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.000s
I20260709 13:16:21.698062 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6469b36c7444ba906df4c1f29f6f0e" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:21.698637 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cca9877952a413a9112ea61481bc582" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:21.702268 16155 raft_consensus.cc:3060] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.713287 16298 raft_consensus.cc:2804] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:21.713726 16298 raft_consensus.cc:493] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.714037 16298 raft_consensus.cc:3060] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.714188 16294 leader_election.cc:290] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:21.670768 16293 consensus_queue.cc:237] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.720758 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7454f1cf21c0400c8a3a8290d5d148dd" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:21.721529 16081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7454f1cf21c0400c8a3a8290d5d148dd" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:21.725683 16201 raft_consensus.cc:359] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.726157 16201 raft_consensus.cc:385] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.726318 16201 raft_consensus.cc:740] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
W20260709 13:16:21.727376 16120 leader_election.cc:343] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.728528 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cca9877952a413a9112ea61481bc582" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:21.729130 15932 raft_consensus.cc:3060] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.737025 16201 consensus_queue.cc:260] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
W20260709 13:16:21.730317 16121 leader_election.cc:343] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.743417 16121 leader_election.cc:304] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:21.744442 16294 raft_consensus.cc:2749] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:21.747045 16298 raft_consensus.cc:515] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.748587 16080 raft_consensus.cc:2393] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 903b3435239241eab01ea54ffb015ee3 in current term 1: Already voted for candidate d4d6e13a8f4a473098351ada96fd7dab in this term.
I20260709 13:16:21.754529 16248 raft_consensus.cc:493] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.755029 16248 raft_consensus.cc:515] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.757062 16248 leader_election.cc:290] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:21.758407 16154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7454f1cf21c0400c8a3a8290d5d148dd" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:21.758747 16201 ts_tablet_manager.cc:1434] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.000s
I20260709 13:16:21.762933 16202 tablet_bootstrap.cc:492] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:21.763396 16202 ts_tablet_manager.cc:1403] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.106s	user 0.017s	sys 0.016s
I20260709 13:16:21.764318 16155 raft_consensus.cc:2468] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:21.758934 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7454f1cf21c0400c8a3a8290d5d148dd" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:21.766379 16048 leader_election.cc:304] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:21.766217 16202 raft_consensus.cc:359] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.767072 16202 raft_consensus.cc:385] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.767349 16202 raft_consensus.cc:740] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.767508 16300 raft_consensus.cc:2804] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:21.767999 16300 raft_consensus.cc:697] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:21.768167 16202 consensus_queue.cc:260] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.768806 16300 consensus_queue.cc:237] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.770570 16202 ts_tablet_manager.cc:1434] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:16:21.771567 16202 tablet_bootstrap.cc:492] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
W20260709 13:16:21.772032 15972 leader_election.cc:343] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.759806 16154 raft_consensus.cc:2468] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:21.774078 15974 leader_election.cc:304] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:21.776995 16251 raft_consensus.cc:515] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.777426 16202 tablet_bootstrap.cc:654] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.778901 16203 tablet_bootstrap.cc:492] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:21.779330 16203 ts_tablet_manager.cc:1403] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.291s	user 0.015s	sys 0.012s
I20260709 13:16:21.781203 16154 raft_consensus.cc:3060] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:21.783036 15879 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:21.781942 16203 raft_consensus.cc:359] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.783862 16203 raft_consensus.cc:385] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.784194 16203 raft_consensus.cc:740] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.785085 16203 consensus_queue.cc:260] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.765143 16201 tablet_bootstrap.cc:492] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:21.788838 16203 ts_tablet_manager.cc:1434] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:16:21.788275 16209 raft_consensus.cc:2804] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:21.789535 16209 raft_consensus.cc:493] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.789880 16203 tablet_bootstrap.cc:492] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:21.789935 16209 raft_consensus.cc:3060] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.791796 15932 raft_consensus.cc:2468] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:21.796554 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6469b36c7444ba906df4c1f29f6f0e" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:21.797232 16155 raft_consensus.cc:3060] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.800074 16298 leader_election.cc:290] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:21.801339 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6469b36c7444ba906df4c1f29f6f0e" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:21.802657 16294 raft_consensus.cc:493] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.803153 16294 raft_consensus.cc:515] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:21.804410 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6469b36c7444ba906df4c1f29f6f0e" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:21.805276 16082 raft_consensus.cc:2393] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 903b3435239241eab01ea54ffb015ee3 in current term 1: Already voted for candidate d4d6e13a8f4a473098351ada96fd7dab in this term.
I20260709 13:16:21.805717 16294 leader_election.cc:290] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:21.793303 16251 leader_election.cc:290] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:21.808449 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718e00ba6fa34542a71fc6d5758e92a1" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:21.808990 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718e00ba6fa34542a71fc6d5758e92a1" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:21.809948 16201 tablet_bootstrap.cc:654] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:21.811684 16121 leader_election.cc:343] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:21.813040 16119 leader_election.cc:343] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.813596 16119 leader_election.cc:304] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:21.814067 16203 tablet_bootstrap.cc:654] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.815734 16204 tablet_bootstrap.cc:492] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:21.816463 16204 ts_tablet_manager.cc:1403] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.131s	user 0.010s	sys 0.023s
I20260709 13:16:21.823132 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab6469b36c7444ba906df4c1f29f6f0e" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:21.824115 16007 raft_consensus.cc:2393] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d4d6e13a8f4a473098351ada96fd7dab in current term 1: Already voted for candidate 903b3435239241eab01ea54ffb015ee3 in this term.
I20260709 13:16:21.816991 16294 raft_consensus.cc:2749] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:21.825489 16047 leader_election.cc:304] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:21.826722 16275 raft_consensus.cc:2749] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:21.828536 16202 tablet_bootstrap.cc:492] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:21.829080 16202 ts_tablet_manager.cc:1403] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.058s	user 0.018s	sys 0.019s
I20260709 13:16:21.831084 16209 raft_consensus.cc:515] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.831537 16155 raft_consensus.cc:2468] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:21.832494 16202 raft_consensus.cc:359] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.833420 16202 raft_consensus.cc:385] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.833969 16202 raft_consensus.cc:740] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.835426 15974 leader_election.cc:304] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:21.836463 16251 raft_consensus.cc:2804] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:21.837064 16251 raft_consensus.cc:697] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:21.838104 16251 consensus_queue.cc:237] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.842504 16204 raft_consensus.cc:359] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.844183 16204 raft_consensus.cc:385] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.844605 16204 raft_consensus.cc:740] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.845602 16204 consensus_queue.cc:260] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.850721 16204 ts_tablet_manager.cc:1434] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 13:16:21.851635 16204 tablet_bootstrap.cc:492] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:21.854920 16202 consensus_queue.cc:260] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.859184 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7454f1cf21c0400c8a3a8290d5d148dd" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:21.855757 16300 consensus_queue.cc:1048] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:16:21.862536 16209 leader_election.cc:290] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:21.864590 16204 tablet_bootstrap.cc:654] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.865270 16201 tablet_bootstrap.cc:492] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:21.865717 16201 ts_tablet_manager.cc:1403] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.101s	user 0.023s	sys 0.001s
W20260709 13:16:21.866298 15972 leader_election.cc:343] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.867296 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7454f1cf21c0400c8a3a8290d5d148dd" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:21.868083 16155 raft_consensus.cc:3060] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.869287 16201 raft_consensus.cc:359] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.869964 16201 raft_consensus.cc:385] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.870222 16201 raft_consensus.cc:740] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.869536 15828 catalog_manager.cc:5697] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 1 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.873221 15829 catalog_manager.cc:5697] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.874827 16201 consensus_queue.cc:260] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.879426 16201 ts_tablet_manager.cc:1434] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:16:21.880415 16201 tablet_bootstrap.cc:492] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:21.879881 15827 catalog_manager.cc:5697] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:21.881793 16202 ts_tablet_manager.cc:1434] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.004s
I20260709 13:16:21.893738 16201 tablet_bootstrap.cc:654] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.893991 16203 tablet_bootstrap.cc:492] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:21.894660 16203 ts_tablet_manager.cc:1403] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.105s	user 0.016s	sys 0.013s
I20260709 13:16:21.899194 16202 tablet_bootstrap.cc:492] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:21.905423 16202 tablet_bootstrap.cc:654] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.907792 16203 raft_consensus.cc:359] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.908731 16203 raft_consensus.cc:385] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.909080 16203 raft_consensus.cc:740] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.910487 16203 consensus_queue.cc:260] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.914253 16155 raft_consensus.cc:2468] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:21.915848 15974 leader_election.cc:304] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:21.916921 16251 raft_consensus.cc:2804] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:21.917500 16251 raft_consensus.cc:697] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:21.918478 16298 raft_consensus.cc:493] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.918305 16251 consensus_queue.cc:237] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.919126 16298 raft_consensus.cc:515] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.922719 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08b5d04b39149c187ddb882b1b5d2de" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:21.923411 15933 raft_consensus.cc:2468] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:21.923168 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08b5d04b39149c187ddb882b1b5d2de" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:21.924500 16045 leader_election.cc:304] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:21.924845 16007 raft_consensus.cc:2468] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:21.925732 16275 raft_consensus.cc:2804] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:21.926108 16275 raft_consensus.cc:493] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.926330 16275 raft_consensus.cc:3060] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.930054 16298 leader_election.cc:290] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:21.932871 16298 raft_consensus.cc:493] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.933494 16298 raft_consensus.cc:515] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.936589 16203 ts_tablet_manager.cc:1434] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.000s
I20260709 13:16:21.936726 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb4ef2e4f6be44efbf83fb70838b4ae7" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:21.937614 15933 raft_consensus.cc:2468] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:21.937945 16203 tablet_bootstrap.cc:492] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:21.939229 16045 leader_election.cc:304] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:21.940366 16300 raft_consensus.cc:2804] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:21.940793 16300 raft_consensus.cc:493] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.941234 16300 raft_consensus.cc:3060] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.943759 16204 tablet_bootstrap.cc:492] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:21.944439 16204 ts_tablet_manager.cc:1403] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.093s	user 0.005s	sys 0.007s
I20260709 13:16:21.948289 16204 raft_consensus.cc:359] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.949141 16204 raft_consensus.cc:385] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.949469 16204 raft_consensus.cc:740] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.950501 16204 consensus_queue.cc:260] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.953310 16204 ts_tablet_manager.cc:1434] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:16:21.955116 16204 tablet_bootstrap.cc:492] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:21.957414 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb4ef2e4f6be44efbf83fb70838b4ae7" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:21.958117 16007 raft_consensus.cc:2468] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:21.960290 16298 leader_election.cc:290] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:21.960556 16201 tablet_bootstrap.cc:492] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:21.961066 16201 ts_tablet_manager.cc:1403] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.081s	user 0.031s	sys 0.030s
I20260709 13:16:21.962783 16202 tablet_bootstrap.cc:492] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:21.963227 16202 ts_tablet_manager.cc:1403] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.064s	user 0.004s	sys 0.011s
I20260709 13:16:21.964236 16300 raft_consensus.cc:515] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.966436 16202 raft_consensus.cc:359] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.965749 16275 raft_consensus.cc:515] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.967103 16202 raft_consensus.cc:385] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.967411 16202 raft_consensus.cc:740] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.967586 16203 tablet_bootstrap.cc:654] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.969463 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb4ef2e4f6be44efbf83fb70838b4ae7" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:21.969282 16202 consensus_queue.cc:260] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.970001 15933 raft_consensus.cc:3060] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.971549 16201 raft_consensus.cc:359] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.972182 16201 raft_consensus.cc:385] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.972429 16201 raft_consensus.cc:740] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.972662 16202 ts_tablet_manager.cc:1434] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:16:21.973078 16201 consensus_queue.cc:260] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.973673 16202 tablet_bootstrap.cc:492] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:21.975271 16201 ts_tablet_manager.cc:1434] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:16:21.976215 16201 tablet_bootstrap.cc:492] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:21.977710 15933 raft_consensus.cc:2468] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:21.979828 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08b5d04b39149c187ddb882b1b5d2de" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:21.984935 15932 raft_consensus.cc:3060] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.989341 16203 tablet_bootstrap.cc:492] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:21.989324 16293 raft_consensus.cc:493] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.990001 16203 ts_tablet_manager.cc:1403] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.052s	user 0.011s	sys 0.001s
I20260709 13:16:21.989975 16293 raft_consensus.cc:515] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:21.992261 16293 leader_election.cc:290] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:21.980826 16204 tablet_bootstrap.cc:654] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.992954 16203 raft_consensus.cc:359] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:21.993584 16203 raft_consensus.cc:385] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:21.993835 16203 raft_consensus.cc:740] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:21.994004 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a39139d4f2b48d09ba9eb320dfcf265" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:21.994558 16203 consensus_queue.cc:260] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
W20260709 13:16:21.995426 16120 leader_election.cc:343] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.996713 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a39139d4f2b48d09ba9eb320dfcf265" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:21.996989 16203 ts_tablet_manager.cc:1434] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:21.997452 16045 leader_election.cc:304] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
W20260709 13:16:21.998175 16119 leader_election.cc:343] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:21.998605 16119 leader_election.cc:304] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:21.981750 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb4ef2e4f6be44efbf83fb70838b4ae7" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:21.999038 16211 raft_consensus.cc:2804] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:21.982648 16251 raft_consensus.cc:493] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.999588 16211 raft_consensus.cc:697] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:21.999886 16251 raft_consensus.cc:515] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.000784 16211 consensus_queue.cc:237] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:21.983685 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e08b5d04b39149c187ddb882b1b5d2de" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:22.002722 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a3e44fcd5b438486e1ec0241ef1e8d" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:22.003427 16008 raft_consensus.cc:3060] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.003801 16251 leader_election.cc:290] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:21.999400 16007 raft_consensus.cc:3060] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:22.004243 15972 leader_election.cc:343] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.005530 16154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a3e44fcd5b438486e1ec0241ef1e8d" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:22.006127 16154 raft_consensus.cc:2468] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:22.007352 15974 leader_election.cc:304] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:22.007591 15932 raft_consensus.cc:2468] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:22.008428 16251 raft_consensus.cc:2804] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:22.008647 16045 leader_election.cc:304] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:22.009526 16298 raft_consensus.cc:2804] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:22.016273 16008 raft_consensus.cc:2468] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:21.984525 16201 tablet_bootstrap.cc:654] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:21.997937 16203 tablet_bootstrap.cc:492] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:21.999315 16293 raft_consensus.cc:2749] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:21.983404 16202 tablet_bootstrap.cc:654] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.008814 16251 raft_consensus.cc:493] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.024132 16251 raft_consensus.cc:3060] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:21.984012 16300 leader_election.cc:290] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:21.984366 16275 leader_election.cc:290] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:22.017688 16007 raft_consensus.cc:2468] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:22.027060 16203 tablet_bootstrap.cc:654] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.032932 16202 tablet_bootstrap.cc:492] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:22.033396 16202 ts_tablet_manager.cc:1403] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.000s
I20260709 13:16:22.033107 16251 raft_consensus.cc:515] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.035233 16251 leader_election.cc:290] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:22.036258 16202 raft_consensus.cc:359] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.036921 16202 raft_consensus.cc:385] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.037175 16202 raft_consensus.cc:740] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.037801 16204 tablet_bootstrap.cc:492] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:22.037784 16202 consensus_queue.cc:260] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.038400 16204 ts_tablet_manager.cc:1403] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.084s	user 0.017s	sys 0.011s
I20260709 13:16:22.039654 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a3e44fcd5b438486e1ec0241ef1e8d" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:22.040174 16202 ts_tablet_manager.cc:1434] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:16:22.041314 16202 tablet_bootstrap.cc:492] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:22.043350 15829 catalog_manager.cc:5697] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:22.045065 16154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a3e44fcd5b438486e1ec0241ef1e8d" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:22.045849 16154 raft_consensus.cc:3060] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.048905 16201 tablet_bootstrap.cc:492] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:22.049381 16201 ts_tablet_manager.cc:1403] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.073s	user 0.007s	sys 0.011s
I20260709 13:16:22.052043 16201 raft_consensus.cc:359] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
W20260709 13:16:22.052942 15972 leader_election.cc:343] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.056452 16201 raft_consensus.cc:385] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.056741 16201 raft_consensus.cc:740] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.057569 16204 raft_consensus.cc:359] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.057456 16201 consensus_queue.cc:260] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.058198 16204 raft_consensus.cc:385] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.061731 16202 tablet_bootstrap.cc:654] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.060719 15831 catalog_manager.cc:5697] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:22.061875 16209 consensus_queue.cc:1048] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:22.058517 16204 raft_consensus.cc:740] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.062966 16204 consensus_queue.cc:260] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.065569 16209 raft_consensus.cc:493] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.066121 16209 raft_consensus.cc:515] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.067409 16204 ts_tablet_manager.cc:1434] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.000s
I20260709 13:16:22.068780 16204 tablet_bootstrap.cc:492] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:22.069252 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718e00ba6fa34542a71fc6d5758e92a1" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
W20260709 13:16:22.070569 15971 leader_election.cc:343] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.080065 16201 ts_tablet_manager.cc:1434] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.001s
I20260709 13:16:22.079473 16298 raft_consensus.cc:697] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:22.088302 16298 consensus_queue.cc:237] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.092373 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718e00ba6fa34542a71fc6d5758e92a1" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:22.093050 16155 raft_consensus.cc:2468] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:22.091187 16154 raft_consensus.cc:2468] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:22.096475 16302 consensus_queue.cc:1048] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:22.096652 15974 leader_election.cc:304] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:22.097330 16204 tablet_bootstrap.cc:654] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.097568 15974 leader_election.cc:304] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:22.097817 16302 raft_consensus.cc:2804] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:22.098196 16302 raft_consensus.cc:493] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.098379 16248 raft_consensus.cc:2804] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:22.098649 16302 raft_consensus.cc:3060] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.098883 16248 raft_consensus.cc:697] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:22.099879 16248 consensus_queue.cc:237] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.080469 16209 leader_election.cc:290] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:22.112329 16201 tablet_bootstrap.cc:492] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:22.126847 16202 tablet_bootstrap.cc:492] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:22.126277 15831 catalog_manager.cc:5697] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:22.127322 16202 ts_tablet_manager.cc:1403] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.086s	user 0.017s	sys 0.020s
I20260709 13:16:22.129351 16237 raft_consensus.cc:493] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.129858 16237 raft_consensus.cc:515] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.132277 16237 leader_election.cc:290] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:22.133934 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a39139d4f2b48d09ba9eb320dfcf265" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:22.134748 16155 raft_consensus.cc:2468] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:22.133839 16202 raft_consensus.cc:359] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.135686 16202 raft_consensus.cc:385] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.135936 16202 raft_consensus.cc:740] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.136562 16202 consensus_queue.cc:260] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.138592 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a39139d4f2b48d09ba9eb320dfcf265" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:22.140462 16202 ts_tablet_manager.cc:1434] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.003s
I20260709 13:16:22.140268 15899 leader_election.cc:304] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:22.143599 16203 tablet_bootstrap.cc:492] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:22.144217 16203 ts_tablet_manager.cc:1403] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.147s	user 0.023s	sys 0.033s
I20260709 13:16:22.145526 16293 raft_consensus.cc:493] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.155169 16202 tablet_bootstrap.cc:492] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:22.154450 16237 raft_consensus.cc:2804] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:22.146039 16293 raft_consensus.cc:515] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.156090 16237 raft_consensus.cc:493] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.156476 16237 raft_consensus.cc:3060] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.159843 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633df69636042288f27156e6084dce9" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:22.159967 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633df69636042288f27156e6084dce9" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:22.161670 16121 leader_election.cc:343] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.165019 16302 raft_consensus.cc:515] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
W20260709 13:16:22.146999 15897 leader_election.cc:343] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.173102 16237 raft_consensus.cc:515] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.169946 16203 raft_consensus.cc:359] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.175709 16204 tablet_bootstrap.cc:492] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:22.175928 16203 raft_consensus.cc:385] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.176265 16204 ts_tablet_manager.cc:1403] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.108s	user 0.019s	sys 0.026s
I20260709 13:16:22.171654 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718e00ba6fa34542a71fc6d5758e92a1" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:22.176882 16155 raft_consensus.cc:3060] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.158149 16293 leader_election.cc:290] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:22.178589 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a39139d4f2b48d09ba9eb320dfcf265" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:22.179159 16204 raft_consensus.cc:359] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.179659 16237 leader_election.cc:290] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:22.179844 16204 raft_consensus.cc:385] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.180136 16204 raft_consensus.cc:740] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.180751 16204 consensus_queue.cc:260] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.169580 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "718e00ba6fa34542a71fc6d5758e92a1" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:22.185491 15932 raft_consensus.cc:3060] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.185611 16298 raft_consensus.cc:493] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.186129 16204 ts_tablet_manager.cc:1434] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:16:22.186170 16298 raft_consensus.cc:515] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.187178 16204 tablet_bootstrap.cc:492] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:22.188598 16298 leader_election.cc:290] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:22.191028 16202 tablet_bootstrap.cc:654] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.190877 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7454f1cf21c0400c8a3a8290d5d148dd" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:22.174873 16119 leader_election.cc:343] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.175283 16201 tablet_bootstrap.cc:654] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.194840 16119 leader_election.cc:304] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:22.176232 16203 raft_consensus.cc:740] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.170607 16302 leader_election.cc:290] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:22.195886 16293 raft_consensus.cc:2749] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:22.198022 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a39139d4f2b48d09ba9eb320dfcf265" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:22.198498 16154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7454f1cf21c0400c8a3a8290d5d148dd" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:22.198688 16156 raft_consensus.cc:3060] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.198321 16203 consensus_queue.cc:260] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.199258 16082 raft_consensus.cc:3060] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.199268 16154 raft_consensus.cc:2393] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4d6e13a8f4a473098351ada96fd7dab in current term 1: Already voted for candidate 903b3435239241eab01ea54ffb015ee3 in this term.
I20260709 13:16:22.202827 16204 tablet_bootstrap.cc:654] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.204435 16048 leader_election.cc:304] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:22.206159 16298 raft_consensus.cc:3060] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.209666 16082 raft_consensus.cc:2468] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:22.198082 15932 raft_consensus.cc:2468] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:22.222965 16293 consensus_queue.cc:1048] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:22.225620 15971 leader_election.cc:304] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:22.228619 16203 ts_tablet_manager.cc:1434] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.000s
I20260709 13:16:22.230281 15897 leader_election.cc:304] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:22.231393 16237 raft_consensus.cc:2804] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:22.232235 16237 raft_consensus.cc:697] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:22.233373 16237 consensus_queue.cc:237] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.241060 15932 raft_consensus.cc:3060] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.246701 16203 tablet_bootstrap.cc:492] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:22.249441 16248 raft_consensus.cc:2804] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:22.250175 16248 raft_consensus.cc:697] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:22.251251 16248 consensus_queue.cc:237] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.253885 16201 tablet_bootstrap.cc:492] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:22.254523 16201 ts_tablet_manager.cc:1403] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.148s	user 0.013s	sys 0.001s
I20260709 13:16:22.261165 16209 consensus_queue.cc:1048] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:22.279759 16251 consensus_queue.cc:1048] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:22.284634 16155 raft_consensus.cc:2468] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:22.291651 15831 catalog_manager.cc:5697] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:22.309219 16203 tablet_bootstrap.cc:654] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.294595 16156 raft_consensus.cc:2468] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:22.296307 16201 raft_consensus.cc:359] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.299479 16320 consensus_queue.cc:1048] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:22.310039 15827 catalog_manager.cc:5697] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:22.310966 16201 raft_consensus.cc:385] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.311452 16201 raft_consensus.cc:740] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.313921 16302 consensus_queue.cc:1048] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:22.318663 16293 consensus_queue.cc:1048] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:22.318867 16201 consensus_queue.cc:260] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.323344 16203 tablet_bootstrap.cc:492] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:22.332156 16201 ts_tablet_manager.cc:1434] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.077s	user 0.005s	sys 0.002s
I20260709 13:16:22.332443 16320 consensus_queue.cc:1048] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:22.333348 16201 tablet_bootstrap.cc:492] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:22.347869 16203 ts_tablet_manager.cc:1403] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.118s	user 0.016s	sys 0.004s
I20260709 13:16:22.302599 16298 raft_consensus.cc:2749] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:22.364163 16204 tablet_bootstrap.cc:492] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:22.364913 16204 ts_tablet_manager.cc:1403] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.178s	user 0.073s	sys 0.043s
I20260709 13:16:22.370461 16203 raft_consensus.cc:359] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.371565 16203 raft_consensus.cc:385] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.371920 16203 raft_consensus.cc:740] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.388979 16201 tablet_bootstrap.cc:654] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.388830 16203 consensus_queue.cc:260] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.390787 16202 tablet_bootstrap.cc:492] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:22.391328 16202 ts_tablet_manager.cc:1403] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.238s	user 0.055s	sys 0.107s
I20260709 13:16:22.377763 16248 consensus_queue.cc:1048] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:16:22.402258 16300 consensus_queue.cc:1048] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:22.403993 16204 raft_consensus.cc:359] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.404942 16204 raft_consensus.cc:385] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.405284 16204 raft_consensus.cc:740] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.406150 16204 consensus_queue.cc:260] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.406502 15831 catalog_manager.cc:5697] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:22.422598 16203 ts_tablet_manager.cc:1434] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.074s	user 0.008s	sys 0.001s
I20260709 13:16:22.424685 16204 ts_tablet_manager.cc:1434] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.059s	user 0.010s	sys 0.000s
I20260709 13:16:22.425581 16203 tablet_bootstrap.cc:492] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:22.441257 16204 tablet_bootstrap.cc:492] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:22.445837 16202 raft_consensus.cc:359] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.446733 16202 raft_consensus.cc:385] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.447120 16203 tablet_bootstrap.cc:654] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.447074 16202 raft_consensus.cc:740] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.448161 16202 consensus_queue.cc:260] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.450454 16201 tablet_bootstrap.cc:492] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:22.450946 16201 ts_tablet_manager.cc:1403] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.118s	user 0.031s	sys 0.024s
I20260709 13:16:22.453583 16201 raft_consensus.cc:359] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.454224 16201 raft_consensus.cc:385] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.454537 16201 raft_consensus.cc:740] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.455263 16201 consensus_queue.cc:260] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.456193 16204 tablet_bootstrap.cc:654] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.457392 16201 ts_tablet_manager.cc:1434] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:16:22.458503 16201 tablet_bootstrap.cc:492] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:22.465101 16275 raft_consensus.cc:493] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.465328 16201 tablet_bootstrap.cc:654] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.465619 16275 raft_consensus.cc:515] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.468787 16275 leader_election.cc:290] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:22.469920 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a3e44fcd5b438486e1ec0241ef1e8d" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:22.471175 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a3e44fcd5b438486e1ec0241ef1e8d" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:22.471963 16156 raft_consensus.cc:2393] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4d6e13a8f4a473098351ada96fd7dab in current term 1: Already voted for candidate 903b3435239241eab01ea54ffb015ee3 in this term.
I20260709 13:16:22.476317 16047 leader_election.cc:304] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:22.477809 16275 raft_consensus.cc:3060] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.481173 16201 tablet_bootstrap.cc:492] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:22.481678 16201 ts_tablet_manager.cc:1403] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.023s	user 0.002s	sys 0.019s
I20260709 13:16:22.482592 16300 consensus_queue.cc:1048] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:22.484519 16293 raft_consensus.cc:493] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.485016 16293 raft_consensus.cc:515] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.487061 16293 leader_election.cc:290] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:22.489545 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034918f94907492ca9a964c053edd50e" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
W20260709 13:16:22.491019 16119 leader_election.cc:343] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.492187 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034918f94907492ca9a964c053edd50e" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:22.492663 16201 raft_consensus.cc:359] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.493328 16201 raft_consensus.cc:385] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:16:22.493578 16120 leader_election.cc:343] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.493597 16201 raft_consensus.cc:740] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.494185 16120 leader_election.cc:304] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:22.494568 16201 consensus_queue.cc:260] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.495261 16344 raft_consensus.cc:2749] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:22.496680 16201 ts_tablet_manager.cc:1434] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.015s	user 0.000s	sys 0.008s
I20260709 13:16:22.497704 16201 tablet_bootstrap.cc:492] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:22.503954 16201 tablet_bootstrap.cc:654] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.504240 16275 raft_consensus.cc:2749] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:22.525434 16203 tablet_bootstrap.cc:492] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:22.526244 16203 ts_tablet_manager.cc:1403] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.101s	user 0.012s	sys 0.007s
I20260709 13:16:22.529903 16203 raft_consensus.cc:359] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.533118 16203 raft_consensus.cc:385] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.533382 16204 tablet_bootstrap.cc:492] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:22.533437 16203 raft_consensus.cc:740] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.534060 16204 ts_tablet_manager.cc:1403] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.103s	user 0.023s	sys 0.028s
I20260709 13:16:22.534278 16201 tablet_bootstrap.cc:492] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:22.534756 16201 ts_tablet_manager.cc:1403] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.037s	user 0.021s	sys 0.012s
I20260709 13:16:22.534713 16203 consensus_queue.cc:260] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.537497 16201 raft_consensus.cc:359] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.538168 16201 raft_consensus.cc:385] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.537930 16204 raft_consensus.cc:359] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.538504 16201 raft_consensus.cc:740] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.538676 16204 raft_consensus.cc:385] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.539019 16204 raft_consensus.cc:740] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.539354 16201 consensus_queue.cc:260] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.539896 16204 consensus_queue.cc:260] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.543064 16202 ts_tablet_manager.cc:1434] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.136s	user 0.006s	sys 0.001s
I20260709 13:16:22.543363 16275 consensus_queue.cc:1048] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:22.544541 16201 ts_tablet_manager.cc:1434] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:16:22.545603 16201 tablet_bootstrap.cc:492] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:22.547250 16204 ts_tablet_manager.cc:1434] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:16:22.548421 16204 tablet_bootstrap.cc:492] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:22.548506 16203 ts_tablet_manager.cc:1434] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:16:22.549837 16203 tablet_bootstrap.cc:492] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:22.551893 16201 tablet_bootstrap.cc:654] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.555222 16204 tablet_bootstrap.cc:654] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.557964 16202 tablet_bootstrap.cc:492] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:22.557215 16203 tablet_bootstrap.cc:654] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.566378 16204 tablet_bootstrap.cc:492] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:22.566830 16204 ts_tablet_manager.cc:1403] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.011s
I20260709 13:16:22.569541 16204 raft_consensus.cc:359] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.570264 16204 raft_consensus.cc:385] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.570580 16204 raft_consensus.cc:740] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.571399 16204 consensus_queue.cc:260] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.572615 16201 tablet_bootstrap.cc:492] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:22.573083 16201 ts_tablet_manager.cc:1403] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.015s
I20260709 13:16:22.573652 16204 ts_tablet_manager.cc:1434] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:22.573962 16203 tablet_bootstrap.cc:492] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:22.574492 16203 ts_tablet_manager.cc:1403] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.006s
I20260709 13:16:22.574762 16204 tablet_bootstrap.cc:492] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:22.575834 16201 raft_consensus.cc:359] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.576485 16201 raft_consensus.cc:385] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.576750 16201 raft_consensus.cc:740] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.577438 16203 raft_consensus.cc:359] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.577531 16201 consensus_queue.cc:260] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.578296 16203 raft_consensus.cc:385] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.580574 16203 raft_consensus.cc:740] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.579717 16201 ts_tablet_manager.cc:1434] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:22.581423 16203 consensus_queue.cc:260] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.582404 16201 tablet_bootstrap.cc:492] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:22.591310 16203 ts_tablet_manager.cc:1434] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:16:22.592386 16201 tablet_bootstrap.cc:654] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.592437 16203 tablet_bootstrap.cc:492] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:22.593870 16204 tablet_bootstrap.cc:654] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.594864 16320 consensus_queue.cc:1048] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:22.600507 16203 tablet_bootstrap.cc:654] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.604187 16230 raft_consensus.cc:493] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.604718 16230 raft_consensus.cc:515] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.607472 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f46c65175b29436c8f0704ffa98d4eb7" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
W20260709 13:16:22.608924 16119 leader_election.cc:343] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.609846 16202 tablet_bootstrap.cc:654] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.610121 16230 leader_election.cc:290] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:22.615967 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f46c65175b29436c8f0704ffa98d4eb7" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:22.618278 16008 raft_consensus.cc:2468] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 0.
I20260709 13:16:22.620002 16121 leader_election.cc:304] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:22.621688 16251 raft_consensus.cc:493] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.622315 16251 raft_consensus.cc:515] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.625789 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ae89d92f6942d196dcc8d146b8f459" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:22.626318 16251 leader_election.cc:290] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:22.626812 16230 raft_consensus.cc:2804] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:22.627058 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ae89d92f6942d196dcc8d146b8f459" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:22.627364 16230 raft_consensus.cc:493] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.627741 16230 raft_consensus.cc:3060] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.620015 16248 raft_consensus.cc:493] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.628423 16248 raft_consensus.cc:515] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.630570 16248 leader_election.cc:290] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:22.631695 16202 tablet_bootstrap.cc:492] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:22.632143 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38e5c73946b4d14825079217a21a21a" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:22.632803 15932 raft_consensus.cc:2468] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:22.634065 15971 leader_election.cc:304] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
W20260709 13:16:22.634124 15974 leader_election.cc:343] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.635200 16248 raft_consensus.cc:2804] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:22.635540 16204 tablet_bootstrap.cc:492] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:22.635607 16248 raft_consensus.cc:493] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.636090 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38e5c73946b4d14825079217a21a21a" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:22.636215 16204 ts_tablet_manager.cc:1403] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.062s	user 0.025s	sys 0.017s
I20260709 13:16:22.636230 16248 raft_consensus.cc:3060] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:22.635421 15972 leader_election.cc:343] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.639195 15972 leader_election.cc:304] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:22.638859 16230 raft_consensus.cc:515] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.639961 16251 raft_consensus.cc:2749] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:22.640393 15972 leader_election.cc:343] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.632175 16202 ts_tablet_manager.cc:1403] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.074s	user 0.012s	sys 0.001s
I20260709 13:16:22.643193 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f46c65175b29436c8f0704ffa98d4eb7" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
W20260709 13:16:22.644574 16119 leader_election.cc:343] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.644527 16248 raft_consensus.cc:515] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.647212 16203 tablet_bootstrap.cc:492] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:22.647069 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f46c65175b29436c8f0704ffa98d4eb7" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:22.647555 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38e5c73946b4d14825079217a21a21a" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:22.647687 16203 ts_tablet_manager.cc:1403] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.055s	user 0.024s	sys 0.000s
I20260709 13:16:22.647742 16008 raft_consensus.cc:3060] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.648125 15933 raft_consensus.cc:3060] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.644670 16202 raft_consensus.cc:359] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.651501 16202 raft_consensus.cc:385] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.651779 16202 raft_consensus.cc:740] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.652349 16302 raft_consensus.cc:493] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.652451 16202 consensus_queue.cc:260] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.652873 16302 raft_consensus.cc:515] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.654897 16302 leader_election.cc:290] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:22.655140 16320 raft_consensus.cc:493] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.655593 16320 raft_consensus.cc:515] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.656105 16204 raft_consensus.cc:359] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.656477 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6319ed0d71e4b4e9608ad673766fb5e" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:22.656781 16204 raft_consensus.cc:385] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.657106 16204 raft_consensus.cc:740] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.657141 15932 raft_consensus.cc:2468] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:22.657644 16320 leader_election.cc:290] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:22.657950 16204 consensus_queue.cc:260] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.658501 15971 leader_election.cc:304] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:22.659906 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034918f94907492ca9a964c053edd50e" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:22.660545 16156 raft_consensus.cc:2468] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:22.661175 16202 ts_tablet_manager.cc:1434] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:16:22.661485 16203 raft_consensus.cc:359] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.661778 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034918f94907492ca9a964c053edd50e" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:22.662113 16203 raft_consensus.cc:385] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.662427 15932 raft_consensus.cc:2468] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:22.663405 16302 raft_consensus.cc:2804] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:22.663856 16302 raft_consensus.cc:493] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.664223 16302 raft_consensus.cc:3060] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.664832 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6319ed0d71e4b4e9608ad673766fb5e" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:22.665433 16082 raft_consensus.cc:2468] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:22.665894 16045 leader_election.cc:304] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:22.659929 16204 ts_tablet_manager.cc:1434] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:16:22.668457 16204 tablet_bootstrap.cc:492] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:22.669245 16320 raft_consensus.cc:2804] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:22.669736 16320 raft_consensus.cc:493] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.670213 16320 raft_consensus.cc:3060] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.670984 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38e5c73946b4d14825079217a21a21a" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:22.674057 16204 tablet_bootstrap.cc:654] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.674427 16248 leader_election.cc:290] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:22.676092 16008 raft_consensus.cc:2468] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 1.
I20260709 13:16:22.663524 16202 tablet_bootstrap.cc:492] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:22.678908 15933 raft_consensus.cc:2468] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:22.666570 16209 raft_consensus.cc:493] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.682631 16209 raft_consensus.cc:515] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.685034 16209 leader_election.cc:290] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:22.689306 16202 tablet_bootstrap.cc:654] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:22.692127 15972 leader_election.cc:343] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.690400 16121 leader_election.cc:304] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:22.693325 16344 raft_consensus.cc:2804] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:22.693956 16344 raft_consensus.cc:697] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
I20260709 13:16:22.695024 16344 consensus_queue.cc:237] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.696125 16230 leader_election.cc:290] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:22.702586 16302 raft_consensus.cc:515] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.703856 16201 tablet_bootstrap.cc:492] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:22.705835 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6319ed0d71e4b4e9608ad673766fb5e" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:22.706573 16080 raft_consensus.cc:3060] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.706760 16201 ts_tablet_manager.cc:1403] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.125s	user 0.010s	sys 0.004s
I20260709 13:16:22.662444 16203 raft_consensus.cc:740] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.712414 16203 consensus_queue.cc:260] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.716181 16203 ts_tablet_manager.cc:1434] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 13:16:22.719135 16203 tablet_bootstrap.cc:492] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:22.705856 16320 raft_consensus.cc:515] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.719726 16202 tablet_bootstrap.cc:492] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:22.722735 16202 ts_tablet_manager.cc:1403] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.059s	user 0.007s	sys 0.005s
I20260709 13:16:22.723706 16237 consensus_queue.cc:1048] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:22.727427 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034918f94907492ca9a964c053edd50e" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:22.727319 16202 raft_consensus.cc:359] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.732764 16202 raft_consensus.cc:385] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.733933 16237 raft_consensus.cc:493] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.734939 16202 raft_consensus.cc:740] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.735091 15831 catalog_manager.cc:5697] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 1 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:22.746654 16080 raft_consensus.cc:2468] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:22.750138 16202 consensus_queue.cc:260] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.732185 16156 raft_consensus.cc:3060] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.731274 16320 leader_election.cc:290] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:22.760471 16202 ts_tablet_manager.cc:1434] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:16:22.744071 16302 leader_election.cc:290] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:22.752698 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034918f94907492ca9a964c053edd50e" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:22.764114 16202 tablet_bootstrap.cc:492] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:22.764963 16154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633df69636042288f27156e6084dce9" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:22.728220 16201 raft_consensus.cc:359] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.777841 16201 raft_consensus.cc:385] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.778275 16201 raft_consensus.cc:740] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.781945 16201 consensus_queue.cc:260] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.750104 16237 raft_consensus.cc:515] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.789373 15972 leader_election.cc:304] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:22.780752 16154 raft_consensus.cc:2468] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:22.779340 15932 raft_consensus.cc:3060] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.797578 16203 tablet_bootstrap.cc:654] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.799957 16201 ts_tablet_manager.cc:1434] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.091s	user 0.010s	sys 0.000s
I20260709 13:16:22.800446 16370 consensus_queue.cc:1048] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.051s
I20260709 13:16:22.801293 16201 tablet_bootstrap.cc:492] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:22.801586 16237 leader_election.cc:290] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:22.803498 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633df69636042288f27156e6084dce9" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:22.804278 16008 raft_consensus.cc:2468] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:22.805612 16237 consensus_queue.cc:1048] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:22.806763 16154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633df69636042288f27156e6084dce9" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:22.807420 16154 raft_consensus.cc:2468] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:22.808990 15898 leader_election.cc:304] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:22.810199 16201 tablet_bootstrap.cc:654] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.817281 16237 raft_consensus.cc:2804] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:22.820907 16237 raft_consensus.cc:493] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.821286 16237 raft_consensus.cc:3060] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.822690 16302 raft_consensus.cc:2804] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:22.823252 16302 raft_consensus.cc:697] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:22.823752 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633df69636042288f27156e6084dce9" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:22.824138 16302 consensus_queue.cc:237] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.826753 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6319ed0d71e4b4e9608ad673766fb5e" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:22.827417 15974 leader_election.cc:304] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:22.827411 15930 raft_consensus.cc:3060] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.828611 15971 leader_election.cc:304] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:22.838510 16237 raft_consensus.cc:515] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.841784 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633df69636042288f27156e6084dce9" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:22.843425 16156 raft_consensus.cc:2468] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:22.843536 16153 raft_consensus.cc:3060] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.843950 15931 raft_consensus.cc:2393] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 903b3435239241eab01ea54ffb015ee3 in current term 1: Already voted for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in this term.
I20260709 13:16:22.845847 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633df69636042288f27156e6084dce9" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:22.846532 16008 raft_consensus.cc:3060] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.849062 15932 raft_consensus.cc:2468] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:22.850101 16048 leader_election.cc:304] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:22.851755 16204 tablet_bootstrap.cc:492] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:22.852258 16204 ts_tablet_manager.cc:1403] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.184s	user 0.033s	sys 0.069s
I20260709 13:16:22.852028 16230 raft_consensus.cc:493] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.855728 16202 tablet_bootstrap.cc:654] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:22.851119 16237 leader_election.cc:290] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:22.856534 16248 raft_consensus.cc:2804] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:22.852743 16251 consensus_queue.cc:1048] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:16:22.865839 16230 raft_consensus.cc:515] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.854463 16320 raft_consensus.cc:2804] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:22.875018 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf30209cb5544714bc183c72359c9656" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:22.875204 16320 raft_consensus.cc:697] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:22.876085 16320 consensus_queue.cc:237] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.878814 16230 leader_election.cc:290] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:22.882123 16120 leader_election.cc:343] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:22.896049 16204 raft_consensus.cc:359] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.896816 15930 raft_consensus.cc:2468] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:22.896811 16204 raft_consensus.cc:385] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.897948 16204 raft_consensus.cc:740] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.898624 16204 consensus_queue.cc:260] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:22.900604 16204 ts_tablet_manager.cc:1434] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 13:16:22.901242 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf30209cb5544714bc183c72359c9656" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:22.901522 16204 tablet_bootstrap.cc:492] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:22.901881 15932 raft_consensus.cc:2468] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 0.
I20260709 13:16:22.903827 16008 raft_consensus.cc:2468] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:22.904637 16362 raft_consensus.cc:2764] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:16:22.919771 16201 tablet_bootstrap.cc:492] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:22.920460 16201 ts_tablet_manager.cc:1403] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.119s	user 0.013s	sys 0.005s
I20260709 13:16:22.907179 16248 raft_consensus.cc:697] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:22.921602 16119 leader_election.cc:304] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:22.921878 15898 leader_election.cc:304] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:22.922174 16248 consensus_queue.cc:237] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:22.922786 16237 raft_consensus.cc:2804] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:22.923334 16237 raft_consensus.cc:697] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:22.924173 16237 consensus_queue.cc:237] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:22.941124 16153 raft_consensus.cc:2468] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:22.926903 16230 raft_consensus.cc:2804] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:22.946223 16230 raft_consensus.cc:493] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.946908 16230 raft_consensus.cc:3060] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:22.950485 16201 raft_consensus.cc:359] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.956346 16201 raft_consensus.cc:385] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:22.957015 16201 raft_consensus.cc:740] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:22.961755 16201 consensus_queue.cc:260] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:22.971536 16374 consensus_queue.cc:1048] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:22.990414 16201 ts_tablet_manager.cc:1434] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.070s	user 0.011s	sys 0.000s
I20260709 13:16:22.995836 16201 tablet_bootstrap.cc:492] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:22.997699 16203 tablet_bootstrap.cc:492] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:22.998164 16203 ts_tablet_manager.cc:1403] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.279s	user 0.019s	sys 0.001s
I20260709 13:16:23.003167 16368 consensus_queue.cc:1048] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:16:23.011796 15879 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:23.017942 16203 raft_consensus.cc:359] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:23.020434 16203 raft_consensus.cc:385] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.020670 16203 raft_consensus.cc:740] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.009857 15830 catalog_manager.cc:5697] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:23.024150 16203 consensus_queue.cc:260] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:23.027416 16202 tablet_bootstrap.cc:492] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:23.027907 16202 ts_tablet_manager.cc:1403] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.265s	user 0.036s	sys 0.000s
I20260709 13:16:23.044380 16203 ts_tablet_manager.cc:1434] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.000s
I20260709 13:16:23.045553 16203 tablet_bootstrap.cc:492] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:23.045691 15827 catalog_manager.cc:5697] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:23.052644 16203 tablet_bootstrap.cc:654] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.053144 16248 raft_consensus.cc:493] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.053814 16248 raft_consensus.cc:515] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:23.059700 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5922376de62747aabde22ad134c86f36" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:23.066278 16201 tablet_bootstrap.cc:654] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.067251 16202 raft_consensus.cc:359] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:23.070118 16202 raft_consensus.cc:385] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.051294 15831 catalog_manager.cc:5697] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:23.070529 16202 raft_consensus.cc:740] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.072055 16202 consensus_queue.cc:260] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:23.075233 16202 ts_tablet_manager.cc:1434] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.047s	user 0.008s	sys 0.000s
W20260709 13:16:23.067387 15971 leader_election.cc:343] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.067924 16248 leader_election.cc:290] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:23.080286 16204 tablet_bootstrap.cc:654] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.080567 16202 tablet_bootstrap.cc:492] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:23.081240 16081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5922376de62747aabde22ad134c86f36" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
W20260709 13:16:23.082706 15972 leader_election.cc:343] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.083122 15972 leader_election.cc:304] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:23.083913 16248 raft_consensus.cc:2749] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:23.091706 16202 tablet_bootstrap.cc:654] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.075134 16230 raft_consensus.cc:515] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:23.106792 16300 raft_consensus.cc:493] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.107525 16300 raft_consensus.cc:515] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:23.108232 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf30209cb5544714bc183c72359c9656" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:23.108867 15930 raft_consensus.cc:3060] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.115111 16300 leader_election.cc:290] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:23.115759 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf30209cb5544714bc183c72359c9656" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:23.116554 16155 raft_consensus.cc:2393] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4d6e13a8f4a473098351ada96fd7dab in current term 1: Already voted for candidate c95f0816c41142699d80a2e326e17ab8 in this term.
I20260709 13:16:23.117334 16081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf30209cb5544714bc183c72359c9656" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:23.117782 15930 raft_consensus.cc:2468] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 1.
I20260709 13:16:23.117903 16081 raft_consensus.cc:3060] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.119102 16119 leader_election.cc:304] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:23.119691 16344 raft_consensus.cc:493] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.119928 16293 raft_consensus.cc:2804] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:23.120190 16344 raft_consensus.cc:515] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.122406 16344 leader_election.cc:290] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:23.123687 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ae89d92f6942d196dcc8d146b8f459" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:23.123881 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ae89d92f6942d196dcc8d146b8f459" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:23.124470 16008 raft_consensus.cc:2468] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 0.
I20260709 13:16:23.124533 16082 raft_consensus.cc:2468] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 0.
I20260709 13:16:23.125866 16120 leader_election.cc:304] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:23.126657 16203 tablet_bootstrap.cc:492] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:23.127166 16203 ts_tablet_manager.cc:1403] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.000s
I20260709 13:16:23.127230 16344 raft_consensus.cc:2804] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:23.127678 16344 raft_consensus.cc:493] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.127988 16344 raft_consensus.cc:3060] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.135070 16203 raft_consensus.cc:359] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:23.137032 16203 raft_consensus.cc:385] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.137375 16203 raft_consensus.cc:740] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.146636 16203 consensus_queue.cc:260] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:23.141723 16230 leader_election.cc:290] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:23.153329 16203 ts_tablet_manager.cc:1434] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:16:23.154523 16203 tablet_bootstrap.cc:492] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
W20260709 13:16:23.164665 16119 consensus_peers.cc:597] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:23.201622 16293 raft_consensus.cc:697] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
I20260709 13:16:23.216419 16204 tablet_bootstrap.cc:492] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:23.219002 16230 raft_consensus.cc:493] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.219234 16204 ts_tablet_manager.cc:1403] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.318s	user 0.013s	sys 0.028s
I20260709 13:16:23.219440 16293 consensus_queue.cc:237] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:23.219550 16230 raft_consensus.cc:515] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:23.222054 16204 raft_consensus.cc:359] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:23.222754 16204 raft_consensus.cc:385] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.222836 16344 raft_consensus.cc:515] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.223181 16204 raft_consensus.cc:740] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.224627 16230 leader_election.cc:290] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:23.225767 16344 leader_election.cc:290] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:23.218515 16081 raft_consensus.cc:2468] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 1.
I20260709 13:16:23.225113 16204 consensus_queue.cc:260] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:23.232240 16202 tablet_bootstrap.cc:492] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:23.232472 16201 tablet_bootstrap.cc:492] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:23.248039 16202 ts_tablet_manager.cc:1403] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.168s	user 0.008s	sys 0.013s
I20260709 13:16:23.245926 16081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ae89d92f6942d196dcc8d146b8f459" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:23.249185 16204 ts_tablet_manager.cc:1434] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 13:16:23.249171 16081 raft_consensus.cc:3060] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.250665 16204 tablet_bootstrap.cc:492] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:23.251001 16202 raft_consensus.cc:359] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.251662 16202 raft_consensus.cc:385] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.251947 16202 raft_consensus.cc:740] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.252702 16202 consensus_queue.cc:260] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.238312 16203 tablet_bootstrap.cc:654] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.257328 16081 raft_consensus.cc:2468] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 1.
I20260709 13:16:23.257504 16202 ts_tablet_manager.cc:1434] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 13:16:23.257797 16204 tablet_bootstrap.cc:654] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.258680 16202 tablet_bootstrap.cc:492] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:23.258816 16120 leader_election.cc:304] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:23.259783 16293 raft_consensus.cc:2804] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:23.260269 16293 raft_consensus.cc:697] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
I20260709 13:16:23.261087 16293 consensus_queue.cc:237] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.248063 16201 ts_tablet_manager.cc:1403] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.253s	user 0.015s	sys 0.023s
I20260709 13:16:23.266976 16202 tablet_bootstrap.cc:654] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.266571 16201 raft_consensus.cc:359] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.273317 16201 raft_consensus.cc:385] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.273667 16201 raft_consensus.cc:740] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.274426 16201 consensus_queue.cc:260] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.293883 15830 catalog_manager.cc:5697] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:23.458144 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f15b348b66b4bffae9c7f9eed8207e2" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:23.458395 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07ae89d92f6942d196dcc8d146b8f459" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:23.459048 16007 raft_consensus.cc:3060] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:23.459614 16121 leader_election.cc:343] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.462889 16293 consensus_queue.cc:1048] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:23.466431 16007 raft_consensus.cc:2468] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 1.
I20260709 13:16:23.480307 16201 ts_tablet_manager.cc:1434] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.216s	user 0.008s	sys 0.002s
I20260709 13:16:23.481406 16201 tablet_bootstrap.cc:492] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:23.487112 16401 consensus_queue.cc:1048] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:23.489874 16202 tablet_bootstrap.cc:492] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:23.490420 16202 ts_tablet_manager.cc:1403] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.232s	user 0.014s	sys 0.003s
I20260709 13:16:23.494527 16202 raft_consensus.cc:359] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:23.495410 16202 raft_consensus.cc:385] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.508507 16300 raft_consensus.cc:493] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.508790 16202 raft_consensus.cc:740] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.509035 16201 tablet_bootstrap.cc:654] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.509830 16202 consensus_queue.cc:260] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:23.512632 16202 ts_tablet_manager.cc:1434] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.004s
I20260709 13:16:23.513893 16202 tablet_bootstrap.cc:492] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:23.501350 16401 raft_consensus.cc:493] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.506923 16394 raft_consensus.cc:493] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.514153 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf30209cb5544714bc183c72359c9656" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:23.515671 15930 raft_consensus.cc:2393] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4d6e13a8f4a473098351ada96fd7dab in current term 1: Already voted for candidate c95f0816c41142699d80a2e326e17ab8 in this term.
I20260709 13:16:23.516122 16401 raft_consensus.cc:515] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:23.518051 16045 leader_election.cc:304] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:23.520817 16320 raft_consensus.cc:2749] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:23.522217 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f46c65175b29436c8f0704ffa98d4eb7" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:23.525110 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f15b348b66b4bffae9c7f9eed8207e2" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:23.526567 16401 leader_election.cc:290] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:23.509035 16300 raft_consensus.cc:515] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:23.527526 16401 consensus_queue.cc:1048] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:23.529415 16300 leader_election.cc:290] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:23.515115 16394 raft_consensus.cc:515] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.526659 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f46c65175b29436c8f0704ffa98d4eb7" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:23.537786 15898 leader_election.cc:304] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:23.539057 16081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebbb30a54a04a4aa567c6e16581ad1a" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:23.539920 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebbb30a54a04a4aa567c6e16581ad1a" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:23.540488 16120 leader_election.cc:343] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.540558 16007 raft_consensus.cc:2468] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 0.
I20260709 13:16:23.542490 16121 leader_election.cc:304] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:23.542948 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38e5c73946b4d14825079217a21a21a" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:23.543380 16230 raft_consensus.cc:2804] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:23.543768 15933 raft_consensus.cc:2393] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4d6e13a8f4a473098351ada96fd7dab in current term 1: Already voted for candidate 903b3435239241eab01ea54ffb015ee3 in this term.
I20260709 13:16:23.543812 16230 raft_consensus.cc:493] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.544378 16230 raft_consensus.cc:3060] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:23.547024 16119 leader_election.cc:343] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.547498 16119 leader_election.cc:304] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:23.548208 16293 raft_consensus.cc:2749] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:23.549190 16394 leader_election.cc:290] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:23.552615 16237 raft_consensus.cc:2749] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:23.556483 16302 consensus_queue.cc:1048] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:23.562077 16230 raft_consensus.cc:515] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.564359 16368 consensus_queue.cc:1048] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:23.565019 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebbb30a54a04a4aa567c6e16581ad1a" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:23.565733 16007 raft_consensus.cc:3060] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.572676 16007 raft_consensus.cc:2468] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 1.
I20260709 13:16:23.575189 16121 leader_election.cc:304] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:23.576045 16393 raft_consensus.cc:2804] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:23.576545 16393 raft_consensus.cc:697] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
I20260709 13:16:23.577361 16393 consensus_queue.cc:237] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.573999 16230 leader_election.cc:290] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:23.589031 16204 tablet_bootstrap.cc:492] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:23.589615 16204 ts_tablet_manager.cc:1403] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.339s	user 0.026s	sys 0.029s
I20260709 13:16:23.591104 16081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebbb30a54a04a4aa567c6e16581ad1a" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
W20260709 13:16:23.599782 16120 leader_election.cc:343] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.602972 16080 raft_consensus.cc:3060] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.608589 16298 consensus_queue.cc:1048] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:23.617415 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38e5c73946b4d14825079217a21a21a" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:23.618850 16047 leader_election.cc:304] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:23.622390 16275 raft_consensus.cc:2749] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:23.624110 16374 consensus_queue.cc:1048] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:23.625286 16209 consensus_queue.cc:1048] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:23.627745 16202 tablet_bootstrap.cc:654] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.638100 16368 raft_consensus.cc:493] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.638631 16368 raft_consensus.cc:515] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.640923 16368 leader_election.cc:290] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:23.642308 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaa175423efe491da6747a37df086375" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:23.642966 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaa175423efe491da6747a37df086375" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
W20260709 13:16:23.646044 15974 leader_election.cc:343] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.646663 16202 tablet_bootstrap.cc:492] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:23.647123 16202 ts_tablet_manager.cc:1403] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.134s	user 0.011s	sys 0.000s
I20260709 13:16:23.652974 16202 raft_consensus.cc:359] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.653644 16202 raft_consensus.cc:385] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.653910 16202 raft_consensus.cc:740] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.654024 16344 consensus_queue.cc:1048] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:23.660944 16202 consensus_queue.cc:260] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.662352 15931 raft_consensus.cc:3060] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.665004 16202 ts_tablet_manager.cc:1434] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
W20260709 13:16:23.664914 15972 leader_election.cc:343] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.665571 15972 leader_election.cc:304] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:23.666296 16362 raft_consensus.cc:2749] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:23.669569 16202 tablet_bootstrap.cc:492] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:23.680931 16202 tablet_bootstrap.cc:654] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.684799 16275 consensus_queue.cc:1048] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:23.689414 16204 raft_consensus.cc:359] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:23.703115 16344 consensus_queue.cc:1048] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:23.703583 16202 tablet_bootstrap.cc:492] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:23.721341 16202 ts_tablet_manager.cc:1403] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.052s	user 0.010s	sys 0.001s
I20260709 13:16:23.721952 16204 raft_consensus.cc:385] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.722329 16204 raft_consensus.cc:740] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.723559 16204 consensus_queue.cc:260] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:23.726927 16202 raft_consensus.cc:359] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:23.727792 16202 raft_consensus.cc:385] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.728099 16202 raft_consensus.cc:740] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.728915 16202 consensus_queue.cc:260] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:23.707508 16203 tablet_bootstrap.cc:492] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:23.730680 16203 ts_tablet_manager.cc:1403] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.576s	user 0.026s	sys 0.027s
I20260709 13:16:23.741032 16203 raft_consensus.cc:359] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.745726 16203 raft_consensus.cc:385] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.746062 16203 raft_consensus.cc:740] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.747581 16202 ts_tablet_manager.cc:1434] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:16:23.748741 16202 tablet_bootstrap.cc:492] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:23.751695 16203 consensus_queue.cc:260] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.752600 16298 raft_consensus.cc:493] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.753074 16298 raft_consensus.cc:515] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.754139 16203 ts_tablet_manager.cc:1434] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:16:23.755237 16203 tablet_bootstrap.cc:492] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:23.759539 16204 ts_tablet_manager.cc:1434] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.170s	user 0.008s	sys 0.000s
I20260709 13:16:23.764981 16204 tablet_bootstrap.cc:492] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:23.771464 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebbb30a54a04a4aa567c6e16581ad1a" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:23.772329 16008 raft_consensus.cc:2393] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4d6e13a8f4a473098351ada96fd7dab in current term 1: Already voted for candidate c95f0816c41142699d80a2e326e17ab8 in this term.
I20260709 13:16:23.776126 16204 tablet_bootstrap.cc:654] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.776310 16202 tablet_bootstrap.cc:654] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.785104 16298 leader_election.cc:290] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:23.789556 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eebbb30a54a04a4aa567c6e16581ad1a" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:23.792683 15831 catalog_manager.cc:5697] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 1 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:23.794785 16048 leader_election.cc:304] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:23.796622 16275 raft_consensus.cc:3060] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.799180 15831 catalog_manager.cc:5697] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 1 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:23.801115 15831 catalog_manager.cc:5697] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 1 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:23.805652 16203 tablet_bootstrap.cc:654] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.807879 16201 tablet_bootstrap.cc:492] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:23.808346 16201 ts_tablet_manager.cc:1403] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.327s	user 0.030s	sys 0.038s
I20260709 13:16:23.815119 16201 raft_consensus.cc:359] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.815753 16201 raft_consensus.cc:385] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.816011 16201 raft_consensus.cc:740] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.817114 16201 consensus_queue.cc:260] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:23.818557 16275 raft_consensus.cc:2749] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:23.820397 16201 ts_tablet_manager.cc:1434] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:16:23.826282 16201 tablet_bootstrap.cc:492] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:23.842042 16202 tablet_bootstrap.cc:492] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:23.842694 16202 ts_tablet_manager.cc:1403] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.094s	user 0.025s	sys 0.014s
I20260709 13:16:23.847458 16201 tablet_bootstrap.cc:654] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.849704 16202 raft_consensus.cc:359] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:23.850258 16202 raft_consensus.cc:385] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.850535 16202 raft_consensus.cc:740] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.851197 16202 consensus_queue.cc:260] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:23.854111 16202 ts_tablet_manager.cc:1434] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:16:23.855336 16202 tablet_bootstrap.cc:492] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:23.863651 16202 tablet_bootstrap.cc:654] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.869863 16203 tablet_bootstrap.cc:492] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:23.870409 16203 ts_tablet_manager.cc:1403] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.115s	user 0.014s	sys 0.013s
I20260709 13:16:23.874300 16203 raft_consensus.cc:359] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:23.889508 16203 raft_consensus.cc:385] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.889942 16203 raft_consensus.cc:740] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.890903 16203 consensus_queue.cc:260] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:23.891719 16293 consensus_queue.cc:1048] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:23.893965 16203 ts_tablet_manager.cc:1434] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.003s
I20260709 13:16:23.880563 16204 tablet_bootstrap.cc:492] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:23.902308 16204 ts_tablet_manager.cc:1403] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.140s	user 0.018s	sys 0.022s
I20260709 13:16:23.911842 16203 tablet_bootstrap.cc:492] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:23.911950 16202 tablet_bootstrap.cc:492] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:23.924430 16202 ts_tablet_manager.cc:1403] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.069s	user 0.015s	sys 0.034s
I20260709 13:16:23.913036 16201 tablet_bootstrap.cc:492] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:23.874446 16393 raft_consensus.cc:493] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.926674 16393 raft_consensus.cc:515] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:23.928902 16393 leader_election.cc:290] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:23.914996 16204 raft_consensus.cc:359] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:23.929441 16204 raft_consensus.cc:385] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.929718 16204 raft_consensus.cc:740] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.930060 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31217a26afbc45258ea518a1623dbb01" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:23.930699 16201 ts_tablet_manager.cc:1403] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.105s	user 0.038s	sys 0.021s
I20260709 13:16:23.930392 16204 consensus_queue.cc:260] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
W20260709 13:16:23.931628 16121 leader_election.cc:343] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.932863 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31217a26afbc45258ea518a1623dbb01" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:23.933058 16204 ts_tablet_manager.cc:1434] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
W20260709 13:16:23.934237 16119 leader_election.cc:343] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.934763 16119 leader_election.cc:304] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:23.936666 16393 raft_consensus.cc:2749] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:23.937753 16386 raft_consensus.cc:493] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.936617 16202 raft_consensus.cc:359] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:23.938263 16386 raft_consensus.cc:515] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:23.938927 16202 raft_consensus.cc:385] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.939393 16202 raft_consensus.cc:740] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.940148 16202 consensus_queue.cc:260] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:23.930357 16203 tablet_bootstrap.cc:654] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.942107 16371 raft_consensus.cc:493] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.942332 16202 ts_tablet_manager.cc:1434] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:16:23.942679 16371 raft_consensus.cc:515] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:23.943291 16081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5922376de62747aabde22ad134c86f36" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:23.943624 16386 leader_election.cc:290] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:23.944764 16371 leader_election.cc:290] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
W20260709 13:16:23.945935 15897 leader_election.cc:343] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.946985 16202 tablet_bootstrap.cc:492] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:23.948091 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f15b348b66b4bffae9c7f9eed8207e2" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:23.948385 16204 tablet_bootstrap.cc:492] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:23.946977 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5922376de62747aabde22ad134c86f36" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:23.949438 16008 raft_consensus.cc:2468] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:23.950727 15898 leader_election.cc:304] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:23.938284 16201 raft_consensus.cc:359] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:23.951828 16201 raft_consensus.cc:385] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:23.951871 16371 raft_consensus.cc:2804] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:23.952152 16201 raft_consensus.cc:740] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:23.952361 16371 raft_consensus.cc:493] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.952788 16371 raft_consensus.cc:3060] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.953420 16202 tablet_bootstrap.cc:654] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:23.948761 16156 raft_consensus.cc:2468] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:23.954607 16201 consensus_queue.cc:260] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:23.956032 15899 leader_election.cc:304] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:23.956858 16386 raft_consensus.cc:2804] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:23.957228 16386 raft_consensus.cc:493] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.957640 16386 raft_consensus.cc:3060] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.948287 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f15b348b66b4bffae9c7f9eed8207e2" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:23.966599 15898 leader_election.cc:343] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:23.972551 16371 raft_consensus.cc:515] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:23.976624 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5922376de62747aabde22ad134c86f36" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:23.977180 16006 raft_consensus.cc:3060] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:23.988706 16368 raft_consensus.cc:493] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.989398 16368 raft_consensus.cc:515] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:23.997263 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20762eaf5e494270a0e86a0b95c1437f" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
W20260709 13:16:23.998636 15971 leader_election.cc:343] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.000346 16368 leader_election.cc:290] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:24.004896 16394 consensus_queue.cc:1048] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:24.015576 16394 consensus_queue.cc:1048] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:24.019189 16081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20762eaf5e494270a0e86a0b95c1437f" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
W20260709 13:16:24.026643 15972 leader_election.cc:343] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.027364 15972 leader_election.cc:304] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:24.029055 16293 raft_consensus.cc:493] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.029560 16293 raft_consensus.cc:515] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:24.031672 16293 leader_election.cc:290] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:24.032792 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc17a40e973a46a18d21e55a0a939417" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:24.033609 16375 raft_consensus.cc:2749] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:24.034222 16119 leader_election.cc:343] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.031554 16201 ts_tablet_manager.cc:1434] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.100s	user 0.003s	sys 0.002s
I20260709 13:16:24.037678 16081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc17a40e973a46a18d21e55a0a939417" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
W20260709 13:16:24.039646 16120 leader_election.cc:343] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.040313 16120 leader_election.cc:304] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:24.041790 16394 raft_consensus.cc:2749] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:24.043092 16201 tablet_bootstrap.cc:492] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:23.985922 16006 raft_consensus.cc:2468] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:24.050187 15898 leader_election.cc:304] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:24.051043 16237 raft_consensus.cc:2804] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:24.051640 16237 raft_consensus.cc:697] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:24.052682 16237 consensus_queue.cc:237] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.074379 16386 raft_consensus.cc:515] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.078115 16371 leader_election.cc:290] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:24.079051 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5922376de62747aabde22ad134c86f36" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
W20260709 13:16:24.081060 15897 leader_election.cc:343] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.040675 16204 tablet_bootstrap.cc:654] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.084915 15830 catalog_manager.cc:5697] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:24.104427 16394 consensus_queue.cc:1048] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:24.152634 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f15b348b66b4bffae9c7f9eed8207e2" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:24.153302 16156 raft_consensus.cc:3060] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.156234 16394 raft_consensus.cc:493] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.156806 16394 raft_consensus.cc:515] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.172856 16394 leader_election.cc:290] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:24.174626 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaa175423efe491da6747a37df086375" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:24.174809 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaa175423efe491da6747a37df086375" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:24.175577 16006 raft_consensus.cc:2468] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 0.
W20260709 13:16:24.176702 16120 leader_election.cc:343] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.179737 16121 leader_election.cc:304] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:24.198589 16394 raft_consensus.cc:2804] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:24.199241 16394 raft_consensus.cc:493] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.199752 16394 raft_consensus.cc:3060] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:24.205838 16428 log.cc:927] Time spent T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:16:24.214059 16386 leader_election.cc:290] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:24.220956 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f15b348b66b4bffae9c7f9eed8207e2" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
W20260709 13:16:24.222607 15898 leader_election.cc:343] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.223179 16156 raft_consensus.cc:2468] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:24.224131 16209 raft_consensus.cc:493] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.224669 16209 raft_consensus.cc:515] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.227698 16201 tablet_bootstrap.cc:654] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.227916 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2e0e0244294ec18481adf638d57ff0" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:24.224289 16204 tablet_bootstrap.cc:492] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:24.230849 16204 ts_tablet_manager.cc:1403] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.283s	user 0.023s	sys 0.019s
I20260709 13:16:24.233768 16204 raft_consensus.cc:359] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:24.233877 15899 leader_election.cc:304] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3
I20260709 13:16:24.235170 16386 raft_consensus.cc:2804] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:24.235733 16203 tablet_bootstrap.cc:492] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:24.235930 16386 raft_consensus.cc:697] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
W20260709 13:16:24.236590 15972 leader_election.cc:343] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.237041 16204 raft_consensus.cc:385] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.237388 16204 raft_consensus.cc:740] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.236194 16203 ts_tablet_manager.cc:1403] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.325s	user 0.063s	sys 0.070s
I20260709 13:16:24.239815 16204 consensus_queue.cc:260] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:24.244688 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2e0e0244294ec18481adf638d57ff0" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
W20260709 13:16:24.246034 15974 leader_election.cc:343] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.246498 15974 leader_election.cc:304] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:24.245832 16203 raft_consensus.cc:359] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.247457 16203 raft_consensus.cc:385] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.247589 16368 raft_consensus.cc:2749] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:24.247774 16203 raft_consensus.cc:740] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.248472 16203 consensus_queue.cc:260] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.237738 16386 consensus_queue.cc:237] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.250443 16209 leader_election.cc:290] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:24.251570 16203 ts_tablet_manager.cc:1434] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:16:24.254650 16202 tablet_bootstrap.cc:492] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:24.255116 16202 ts_tablet_manager.cc:1403] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.308s	user 0.084s	sys 0.116s
I20260709 13:16:24.255353 16298 raft_consensus.cc:493] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.255990 16298 raft_consensus.cc:515] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.256330 16394 raft_consensus.cc:515] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.259487 16204 ts_tablet_manager.cc:1434] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 13:16:24.259433 16298 leader_election.cc:290] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:24.262689 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaa175423efe491da6747a37df086375" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:24.263113 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2e0e0244294ec18481adf638d57ff0" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:24.264101 16202 raft_consensus.cc:359] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.265202 16203 tablet_bootstrap.cc:492] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:24.266677 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2e0e0244294ec18481adf638d57ff0" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:24.273407 16202 raft_consensus.cc:385] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.273875 16202 raft_consensus.cc:740] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.274474 16008 raft_consensus.cc:3060] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.275040 16204 tablet_bootstrap.cc:492] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
W20260709 13:16:24.275568 15879 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:24.275154 16006 raft_consensus.cc:2468] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:24.277006 16047 leader_election.cc:304] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:24.277861 16298 raft_consensus.cc:2804] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:24.278158 16298 raft_consensus.cc:493] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.278414 16298 raft_consensus.cc:3060] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.281775 16008 raft_consensus.cc:2468] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 1.
I20260709 13:16:24.284114 16202 consensus_queue.cc:260] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.285070 16156 raft_consensus.cc:2468] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:24.288376 16201 tablet_bootstrap.cc:492] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:24.288889 16201 ts_tablet_manager.cc:1403] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.246s	user 0.029s	sys 0.009s
I20260709 13:16:24.290989 16298 raft_consensus.cc:515] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.291656 16201 raft_consensus.cc:359] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.292322 16201 raft_consensus.cc:385] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.292585 16201 raft_consensus.cc:740] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.293130 16204 tablet_bootstrap.cc:654] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.293320 16201 consensus_queue.cc:260] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.294075 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2e0e0244294ec18481adf638d57ff0" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:24.294673 16007 raft_consensus.cc:3060] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.295473 16201 ts_tablet_manager.cc:1434] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:16:24.296406 16202 ts_tablet_manager.cc:1434] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.000s
I20260709 13:16:24.297711 16202 tablet_bootstrap.cc:492] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:24.302924 16203 tablet_bootstrap.cc:654] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:24.302379 16029 debug-util.cc:398] Leaking SignalData structure 0x7b08002ca8c0 after lost signal to thread 6940
I20260709 13:16:24.304838 16201 tablet_bootstrap.cc:492] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
W20260709 13:16:24.306267 16029 debug-util.cc:398] Leaking SignalData structure 0x7b08000e9680 after lost signal to thread 15865
W20260709 13:16:24.307668 16029 debug-util.cc:398] Leaking SignalData structure 0x7b08000de3a0 after lost signal to thread 15957
I20260709 13:16:24.308279 16007 raft_consensus.cc:2468] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
W20260709 13:16:24.308842 16029 debug-util.cc:398] Leaking SignalData structure 0x7b08001c4800 after lost signal to thread 16032
W20260709 13:16:24.310014 16029 debug-util.cc:398] Leaking SignalData structure 0x7b08002db660 after lost signal to thread 16106
I20260709 13:16:24.311235 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2e0e0244294ec18481adf638d57ff0" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:24.312029 16156 raft_consensus.cc:3060] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.318517 16298 leader_election.cc:290] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
W20260709 13:16:24.319175 16029 debug-util.cc:398] Leaking SignalData structure 0x7b0800311360 after lost signal to thread 16180
I20260709 13:16:24.322454 16121 leader_election.cc:304] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:24.323383 16393 raft_consensus.cc:2804] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:24.323856 16393 raft_consensus.cc:697] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
W20260709 13:16:24.322448 16029 debug-util.cc:398] Leaking SignalData structure 0x7b0800155d80 after lost signal to thread 16311
W20260709 13:16:24.324505 16029 debug-util.cc:398] Leaking SignalData structure 0x7b08003e27a0 after lost signal to thread 16336
I20260709 13:16:24.325033 16156 raft_consensus.cc:2468] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:24.324659 16393 consensus_queue.cc:237] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.327217 16047 leader_election.cc:304] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:24.328650 16298 raft_consensus.cc:2804] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:24.329285 16298 raft_consensus.cc:697] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:24.330154 16201 tablet_bootstrap.cc:654] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.330271 16298 consensus_queue.cc:237] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.330818 16293 raft_consensus.cc:493] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.342869 16202 tablet_bootstrap.cc:654] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.350871 16203 tablet_bootstrap.cc:492] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:24.350826 16230 raft_consensus.cc:493] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.351459 16394 leader_election.cc:290] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:24.351486 16230 raft_consensus.cc:515] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:24.353704 16230 leader_election.cc:290] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:24.355042 16204 tablet_bootstrap.cc:492] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:24.355199 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b94b09aec6746f69853f14b417f18e3" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
W20260709 13:16:24.357842 16119 leader_election.cc:343] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.335280 16293 raft_consensus.cc:515] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:24.361727 16293 leader_election.cc:290] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:24.362330 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b94b09aec6746f69853f14b417f18e3" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:24.355505 16204 ts_tablet_manager.cc:1403] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.082s	user 0.007s	sys 0.006s
I20260709 13:16:24.364145 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6e83ffadf17413a95b361a2cf2b1b7f" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:24.364722 16080 raft_consensus.cc:2468] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 0.
W20260709 13:16:24.365898 16120 leader_election.cc:343] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.361778 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaa175423efe491da6747a37df086375" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:24.366955 16204 raft_consensus.cc:359] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.367645 16204 raft_consensus.cc:385] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.367420 16120 leader_election.cc:304] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:24.367931 16204 raft_consensus.cc:740] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.359179 16203 ts_tablet_manager.cc:1403] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.094s	user 0.011s	sys 0.006s
I20260709 13:16:24.369581 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6e83ffadf17413a95b361a2cf2b1b7f" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:24.368965 16204 consensus_queue.cc:260] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.368462 15830 catalog_manager.cc:5697] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:24.372808 16202 tablet_bootstrap.cc:492] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
W20260709 13:16:24.373109 16119 leader_election.cc:343] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.373275 16202 ts_tablet_manager.cc:1403] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.076s	user 0.010s	sys 0.000s
I20260709 13:16:24.374261 16230 raft_consensus.cc:2749] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:24.375381 16120 leader_election.cc:304] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:24.375931 16202 raft_consensus.cc:359] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.375429 15829 catalog_manager.cc:5697] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 1 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } }
W20260709 13:16:24.377640 16120 leader_election.cc:343] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.378458 16202 raft_consensus.cc:385] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.381647 16202 raft_consensus.cc:740] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.382987 16202 consensus_queue.cc:260] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.387624 16204 ts_tablet_manager.cc:1434] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 13:16:24.390262 16203 raft_consensus.cc:359] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.390919 16203 raft_consensus.cc:385] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.391184 16203 raft_consensus.cc:740] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.382531 16230 raft_consensus.cc:2804] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:24.391726 16230 raft_consensus.cc:493] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.392093 16230 raft_consensus.cc:3060] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.391848 16203 consensus_queue.cc:260] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.398083 16202 ts_tablet_manager.cc:1434] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.001s
I20260709 13:16:24.404206 16204 tablet_bootstrap.cc:492] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:24.404588 16202 tablet_bootstrap.cc:492] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:24.409077 16230 raft_consensus.cc:515] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:24.411047 16203 ts_tablet_manager.cc:1434] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 13:16:24.420516 16203 tablet_bootstrap.cc:492] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:24.421136 16230 leader_election.cc:290] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:24.421847 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6e83ffadf17413a95b361a2cf2b1b7f" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:24.411898 16202 tablet_bootstrap.cc:654] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.422286 16082 raft_consensus.cc:3060] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.423478 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6e83ffadf17413a95b361a2cf2b1b7f" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
W20260709 13:16:24.424909 16119 leader_election.cc:343] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.416143 16201 tablet_bootstrap.cc:492] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:24.396555 15830 catalog_manager.cc:5697] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:24.426716 16201 ts_tablet_manager.cc:1403] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.122s	user 0.014s	sys 0.008s
I20260709 13:16:24.419278 16204 tablet_bootstrap.cc:654] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.429783 16201 raft_consensus.cc:359] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.430444 16201 raft_consensus.cc:385] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.430721 16201 raft_consensus.cc:740] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.431283 16082 raft_consensus.cc:2468] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 1.
I20260709 13:16:24.431666 16201 consensus_queue.cc:260] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.434877 16120 leader_election.cc:304] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:24.436385 16230 raft_consensus.cc:2804] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:24.437265 16230 raft_consensus.cc:697] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
I20260709 13:16:24.440034 16201 ts_tablet_manager.cc:1434] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:16:24.432514 16203 tablet_bootstrap.cc:654] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.438365 16230 consensus_queue.cc:237] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:24.455623 16204 tablet_bootstrap.cc:492] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:24.456135 16204 ts_tablet_manager.cc:1403] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.052s	user 0.013s	sys 0.002s
I20260709 13:16:24.459131 16204 raft_consensus.cc:359] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.459723 16204 raft_consensus.cc:385] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.459966 16204 raft_consensus.cc:740] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.462376 16204 consensus_queue.cc:260] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.464322 16201 tablet_bootstrap.cc:492] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:24.464975 16203 tablet_bootstrap.cc:492] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:24.465464 16203 ts_tablet_manager.cc:1403] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.045s	user 0.012s	sys 0.000s
I20260709 13:16:24.465646 16209 raft_consensus.cc:493] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.466122 16209 raft_consensus.cc:515] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.467684 16202 tablet_bootstrap.cc:492] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:24.468132 16202 ts_tablet_manager.cc:1403] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.027s
I20260709 13:16:24.468248 16209 leader_election.cc:290] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:24.469931 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31217a26afbc45258ea518a1623dbb01" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:24.470566 15932 raft_consensus.cc:2468] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:24.470840 16202 raft_consensus.cc:359] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.471493 16202 raft_consensus.cc:385] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.472080 16201 tablet_bootstrap.cc:654] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.472291 16203 raft_consensus.cc:359] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.472913 16203 raft_consensus.cc:385] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.473199 16203 raft_consensus.cc:740] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.474689 16203 consensus_queue.cc:260] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.475544 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31217a26afbc45258ea518a1623dbb01" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:24.476157 16156 raft_consensus.cc:2468] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:24.478524 15971 leader_election.cc:304] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:24.474538 16202 raft_consensus.cc:740] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.479773 16204 ts_tablet_manager.cc:1434] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:16:24.480258 16209 raft_consensus.cc:2804] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:24.480698 16204 tablet_bootstrap.cc:492] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:24.480955 16202 consensus_queue.cc:260] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.480683 16209 raft_consensus.cc:493] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.482790 16203 ts_tablet_manager.cc:1434] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:16:24.483342 16202 ts_tablet_manager.cc:1434] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:16:24.483742 16203 tablet_bootstrap.cc:492] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:24.484278 16202 tablet_bootstrap.cc:492] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:24.485565 16371 consensus_queue.cc:1048] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:24.482200 16209 raft_consensus.cc:3060] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.496018 16371 raft_consensus.cc:493] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.496349 16209 raft_consensus.cc:515] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.496490 16371 raft_consensus.cc:515] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:24.489490 16230 raft_consensus.cc:493] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.498760 16230 raft_consensus.cc:515] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.499478 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31217a26afbc45258ea518a1623dbb01" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:24.500106 15932 raft_consensus.cc:3060] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.500355 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc17a40e973a46a18d21e55a0a939417" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:24.501080 16156 raft_consensus.cc:2468] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:24.501143 16230 leader_election.cc:290] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:24.502249 15899 leader_election.cc:304] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:24.503027 16386 raft_consensus.cc:2804] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:24.503352 16371 leader_election.cc:290] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:24.503423 16386 raft_consensus.cc:493] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.503548 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a21a302bc17473392974aeea5a31d60" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:24.504099 16386 raft_consensus.cc:3060] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.505326 16202 tablet_bootstrap.cc:654] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.506356 16209 leader_election.cc:290] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:24.507540 16119 leader_election.cc:343] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.510700 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc17a40e973a46a18d21e55a0a939417" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
W20260709 13:16:24.512200 15897 leader_election.cc:343] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.514163 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a21a302bc17473392974aeea5a31d60" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:24.515921 16121 leader_election.cc:343] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.516654 16121 leader_election.cc:304] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:24.514140 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31217a26afbc45258ea518a1623dbb01" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:24.519240 16203 tablet_bootstrap.cc:654] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.519340 16156 raft_consensus.cc:3060] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.520264 16230 raft_consensus.cc:2749] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:24.522872 16362 raft_consensus.cc:493] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.526132 16362 raft_consensus.cc:515] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.528384 16362 leader_election.cc:290] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:24.529232 16204 tablet_bootstrap.cc:654] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.530040 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f15b348b66b4bffae9c7f9eed8207e2" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:24.537200 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f15b348b66b4bffae9c7f9eed8207e2" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:24.538210 16153 raft_consensus.cc:2393] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 903b3435239241eab01ea54ffb015ee3 in current term 1: Already voted for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in this term.
I20260709 13:16:24.544479 15829 catalog_manager.cc:5697] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 1 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:24.548199 16156 raft_consensus.cc:2468] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:24.549806 15974 leader_election.cc:304] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:24.553901 16386 raft_consensus.cc:515] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:24.555877 15974 leader_election.cc:304] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:24.556499 16386 leader_election.cc:290] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:24.556664 16368 raft_consensus.cc:2804] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:24.557164 16368 raft_consensus.cc:697] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:24.551623 16209 raft_consensus.cc:3060] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.567868 16368 consensus_queue.cc:237] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.568645 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc17a40e973a46a18d21e55a0a939417" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:24.569355 16153 raft_consensus.cc:3060] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.573442 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc17a40e973a46a18d21e55a0a939417" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:24.581737 16202 tablet_bootstrap.cc:492] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:24.581868 16298 raft_consensus.cc:493] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.582415 16298 raft_consensus.cc:515] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.585448 16153 raft_consensus.cc:2468] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
W20260709 13:16:24.582468 15897 leader_election.cc:343] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.586450 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5565bfb243b4eeab5cbb5549d639378" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:24.587924 16047 leader_election.cc:343] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.588229 15899 leader_election.cc:304] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:24.591202 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5565bfb243b4eeab5cbb5549d639378" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:24.591838 15931 raft_consensus.cc:2468] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:24.582314 16202 ts_tablet_manager.cc:1403] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.098s	user 0.014s	sys 0.015s
I20260709 13:16:24.592525 16237 raft_consensus.cc:2804] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:24.594264 16298 leader_election.cc:290] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:24.601441 16201 tablet_bootstrap.cc:492] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:24.601900 16201 ts_tablet_manager.cc:1403] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.138s	user 0.016s	sys 0.018s
I20260709 13:16:24.599296 16045 leader_election.cc:304] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3
I20260709 13:16:24.611346 15829 catalog_manager.cc:5697] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:24.617125 16203 tablet_bootstrap.cc:492] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:24.617486 16203 ts_tablet_manager.cc:1403] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.134s	user 0.010s	sys 0.009s
I20260709 13:16:24.619020 16237 raft_consensus.cc:697] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:24.618177 16275 raft_consensus.cc:2804] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:24.629488 16237 consensus_queue.cc:237] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:24.625661 16275 raft_consensus.cc:493] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.630738 16275 raft_consensus.cc:3060] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.638604 15932 raft_consensus.cc:2468] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:24.654151 16201 raft_consensus.cc:359] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.657021 16209 raft_consensus.cc:2749] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:24.657801 16275 raft_consensus.cc:515] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.662691 16450 consensus_queue.cc:1048] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 13:16:24.664397 15897 consensus_peers.cc:597] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e -> Peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Couldn't send request to peer d4d6e13a8f4a473098351ada96fd7dab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:24.665294 16201 raft_consensus.cc:385] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.668133 16201 raft_consensus.cc:740] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.670099 16201 consensus_queue.cc:260] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.675185 16201 ts_tablet_manager.cc:1434] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.073s	user 0.007s	sys 0.000s
I20260709 13:16:24.676188 16202 raft_consensus.cc:359] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.676856 16202 raft_consensus.cc:385] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.677325 16202 raft_consensus.cc:740] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.676833 16203 raft_consensus.cc:359] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.677714 16203 raft_consensus.cc:385] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.677991 16203 raft_consensus.cc:740] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.678799 16203 consensus_queue.cc:260] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.678063 16202 consensus_queue.cc:260] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.681093 16275 leader_election.cc:290] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:24.684528 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5565bfb243b4eeab5cbb5549d639378" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:24.687067 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5565bfb243b4eeab5cbb5549d639378" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:24.687714 15932 raft_consensus.cc:3060] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:24.688566 16047 leader_election.cc:343] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.690615 16202 ts_tablet_manager.cc:1434] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.095s	user 0.007s	sys 0.000s
I20260709 13:16:24.691814 16202 tablet_bootstrap.cc:492] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:24.692380 16201 tablet_bootstrap.cc:492] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:24.693823 16203 ts_tablet_manager.cc:1434] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.000s
I20260709 13:16:24.701252 16275 raft_consensus.cc:493] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.709694 16275 raft_consensus.cc:515] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.711196 16204 tablet_bootstrap.cc:492] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:24.714743 16204 ts_tablet_manager.cc:1403] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.234s	user 0.020s	sys 0.000s
I20260709 13:16:24.712924 16202 tablet_bootstrap.cc:654] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.713279 16203 tablet_bootstrap.cc:492] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:24.716513 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20762eaf5e494270a0e86a0b95c1437f" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
W20260709 13:16:24.720664 16045 leader_election.cc:343] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.722458 16275 leader_election.cc:290] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:24.724009 15830 catalog_manager.cc:5697] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:24.722163 16204 raft_consensus.cc:359] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.725603 16204 raft_consensus.cc:385] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.725818 16204 raft_consensus.cc:740] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.726316 16204 consensus_queue.cc:260] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.726940 16201 tablet_bootstrap.cc:654] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.729699 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20762eaf5e494270a0e86a0b95c1437f" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:24.730315 16007 raft_consensus.cc:2468] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:24.731266 16204 ts_tablet_manager.cc:1434] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 13:16:24.732126 16047 leader_election.cc:304] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:24.732787 16204 tablet_bootstrap.cc:492] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:24.732959 16275 raft_consensus.cc:2804] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:24.733332 16275 raft_consensus.cc:493] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.734681 16275 raft_consensus.cc:3060] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.724877 15932 raft_consensus.cc:2468] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:24.736335 16045 leader_election.cc:304] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3
I20260709 13:16:24.737676 16453 raft_consensus.cc:2804] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:24.738152 16453 raft_consensus.cc:697] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:24.740078 16453 consensus_queue.cc:237] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.755836 16203 tablet_bootstrap.cc:654] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.760000 16230 raft_consensus.cc:493] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.760653 16230 raft_consensus.cc:515] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.748603 16275 raft_consensus.cc:515] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.766510 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ac7645b9954621a685e4fc9ddde0dd" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:24.766965 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ac7645b9954621a685e4fc9ddde0dd" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
W20260709 13:16:24.768679 16119 leader_election.cc:343] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.769399 16230 leader_election.cc:290] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:24.773298 15829 catalog_manager.cc:5697] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:24.774588 16204 tablet_bootstrap.cc:654] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:24.775345 16121 leader_election.cc:343] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.788311 16121 leader_election.cc:304] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:24.789103 16230 raft_consensus.cc:2749] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:24.777019 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20762eaf5e494270a0e86a0b95c1437f" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:24.794163 15932 raft_consensus.cc:3060] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.794458 16275 leader_election.cc:290] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:24.777848 16451 consensus_queue.cc:1048] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:24.781960 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20762eaf5e494270a0e86a0b95c1437f" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:24.815225 16007 raft_consensus.cc:3060] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.841398 16230 consensus_queue.cc:1048] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:24.852738 16119 consensus_peers.cc:597] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:24.863212 15932 raft_consensus.cc:2468] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:24.864488 16045 leader_election.cc:304] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:24.867975 16201 tablet_bootstrap.cc:492] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:24.868487 16201 ts_tablet_manager.cc:1403] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.184s	user 0.015s	sys 0.023s
I20260709 13:16:24.823935 16007 raft_consensus.cc:2468] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:24.882560 16201 raft_consensus.cc:359] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.883365 16201 raft_consensus.cc:385] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.884145 16201 raft_consensus.cc:740] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.885967 16201 consensus_queue.cc:260] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.865346 16275 raft_consensus.cc:2804] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:24.881484 16230 raft_consensus.cc:493] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.894276 16201 ts_tablet_manager.cc:1434] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:16:24.896281 16080 raft_consensus.cc:3060] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:24.890714 16275 raft_consensus.cc:697] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:24.900559 16275 consensus_queue.cc:237] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.903568 16298 consensus_queue.cc:1048] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:24.897641 16230 raft_consensus.cc:515] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.906658 16453 raft_consensus.cc:493] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.907152 16453 raft_consensus.cc:515] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:24.907704 16230 leader_election.cc:290] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:24.909209 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dadf33b9db824fdcbb63864c9e975c6b" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:24.910640 16121 leader_election.cc:343] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.924786 16202 tablet_bootstrap.cc:492] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:24.925356 16202 ts_tablet_manager.cc:1403] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.234s	user 0.031s	sys 0.060s
I20260709 13:16:24.925849 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaa175423efe491da6747a37df086375" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:24.926645 16293 consensus_queue.cc:1048] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:24.927304 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dadf33b9db824fdcbb63864c9e975c6b" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:24.927505 16453 leader_election.cc:290] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:24.927917 16202 raft_consensus.cc:359] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.928577 16202 raft_consensus.cc:385] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.928831 16202 raft_consensus.cc:740] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.929497 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aaa175423efe491da6747a37df086375" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:24.929544 16202 consensus_queue.cc:260] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:24.932073 16202 ts_tablet_manager.cc:1434] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:16:24.932930 16202 tablet_bootstrap.cc:492] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
W20260709 13:16:24.936605 16120 leader_election.cc:343] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:24.937110 16120 leader_election.cc:304] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:24.937836 16203 tablet_bootstrap.cc:492] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:24.938279 16203 ts_tablet_manager.cc:1403] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.236s	user 0.014s	sys 0.019s
I20260709 13:16:24.944460 16293 raft_consensus.cc:2749] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:24.940959 16203 raft_consensus.cc:359] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.947983 16203 raft_consensus.cc:385] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.948294 16203 raft_consensus.cc:740] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.949055 16203 consensus_queue.cc:260] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.950807 16368 raft_consensus.cc:493] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.951340 16368 raft_consensus.cc:515] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:24.952107 16204 tablet_bootstrap.cc:492] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:24.952565 16204 ts_tablet_manager.cc:1403] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.220s	user 0.012s	sys 0.015s
I20260709 13:16:24.957499 16368 leader_election.cc:290] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:24.959717 16201 tablet_bootstrap.cc:492] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:24.972733 16203 ts_tablet_manager.cc:1434] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:16:24.976359 16203 tablet_bootstrap.cc:492] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
W20260709 13:16:24.976291 16047 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:32879, user_credentials={real_user=slave}} blocked reactor thread for 54302us
I20260709 13:16:24.978837 16204 raft_consensus.cc:359] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:24.979534 16204 raft_consensus.cc:385] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:24.979816 16204 raft_consensus.cc:740] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:24.979933 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5565bfb243b4eeab5cbb5549d639378" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:24.980528 16204 consensus_queue.cc:260] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:24.981910 16201 tablet_bootstrap.cc:654] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.982887 16202 tablet_bootstrap.cc:654] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:24.984414 16204 ts_tablet_manager.cc:1434] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.001s
I20260709 13:16:24.975728 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5565bfb243b4eeab5cbb5549d639378" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:24.987547 16204 tablet_bootstrap.cc:492] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:24.990458 16047 leader_election.cc:304] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:24.992758 15932 raft_consensus.cc:2393] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 903b3435239241eab01ea54ffb015ee3 in current term 1: Already voted for candidate d4d6e13a8f4a473098351ada96fd7dab in this term.
I20260709 13:16:25.014542 16209 consensus_queue.cc:1048] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:25.002868 16466 raft_consensus.cc:2749] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:25.020970 15972 leader_election.cc:304] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:25.023770 16362 raft_consensus.cc:3060] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.024268 16204 tablet_bootstrap.cc:654] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.018244 16203 tablet_bootstrap.cc:654] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.034386 16362 raft_consensus.cc:2749] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:25.039038 16230 raft_consensus.cc:493] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.039650 16230 raft_consensus.cc:515] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.042868 16230 leader_election.cc:290] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:25.054963 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28cbf604c51f48da99cb7e3089651236" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:25.055975 16204 tablet_bootstrap.cc:492] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:25.056438 16204 ts_tablet_manager.cc:1403] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.069s	user 0.012s	sys 0.001s
I20260709 13:16:25.064409 16204 raft_consensus.cc:359] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:25.065166 16204 raft_consensus.cc:385] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.065514 16204 raft_consensus.cc:740] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.076038 16204 consensus_queue.cc:260] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:25.056448 16451 raft_consensus.cc:493] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:25.076604 16119 leader_election.cc:343] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.085783 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28cbf604c51f48da99cb7e3089651236" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:25.086043 16368 raft_consensus.cc:493] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.086555 16368 raft_consensus.cc:515] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
W20260709 13:16:25.087235 16121 leader_election.cc:343] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.087692 16121 leader_election.cc:304] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:25.088491 16368 leader_election.cc:290] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:25.088732 16394 raft_consensus.cc:2749] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:25.085216 16201 tablet_bootstrap.cc:492] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:25.090044 16201 ts_tablet_manager.cc:1403] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.131s	user 0.015s	sys 0.000s
I20260709 13:16:25.083246 16451 raft_consensus.cc:515] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.092409 16451 leader_election.cc:290] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:25.093479 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a47fef045e183c154c25260da37" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:25.094120 15931 raft_consensus.cc:2468] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:25.096474 16204 ts_tablet_manager.cc:1434] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.001s
I20260709 13:16:25.092691 16201 raft_consensus.cc:359] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.097693 16201 raft_consensus.cc:385] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.098069 16201 raft_consensus.cc:740] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.101974 15971 leader_election.cc:304] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:25.103188 16081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b94b09aec6746f69853f14b417f18e3" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:25.103533 16469 consensus_queue.cc:1048] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:25.104789 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b94b09aec6746f69853f14b417f18e3" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:25.105444 16149 raft_consensus.cc:2468] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:25.106848 16201 consensus_queue.cc:260] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.110064 16204 tablet_bootstrap.cc:492] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:25.109696 15827 catalog_manager.cc:5697] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:25.112929 16362 raft_consensus.cc:2804] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:25.116736 16362 raft_consensus.cc:493] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.117440 16362 raft_consensus.cc:3060] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:25.111757 15897 leader_election.cc:343] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.119334 16203 tablet_bootstrap.cc:492] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:25.119773 16203 ts_tablet_manager.cc:1403] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.146s	user 0.011s	sys 0.009s
I20260709 13:16:25.124938 16080 raft_consensus.cc:3060] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.127326 15899 leader_election.cc:304] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:25.128230 16237 raft_consensus.cc:2804] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:25.128634 16237 raft_consensus.cc:493] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.128973 16237 raft_consensus.cc:3060] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.130815 16394 consensus_queue.cc:1048] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:25.131752 16204 tablet_bootstrap.cc:654] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.136425 16201 ts_tablet_manager.cc:1434] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 13:16:25.136368 16362 raft_consensus.cc:515] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:25.137544 16201 tablet_bootstrap.cc:492] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:25.139360 16202 tablet_bootstrap.cc:492] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:25.139420 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a47fef045e183c154c25260da37" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:25.139942 16202 ts_tablet_manager.cc:1403] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.207s	user 0.011s	sys 0.031s
I20260709 13:16:25.140060 15931 raft_consensus.cc:3060] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.142552 16451 consensus_queue.cc:1048] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:25.142679 16202 raft_consensus.cc:359] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.146729 16202 raft_consensus.cc:385] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.147027 16202 raft_consensus.cc:740] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.147984 16362 leader_election.cc:290] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:25.154429 16469 consensus_queue.cc:1048] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:25.151402 16202 consensus_queue.cc:260] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
W20260709 13:16:25.172058 15897 consensus_peers.cc:597] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e -> Peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Couldn't send request to peer d4d6e13a8f4a473098351ada96fd7dab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:25.173668 16298 consensus_queue.cc:1048] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:25.152024 15972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:32795, user_credentials={real_user=slave}} blocked reactor thread for 63583.4us
I20260709 13:16:25.176455 16079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a47fef045e183c154c25260da37" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:25.178166 16203 raft_consensus.cc:359] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.179239 16203 raft_consensus.cc:385] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.179668 16203 raft_consensus.cc:740] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.180927 16203 consensus_queue.cc:260] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.183694 16202 ts_tablet_manager.cc:1434] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 13:16:25.185675 16202 tablet_bootstrap.cc:492] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:25.187287 16203 ts_tablet_manager.cc:1434] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.000s
I20260709 13:16:25.191994 16079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a574a47fef045e183c154c25260da37" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:25.189779 16203 tablet_bootstrap.cc:492] T 0a574a47fef045e183c154c25260da37 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:25.198071 16362 consensus_queue.cc:1048] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:25.214381 16202 tablet_bootstrap.cc:654] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.177814 15931 raft_consensus.cc:2468] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
W20260709 13:16:25.211331 15972 leader_election.cc:343] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.236008 16201 tablet_bootstrap.cc:654] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.236166 16237 raft_consensus.cc:515] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.240748 16237 leader_election.cc:290] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:25.242087 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b94b09aec6746f69853f14b417f18e3" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:25.249269 15971 leader_election.cc:304] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:25.263839 16203 tablet_bootstrap.cc:654] T 0a574a47fef045e183c154c25260da37 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.270532 16476 raft_consensus.cc:2804] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:25.271090 16476 raft_consensus.cc:697] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:25.271931 16476 consensus_queue.cc:237] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:25.251394 16156 raft_consensus.cc:3060] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.283629 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b94b09aec6746f69853f14b417f18e3" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:25.292979 16204 tablet_bootstrap.cc:492] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
W20260709 13:16:25.297057 15972 leader_election.cc:343] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.293452 16204 ts_tablet_manager.cc:1403] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.187s	user 0.017s	sys 0.018s
I20260709 13:16:25.318317 16202 tablet_bootstrap.cc:492] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:25.320472 16202 ts_tablet_manager.cc:1403] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.135s	user 0.016s	sys 0.014s
I20260709 13:16:25.322392 16275 consensus_queue.cc:1048] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:25.326727 15897 leader_election.cc:343] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.330159 16204 raft_consensus.cc:359] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.335264 16204 raft_consensus.cc:385] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.335630 16204 raft_consensus.cc:740] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.336560 16204 consensus_queue.cc:260] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.363158 16204 ts_tablet_manager.cc:1434] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.001s
I20260709 13:16:25.355871 16202 raft_consensus.cc:359] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.366030 16202 raft_consensus.cc:385] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.354631 16375 raft_consensus.cc:493] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.366567 16202 raft_consensus.cc:740] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.367638 16466 consensus_queue.cc:1048] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:25.367599 16202 consensus_queue.cc:260] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.369688 16202 ts_tablet_manager.cc:1434] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.048s	user 0.003s	sys 0.002s
I20260709 13:16:25.370616 16202 tablet_bootstrap.cc:492] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:25.371110 16375 raft_consensus.cc:515] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.371500 16156 raft_consensus.cc:2468] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:25.372540 16204 tablet_bootstrap.cc:492] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:25.373221 16375 leader_election.cc:290] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:25.365646 16203 tablet_bootstrap.cc:492] T 0a574a47fef045e183c154c25260da37 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:25.374923 16203 ts_tablet_manager.cc:1403] T 0a574a47fef045e183c154c25260da37 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.185s	user 0.029s	sys 0.019s
I20260709 13:16:25.375016 15899 leader_election.cc:304] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:25.375355 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ac7645b9954621a685e4fc9ddde0dd" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:25.376015 15933 raft_consensus.cc:2468] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:25.376616 16451 raft_consensus.cc:2804] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:25.377241 16451 raft_consensus.cc:697] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:25.377535 16203 raft_consensus.cc:359] T 0a574a47fef045e183c154c25260da37 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:25.377907 15971 leader_election.cc:304] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:25.380460 16476 raft_consensus.cc:2804] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:25.380841 16476 raft_consensus.cc:493] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.381186 16476 raft_consensus.cc:3060] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.378139 16451 consensus_queue.cc:237] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.384541 16202 tablet_bootstrap.cc:654] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.378244 16203 raft_consensus.cc:385] T 0a574a47fef045e183c154c25260da37 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.387174 16203 raft_consensus.cc:740] T 0a574a47fef045e183c154c25260da37 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.390748 16201 tablet_bootstrap.cc:492] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:25.391204 16201 ts_tablet_manager.cc:1403] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.254s	user 0.016s	sys 0.002s
I20260709 13:16:25.393184 16203 consensus_queue.cc:260] T 0a574a47fef045e183c154c25260da37 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:25.380060 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ac7645b9954621a685e4fc9ddde0dd" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:25.404896 16149 raft_consensus.cc:2468] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:25.406363 16203 ts_tablet_manager.cc:1434] T 0a574a47fef045e183c154c25260da37 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:16:25.407557 16203 tablet_bootstrap.cc:492] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:25.408345 16204 tablet_bootstrap.cc:654] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:25.410948 16479 log.cc:927] Time spent T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:16:25.415570 16203 tablet_bootstrap.cc:654] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.431519 16201 raft_consensus.cc:359] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.432511 16201 raft_consensus.cc:385] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.432741 16201 raft_consensus.cc:740] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.434142 16201 consensus_queue.cc:260] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.436825 16201 ts_tablet_manager.cc:1434] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.045s	user 0.005s	sys 0.000s
I20260709 13:16:25.458781 16201 tablet_bootstrap.cc:492] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:25.481132 15830 catalog_manager.cc:5697] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:25.495801 16476 raft_consensus.cc:515] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.497898 16476 leader_election.cc:290] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:25.503432 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ac7645b9954621a685e4fc9ddde0dd" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:25.504143 15933 raft_consensus.cc:3060] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.502218 16230 raft_consensus.cc:493] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.506940 16230 raft_consensus.cc:515] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.509066 16230 leader_election.cc:290] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:25.511876 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0c303e2dd466caf0e684c6ea83284" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:25.511880 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0c303e2dd466caf0e684c6ea83284" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
W20260709 13:16:25.515360 16120 leader_election.cc:343] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:25.518595 16119 leader_election.cc:343] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.519013 16119 leader_election.cc:304] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:25.520705 16204 tablet_bootstrap.cc:492] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:25.521183 16204 ts_tablet_manager.cc:1403] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.149s	user 0.020s	sys 0.038s
I20260709 13:16:25.522910 16203 tablet_bootstrap.cc:492] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:25.523677 16203 ts_tablet_manager.cc:1403] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.117s	user 0.017s	sys 0.028s
I20260709 13:16:25.524955 15830 catalog_manager.cc:5697] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:25.526675 16201 tablet_bootstrap.cc:654] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.527868 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62ac7645b9954621a685e4fc9ddde0dd" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:25.528435 16149 raft_consensus.cc:3060] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.526569 16230 raft_consensus.cc:2749] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:25.538938 16203 raft_consensus.cc:359] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.539722 16203 raft_consensus.cc:385] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.540025 16203 raft_consensus.cc:740] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.540676 16203 consensus_queue.cc:260] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.545742 15933 raft_consensus.cc:2468] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:25.546741 16203 ts_tablet_manager.cc:1434] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:16:25.546870 15971 leader_election.cc:304] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:25.547710 16362 raft_consensus.cc:2804] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:25.548410 16362 raft_consensus.cc:697] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:25.540863 16202 tablet_bootstrap.cc:492] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:25.549461 16202 ts_tablet_manager.cc:1403] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.179s	user 0.012s	sys 0.002s
I20260709 13:16:25.549285 16362 consensus_queue.cc:237] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.553059 16203 tablet_bootstrap.cc:492] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
W20260709 13:16:25.544970 15879 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:25.556442 16202 raft_consensus.cc:359] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.558907 16202 raft_consensus.cc:385] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.559281 16202 raft_consensus.cc:740] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.560142 16204 raft_consensus.cc:359] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.560865 16204 raft_consensus.cc:385] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.560282 16202 consensus_queue.cc:260] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.564576 16203 tablet_bootstrap.cc:654] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.566990 16149 raft_consensus.cc:2468] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:25.561165 16204 raft_consensus.cc:740] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.572110 16202 ts_tablet_manager.cc:1434] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 13:16:25.571786 16204 consensus_queue.cc:260] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.573182 16230 raft_consensus.cc:493] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.573757 16204 ts_tablet_manager.cc:1434] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.001s
I20260709 13:16:25.573629 16230 raft_consensus.cc:515] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.575672 16204 tablet_bootstrap.cc:492] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:25.576759 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad97b67157f44c618ddfebbc1c25588b" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:25.578649 16121 leader_election.cc:343] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.581173 16079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad97b67157f44c618ddfebbc1c25588b" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
W20260709 13:16:25.582931 16120 leader_election.cc:343] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.582814 15831 catalog_manager.cc:5697] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:25.583567 16120 leader_election.cc:304] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:25.585137 16293 raft_consensus.cc:2749] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:25.586031 16230 leader_election.cc:290] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:25.587114 16202 tablet_bootstrap.cc:492] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:25.593500 16204 tablet_bootstrap.cc:654] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.607973 16201 tablet_bootstrap.cc:492] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:25.608479 16201 ts_tablet_manager.cc:1403] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.159s	user 0.008s	sys 0.004s
I20260709 13:16:25.609962 16202 tablet_bootstrap.cc:654] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.611006 16201 raft_consensus.cc:359] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.611690 16201 raft_consensus.cc:385] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.611964 16201 raft_consensus.cc:740] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.613837 16201 consensus_queue.cc:260] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.614985 16204 tablet_bootstrap.cc:492] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:25.615438 16204 ts_tablet_manager.cc:1403] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.001s
I20260709 13:16:25.615597 16203 tablet_bootstrap.cc:492] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:25.615851 16201 ts_tablet_manager.cc:1434] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:25.616025 16203 ts_tablet_manager.cc:1403] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.063s	user 0.028s	sys 0.016s
I20260709 13:16:25.617928 16204 raft_consensus.cc:359] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:25.618713 16201 tablet_bootstrap.cc:492] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:25.618816 16204 raft_consensus.cc:385] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.618130 16203 raft_consensus.cc:359] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.619182 16204 raft_consensus.cc:740] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.619302 16203 raft_consensus.cc:385] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.619580 16230 raft_consensus.cc:493] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.619664 16203 raft_consensus.cc:740] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.619952 16204 consensus_queue.cc:260] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:25.620088 16230 raft_consensus.cc:515] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.620369 16203 consensus_queue.cc:260] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.621889 16230 leader_election.cc:290] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:25.622380 16203 ts_tablet_manager.cc:1434] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:16:25.623304 16203 tablet_bootstrap.cc:492] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:25.623404 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7a15b080c54551b2c51bee7af64600" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:25.627848 16201 tablet_bootstrap.cc:654] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.628417 16079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7a15b080c54551b2c51bee7af64600" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:25.629688 16202 tablet_bootstrap.cc:492] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:25.630137 16202 ts_tablet_manager.cc:1403] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.057s	user 0.011s	sys 0.000s
W20260709 13:16:25.632020 16121 leader_election.cc:343] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.632836 16202 raft_consensus.cc:359] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.633472 16202 raft_consensus.cc:385] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.633713 16202 raft_consensus.cc:740] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.634305 16202 consensus_queue.cc:260] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.634678 16204 ts_tablet_manager.cc:1434] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
W20260709 13:16:25.639150 16120 leader_election.cc:343] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.639565 16120 leader_election.cc:304] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:25.641342 16293 raft_consensus.cc:2749] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:25.642022 16204 tablet_bootstrap.cc:492] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:25.642408 16203 tablet_bootstrap.cc:654] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.645983 16202 ts_tablet_manager.cc:1434] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 13:16:25.647161 16202 tablet_bootstrap.cc:492] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:25.652915 16202 tablet_bootstrap.cc:654] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.665283 16204 tablet_bootstrap.cc:654] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.665695 16202 tablet_bootstrap.cc:492] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:25.666110 16202 ts_tablet_manager.cc:1403] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.003s
I20260709 13:16:25.666900 16386 raft_consensus.cc:493] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.667409 16386 raft_consensus.cc:515] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:25.668946 16202 raft_consensus.cc:359] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.669384 16386 leader_election.cc:290] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:25.669615 16202 raft_consensus.cc:385] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.669905 16202 raft_consensus.cc:740] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.670301 16453 raft_consensus.cc:493] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.670637 16202 consensus_queue.cc:260] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.670801 16453 raft_consensus.cc:515] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.672761 16453 leader_election.cc:290] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:25.673411 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c53d28c984a4016b8461698b47d3300" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:25.674932 16203 tablet_bootstrap.cc:492] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:25.676077 16202 ts_tablet_manager.cc:1434] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.001s
I20260709 13:16:25.676353 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc17a40e973a46a18d21e55a0a939417" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:25.676903 16203 ts_tablet_manager.cc:1403] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.054s	user 0.009s	sys 0.006s
I20260709 13:16:25.677300 16202 tablet_bootstrap.cc:492] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:25.677594 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc17a40e973a46a18d21e55a0a939417" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
W20260709 13:16:25.679044 15898 leader_election.cc:343] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.674870 16079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c53d28c984a4016b8461698b47d3300" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:25.679754 16203 raft_consensus.cc:359] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.680502 16203 raft_consensus.cc:385] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.680796 16203 raft_consensus.cc:740] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.681226 16045 leader_election.cc:304] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8
W20260709 13:16:25.681591 15897 leader_election.cc:343] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.681535 16203 consensus_queue.cc:260] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.682196 15897 leader_election.cc:304] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:25.682518 16453 raft_consensus.cc:2749] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:25.683318 16386 raft_consensus.cc:2749] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:25.683625 16202 tablet_bootstrap.cc:654] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.684115 16203 ts_tablet_manager.cc:1434] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:16:25.685034 16203 tablet_bootstrap.cc:492] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:25.688704 16202 tablet_bootstrap.cc:492] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:25.689263 16202 ts_tablet_manager.cc:1403] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.012s	user 0.008s	sys 0.004s
I20260709 13:16:25.689667 16201 tablet_bootstrap.cc:492] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:25.690102 16201 ts_tablet_manager.cc:1403] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.072s	user 0.020s	sys 0.008s
I20260709 13:16:25.690194 16203 tablet_bootstrap.cc:654] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.691936 16202 raft_consensus.cc:359] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.692592 16202 raft_consensus.cc:385] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.692895 16202 raft_consensus.cc:740] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.693935 16202 consensus_queue.cc:260] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.700116 16079 raft_consensus.cc:3060] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.705154 16201 raft_consensus.cc:359] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:25.708150 16201 raft_consensus.cc:385] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.708463 16201 raft_consensus.cc:740] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.709357 16201 consensus_queue.cc:260] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:25.710492 16202 ts_tablet_manager.cc:1434] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:16:25.712303 16204 tablet_bootstrap.cc:492] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:25.714540 16204 ts_tablet_manager.cc:1403] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.079s	user 0.012s	sys 0.002s
I20260709 13:16:25.716056 16201 ts_tablet_manager.cc:1434] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.026s	user 0.002s	sys 0.003s
I20260709 13:16:25.717228 16204 raft_consensus.cc:359] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.717878 16204 raft_consensus.cc:385] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.718276 16202 tablet_bootstrap.cc:492] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:25.719866 16082 raft_consensus.cc:3060] T 0a574a47fef045e183c154c25260da37 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.713034 16203 tablet_bootstrap.cc:492] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:25.721863 16203 ts_tablet_manager.cc:1403] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.019s
I20260709 13:16:25.724578 16203 raft_consensus.cc:359] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.725214 16203 raft_consensus.cc:385] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.725471 16203 raft_consensus.cc:740] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.718140 16204 raft_consensus.cc:740] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.727581 16204 consensus_queue.cc:260] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.729872 16204 ts_tablet_manager.cc:1434] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.002s
I20260709 13:16:25.726068 16203 consensus_queue.cc:260] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.730154 16450 raft_consensus.cc:493] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.731248 16450 raft_consensus.cc:515] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.731855 16203 ts_tablet_manager.cc:1434] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:16:25.732712 16203 tablet_bootstrap.cc:492] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:25.733242 16450 leader_election.cc:290] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:25.735275 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a21a302bc17473392974aeea5a31d60" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:25.735904 16008 raft_consensus.cc:2468] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:25.737833 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a21a302bc17473392974aeea5a31d60" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:25.738108 16203 tablet_bootstrap.cc:654] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.738950 15898 leader_election.cc:304] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:25.739768 16386 raft_consensus.cc:2804] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:25.740036 16149 raft_consensus.cc:2468] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:25.741636 16204 tablet_bootstrap.cc:492] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:25.740146 16386 raft_consensus.cc:493] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.742269 16386 raft_consensus.cc:3060] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.743068 16362 raft_consensus.cc:493] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.743534 16362 raft_consensus.cc:515] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:25.745584 16362 leader_election.cc:290] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:25.748924 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c53d28c984a4016b8461698b47d3300" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:25.751000 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c53d28c984a4016b8461698b47d3300" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:25.751847 15933 raft_consensus.cc:2468] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:25.753407 15971 leader_election.cc:304] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:25.752070 16201 tablet_bootstrap.cc:492] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:25.755139 16476 raft_consensus.cc:2804] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:25.755503 16476 raft_consensus.cc:493] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.755831 16476 raft_consensus.cc:3060] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.756345 16386 raft_consensus.cc:515] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
W20260709 13:16:25.759043 15972 leader_election.cc:343] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.752242 16204 tablet_bootstrap.cc:654] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.760267 16368 consensus_queue.cc:1048] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:25.765407 16476 raft_consensus.cc:515] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:25.767777 16476 leader_election.cc:290] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:25.770167 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c53d28c984a4016b8461698b47d3300" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:25.770762 15931 raft_consensus.cc:3060] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.772981 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a21a302bc17473392974aeea5a31d60" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:25.773746 16008 raft_consensus.cc:3060] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.775347 16080 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c53d28c984a4016b8461698b47d3300" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:25.776304 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a21a302bc17473392974aeea5a31d60" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:25.776849 16149 raft_consensus.cc:3060] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.782912 16008 raft_consensus.cc:2468] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:25.784754 15898 leader_election.cc:304] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:25.785557 16237 raft_consensus.cc:2804] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:25.786058 16237 raft_consensus.cc:697] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:25.786863 16237 consensus_queue.cc:237] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.788671 16386 leader_election.cc:290] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
W20260709 13:16:25.790726 15972 leader_election.cc:343] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.792042 15933 raft_consensus.cc:3060] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.799995 16202 tablet_bootstrap.cc:654] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.800791 16149 raft_consensus.cc:2468] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:25.803489 15931 raft_consensus.cc:2468] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:25.804781 15971 leader_election.cc:304] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:25.806028 16362 raft_consensus.cc:2804] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:25.806895 16362 raft_consensus.cc:697] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:25.807801 16362 consensus_queue.cc:237] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:25.817341 16476 consensus_queue.cc:1048] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:25.818550 15827 catalog_manager.cc:5697] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:25.827791 16201 tablet_bootstrap.cc:654] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.845678 15830 catalog_manager.cc:5697] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:25.848152 16204 tablet_bootstrap.cc:492] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:25.862682 16204 ts_tablet_manager.cc:1403] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.121s	user 0.014s	sys 0.027s
I20260709 13:16:25.852171 16203 tablet_bootstrap.cc:492] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:25.854439 16202 tablet_bootstrap.cc:492] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:25.865558 16203 ts_tablet_manager.cc:1403] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.133s	user 0.017s	sys 0.028s
I20260709 13:16:25.865634 16202 ts_tablet_manager.cc:1403] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.148s	user 0.018s	sys 0.001s
I20260709 13:16:25.865873 16204 raft_consensus.cc:359] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.866737 16204 raft_consensus.cc:385] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.867051 16204 raft_consensus.cc:740] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.868377 16203 raft_consensus.cc:359] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.868583 16202 raft_consensus.cc:359] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.869040 16203 raft_consensus.cc:385] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.869263 16202 raft_consensus.cc:385] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.869382 16203 raft_consensus.cc:740] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.869661 16202 raft_consensus.cc:740] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.870491 16202 consensus_queue.cc:260] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.871829 16204 consensus_queue.cc:260] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.874183 16204 ts_tablet_manager.cc:1434] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:16:25.876835 16204 tablet_bootstrap.cc:492] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:25.882427 16453 raft_consensus.cc:493] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.882926 16453 raft_consensus.cc:515] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.885233 16275 raft_consensus.cc:493] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.885474 16204 tablet_bootstrap.cc:654] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.885737 16275 raft_consensus.cc:515] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.901345 16275 leader_election.cc:290] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:25.902900 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dadf33b9db824fdcbb63864c9e975c6b" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:25.887100 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b94b09aec6746f69853f14b417f18e3" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:25.903517 16008 raft_consensus.cc:2468] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:25.903893 16156 raft_consensus.cc:2393] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4d6e13a8f4a473098351ada96fd7dab in current term 1: Already voted for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in this term.
I20260709 13:16:25.887032 16202 ts_tablet_manager.cc:1434] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:16:25.904842 16047 leader_election.cc:304] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:25.905272 16202 tablet_bootstrap.cc:492] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:25.905596 16275 raft_consensus.cc:2804] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:25.906036 16275 raft_consensus.cc:493] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.906422 16275 raft_consensus.cc:3060] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.888952 16230 raft_consensus.cc:493] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.890679 16201 tablet_bootstrap.cc:492] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:25.910779 16201 ts_tablet_manager.cc:1403] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.159s	user 0.008s	sys 0.003s
I20260709 13:16:25.910605 16230 raft_consensus.cc:515] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.912437 16230 leader_election.cc:290] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:25.913388 16201 raft_consensus.cc:359] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.914093 16201 raft_consensus.cc:385] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.915118 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a3eeeeab11406e90e1fd10d63228e8" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
W20260709 13:16:25.916569 16119 leader_election.cc:343] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.916836 16079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a3eeeeab11406e90e1fd10d63228e8" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
W20260709 13:16:25.918059 16120 leader_election.cc:343] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:25.918156 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dadf33b9db824fdcbb63864c9e975c6b" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:25.918622 16120 leader_election.cc:304] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:25.918992 16156 raft_consensus.cc:2468] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:25.920456 16293 raft_consensus.cc:2749] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:25.921070 16237 raft_consensus.cc:493] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.921576 16237 raft_consensus.cc:515] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.923671 16237 leader_election.cc:290] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:25.917400 16201 raft_consensus.cc:740] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.924643 16201 consensus_queue.cc:260] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.927753 16079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0c303e2dd466caf0e684c6ea83284" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:25.928381 16079 raft_consensus.cc:2468] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:25.929569 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0c303e2dd466caf0e684c6ea83284" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:25.930231 16156 raft_consensus.cc:2468] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:25.931551 15897 leader_election.cc:304] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:25.932855 16451 raft_consensus.cc:2804] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:25.933243 16451 raft_consensus.cc:493] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.933564 16451 raft_consensus.cc:3060] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.935976 16386 consensus_queue.cc:1048] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:25.940302 16451 raft_consensus.cc:515] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.896104 16453 leader_election.cc:290] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:25.897308 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b94b09aec6746f69853f14b417f18e3" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:25.940738 16202 tablet_bootstrap.cc:654] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.944041 16045 leader_election.cc:304] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:25.944267 16079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0c303e2dd466caf0e684c6ea83284" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:25.944901 16079 raft_consensus.cc:3060] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.945315 16453 raft_consensus.cc:3060] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.941800 16201 ts_tablet_manager.cc:1434] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:16:25.951218 16201 tablet_bootstrap.cc:492] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:25.870266 16203 consensus_queue.cc:260] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.895340 16204 tablet_bootstrap.cc:492] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:25.952600 16204 ts_tablet_manager.cc:1403] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.000s
I20260709 13:16:25.955442 16204 raft_consensus.cc:359] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.959569 16079 raft_consensus.cc:2468] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:25.963547 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0c303e2dd466caf0e684c6ea83284" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:25.964228 16149 raft_consensus.cc:3060] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.964964 15897 leader_election.cc:304] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:25.965874 16237 raft_consensus.cc:2804] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:25.961257 16476 consensus_queue.cc:1048] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:25.961916 16453 raft_consensus.cc:2749] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:25.966506 16237 raft_consensus.cc:697] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:25.969660 16237 consensus_queue.cc:237] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:25.973147 16386 consensus_queue.cc:1048] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:25.963599 16204 raft_consensus.cc:385] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:25.976289 16204 raft_consensus.cc:740] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:25.977087 16204 consensus_queue.cc:260] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:25.979529 16204 ts_tablet_manager.cc:1434] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.027s	user 0.002s	sys 0.004s
I20260709 13:16:25.963198 16451 leader_election.cc:290] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:25.982033 16201 tablet_bootstrap.cc:654] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:25.914539 16275 raft_consensus.cc:515] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:25.992339 16275 leader_election.cc:290] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:25.992398 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dadf33b9db824fdcbb63864c9e975c6b" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:25.993158 16008 raft_consensus.cc:3060] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:25.993650 16204 tablet_bootstrap.cc:492] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:25.996608 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dadf33b9db824fdcbb63864c9e975c6b" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:25.997339 16156 raft_consensus.cc:3060] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.001329 16204 tablet_bootstrap.cc:654] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.009783 16203 ts_tablet_manager.cc:1434] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.144s	user 0.006s	sys 0.000s
I20260709 13:16:26.017098 16203 tablet_bootstrap.cc:492] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:26.008131 16149 raft_consensus.cc:2468] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:26.026521 16230 raft_consensus.cc:493] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.027135 16230 raft_consensus.cc:515] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.038229 16202 tablet_bootstrap.cc:492] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:26.038872 16202 ts_tablet_manager.cc:1403] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.134s	user 0.014s	sys 0.001s
I20260709 13:16:26.045434 16453 raft_consensus.cc:493] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.045930 16453 raft_consensus.cc:515] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.049289 16230 leader_election.cc:290] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:26.047983 16203 tablet_bootstrap.cc:654] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.035184 16476 raft_consensus.cc:493] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.067545 16453 leader_election.cc:290] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:26.068714 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f72ec4c0d31742baa2d0d9d0aad7caed" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:26.069411 16153 raft_consensus.cc:2468] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:26.070708 16202 raft_consensus.cc:359] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.071393 16202 raft_consensus.cc:385] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.071636 16202 raft_consensus.cc:740] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.072194 16202 consensus_queue.cc:260] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.075075 16008 raft_consensus.cc:2468] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:26.065941 16476 raft_consensus.cc:515] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.076231 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f72ec4c0d31742baa2d0d9d0aad7caed" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:26.077827 16047 leader_election.cc:304] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:26.077978 16476 leader_election.cc:290] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:26.074270 16202 ts_tablet_manager.cc:1434] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.035s	user 0.002s	sys 0.001s
I20260709 13:16:26.080871 16048 leader_election.cc:304] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:26.081115 16275 raft_consensus.cc:2804] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:26.081578 16275 raft_consensus.cc:697] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:26.082365 16275 consensus_queue.cc:237] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.084571 16469 raft_consensus.cc:2804] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:26.084977 16469 raft_consensus.cc:493] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.085301 16469 raft_consensus.cc:3060] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:26.086668 16045 leader_election.cc:343] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.087502 16362 consensus_queue.cc:1048] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:26.080518 16202 tablet_bootstrap.cc:492] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:26.105700 16526 raft_consensus.cc:493] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.091380 16156 raft_consensus.cc:2468] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:26.106213 16526 raft_consensus.cc:515] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.108616 16526 leader_election.cc:290] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:26.125279 16202 tablet_bootstrap.cc:654] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.137653 16201 tablet_bootstrap.cc:492] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:26.138777 16201 ts_tablet_manager.cc:1403] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.187s	user 0.025s	sys 0.021s
I20260709 13:16:26.158887 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7a15b080c54551b2c51bee7af64600" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:26.159709 16156 raft_consensus.cc:2468] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:26.163551 16230 raft_consensus.cc:493] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.164225 16079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f72ec4c0d31742baa2d0d9d0aad7caed" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:26.164179 16230 raft_consensus.cc:515] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.167414 16230 leader_election.cc:290] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:26.167891 16526 raft_consensus.cc:493] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.168406 16526 raft_consensus.cc:515] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.174265 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f72ec4c0d31742baa2d0d9d0aad7caed" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:26.175074 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e6891e9cab4a7298664c3c431ce36d" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:26.176285 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d7d4e0bc194613bb410e793fd7bd39" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
W20260709 13:16:26.177997 16119 leader_election.cc:343] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:26.178678 16119 leader_election.cc:343] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.179786 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d7d4e0bc194613bb410e793fd7bd39" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:26.180002 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e6891e9cab4a7298664c3c431ce36d" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:26.180154 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ddd2107d2e4e73a47f9e4f6bdc73fb" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:26.182106 16119 leader_election.cc:343] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:26.185771 16121 leader_election.cc:343] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.162170 16201 raft_consensus.cc:359] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.187222 16201 raft_consensus.cc:385] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.187561 16201 raft_consensus.cc:740] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.188628 16201 consensus_queue.cc:260] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.194978 16201 ts_tablet_manager.cc:1434] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.001s
I20260709 13:16:26.186200 16121 leader_election.cc:304] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:26.214322 16201 tablet_bootstrap.cc:492] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:26.222472 16530 raft_consensus.cc:2749] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:26.222637 16121 leader_election.cc:343] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.243216 16451 raft_consensus.cc:493] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.261850 16204 tablet_bootstrap.cc:492] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:26.262363 16204 ts_tablet_manager.cc:1403] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.269s	user 0.008s	sys 0.016s
W20260709 13:16:26.258618 16121 leader_election.cc:343] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.267071 16121 leader_election.cc:304] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:26.271358 16077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7a15b080c54551b2c51bee7af64600" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:26.271656 16275 raft_consensus.cc:493] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.272192 16275 raft_consensus.cc:515] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.275542 16077 raft_consensus.cc:2468] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:26.277042 16230 raft_consensus.cc:2749] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:26.261405 16469 raft_consensus.cc:515] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.274192 16275 leader_election.cc:290] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:26.293267 16202 tablet_bootstrap.cc:492] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:26.293701 16202 ts_tablet_manager.cc:1403] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.213s	user 0.022s	sys 0.035s
I20260709 13:16:26.297215 16202 raft_consensus.cc:359] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.298034 16202 raft_consensus.cc:385] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.298388 16202 raft_consensus.cc:740] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.299199 16202 consensus_queue.cc:260] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.300140 16079 raft_consensus.cc:2393] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c95f0816c41142699d80a2e326e17ab8 in current term 1: Already voted for candidate d4d6e13a8f4a473098351ada96fd7dab in this term.
I20260709 13:16:26.301132 16202 ts_tablet_manager.cc:1434] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.002s
I20260709 13:16:26.293718 16203 tablet_bootstrap.cc:492] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:26.302907 16203 ts_tablet_manager.cc:1403] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.292s	user 0.015s	sys 0.026s
I20260709 13:16:26.264689 16451 raft_consensus.cc:515] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.303342 16202 tablet_bootstrap.cc:492] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:26.322553 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7a15b080c54551b2c51bee7af64600" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:26.311419 15974 leader_election.cc:304] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:26.323271 16006 raft_consensus.cc:2468] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:26.324137 16362 raft_consensus.cc:2804] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:26.324491 16362 raft_consensus.cc:493] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.324836 16362 raft_consensus.cc:3060] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.326303 16202 tablet_bootstrap.cc:654] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.336843 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7a15b080c54551b2c51bee7af64600" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:26.337517 16152 raft_consensus.cc:2468] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:26.338804 16451 leader_election.cc:290] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:26.339195 16048 leader_election.cc:304] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:26.340231 16469 leader_election.cc:290] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:26.340248 16275 raft_consensus.cc:2804] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:26.341001 16275 raft_consensus.cc:493] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.341336 16275 raft_consensus.cc:3060] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.342281 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28cbf604c51f48da99cb7e3089651236" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:26.342929 16152 raft_consensus.cc:2468] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:26.343333 16201 tablet_bootstrap.cc:654] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.350404 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f72ec4c0d31742baa2d0d9d0aad7caed" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:26.351063 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f72ec4c0d31742baa2d0d9d0aad7caed" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
W20260709 13:16:26.352650 16045 leader_election.cc:343] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.352947 16152 raft_consensus.cc:3060] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.353888 16275 raft_consensus.cc:515] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.356436 16204 raft_consensus.cc:359] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.357124 16204 raft_consensus.cc:385] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.357384 16204 raft_consensus.cc:740] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.357986 16204 consensus_queue.cc:260] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.358531 16120 leader_election.cc:304] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:26.360139 16203 raft_consensus.cc:359] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:26.363149 16203 raft_consensus.cc:385] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.365164 16204 ts_tablet_manager.cc:1434] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.103s	user 0.006s	sys 0.000s
I20260709 13:16:26.366477 16275 leader_election.cc:290] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:26.363420 16203 raft_consensus.cc:740] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.367929 16203 consensus_queue.cc:260] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:26.375590 16203 ts_tablet_manager.cc:1434] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.072s	user 0.002s	sys 0.005s
I20260709 13:16:26.376708 16201 tablet_bootstrap.cc:492] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:26.376561 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7a15b080c54551b2c51bee7af64600" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:26.377156 16201 ts_tablet_manager.cc:1403] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.163s	user 0.012s	sys 0.000s
I20260709 13:16:26.364140 16526 leader_election.cc:290] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:26.378497 16149 raft_consensus.cc:3060] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.380735 16203 tablet_bootstrap.cc:492] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
W20260709 13:16:26.373888 15971 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45841, user_credentials={real_user=slave}} blocked reactor thread for 44988.5us
I20260709 13:16:26.386523 16149 raft_consensus.cc:2468] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:26.387550 16048 leader_election.cc:304] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:26.388128 16469 raft_consensus.cc:2804] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:26.388576 16469 raft_consensus.cc:697] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:26.389351 16469 consensus_queue.cc:237] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.395345 16077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ddd2107d2e4e73a47f9e4f6bdc73fb" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:26.391136 15826 catalog_manager.cc:5697] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:26.373924 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7a15b080c54551b2c51bee7af64600" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:26.405140 16204 tablet_bootstrap.cc:492] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:26.419637 16203 tablet_bootstrap.cc:654] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.425431 16201 raft_consensus.cc:359] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.426132 16201 raft_consensus.cc:385] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.426421 16201 raft_consensus.cc:740] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.432535 16538 consensus_queue.cc:1048] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:26.433231 16201 consensus_queue.cc:260] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.435694 16201 ts_tablet_manager.cc:1434] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.000s
I20260709 13:16:26.436245 16375 raft_consensus.cc:493] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.436723 16375 raft_consensus.cc:515] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.437101 16451 raft_consensus.cc:493] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.437572 16451 raft_consensus.cc:515] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.437688 16362 raft_consensus.cc:515] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.438866 16006 raft_consensus.cc:2393] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d4d6e13a8f4a473098351ada96fd7dab in current term 1: Already voted for candidate 903b3435239241eab01ea54ffb015ee3 in this term.
I20260709 13:16:26.439661 16451 leader_election.cc:290] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:26.441452 16077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad97b67157f44c618ddfebbc1c25588b" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:26.441556 16079 raft_consensus.cc:3060] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.441608 16078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f72ec4c0d31742baa2d0d9d0aad7caed" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:26.442546 16078 raft_consensus.cc:2393] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bdeb23ab98c34a269f5b2e71b03c5e4e in current term 1: Already voted for candidate d4d6e13a8f4a473098351ada96fd7dab in this term.
W20260709 13:16:26.444034 15972 leader_election.cc:343] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.445350 16152 raft_consensus.cc:2468] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:26.445997 16293 raft_consensus.cc:2749] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:26.446717 15899 leader_election.cc:304] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:26.447990 16386 raft_consensus.cc:2804] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:16:26.429087 16120 leader_election.cc:343] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.448335 16048 leader_election.cc:304] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:26.448647 16120 leader_election.cc:304] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:26.449930 16539 raft_consensus.cc:2749] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:26.450528 16528 consensus_queue.cc:1048] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:26.448338 16386 raft_consensus.cc:493] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.451418 16386 raft_consensus.cc:3060] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.453207 16450 consensus_queue.cc:1048] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:26.456044 16201 tablet_bootstrap.cc:492] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:26.456187 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad97b67157f44c618ddfebbc1c25588b" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:26.456990 16153 raft_consensus.cc:2468] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:26.456377 16375 leader_election.cc:290] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:26.458284 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f72ec4c0d31742baa2d0d9d0aad7caed" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:26.459075 16152 raft_consensus.cc:2393] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bdeb23ab98c34a269f5b2e71b03c5e4e in current term 1: Already voted for candidate d4d6e13a8f4a473098351ada96fd7dab in this term.
I20260709 13:16:26.453856 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28cbf604c51f48da99cb7e3089651236" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:26.466781 16006 raft_consensus.cc:2468] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:26.464872 16293 raft_consensus.cc:493] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.471498 16293 raft_consensus.cc:515] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.473548 16204 tablet_bootstrap.cc:654] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.464658 15974 leader_election.cc:304] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:26.476455 16368 raft_consensus.cc:2804] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:26.476858 16368 raft_consensus.cc:493] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.477190 16368 raft_consensus.cc:3060] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.478843 16293 leader_election.cc:290] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:26.480686 16201 tablet_bootstrap.cc:654] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.493382 16202 tablet_bootstrap.cc:492] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:26.493903 16202 ts_tablet_manager.cc:1403] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.192s	user 0.026s	sys 0.015s
I20260709 13:16:26.501363 16386 raft_consensus.cc:515] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.508129 16535 raft_consensus.cc:2804] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:26.509738 16535 raft_consensus.cc:697] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:26.510610 16535 consensus_queue.cc:237] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.513397 16542 consensus_queue.cc:1048] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:26.517143 16077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7a15b080c54551b2c51bee7af64600" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:26.519255 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28cbf604c51f48da99cb7e3089651236" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:26.519945 16008 raft_consensus.cc:3060] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.514753 16386 leader_election.cc:290] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:26.521325 16362 leader_election.cc:290] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:26.522729 16368 raft_consensus.cc:515] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.525194 16202 raft_consensus.cc:359] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.525856 16202 raft_consensus.cc:385] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.526108 16202 raft_consensus.cc:740] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.527314 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de7a15b080c54551b2c51bee7af64600" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:26.526981 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3132182819ce40fdb691736db3dd1061" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:26.534307 16077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3132182819ce40fdb691736db3dd1061" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:26.534471 16450 consensus_queue.cc:1048] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:26.535467 16534 consensus_queue.cc:1048] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:26.539620 16008 raft_consensus.cc:2468] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:26.540719 16202 consensus_queue.cc:260] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.545796 16203 tablet_bootstrap.cc:492] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:26.546478 16203 ts_tablet_manager.cc:1403] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.169s	user 0.015s	sys 0.006s
I20260709 13:16:26.559422 16152 raft_consensus.cc:2393] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 903b3435239241eab01ea54ffb015ee3 in current term 1: Already voted for candidate d4d6e13a8f4a473098351ada96fd7dab in this term.
I20260709 13:16:26.569209 16368 leader_election.cc:290] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:26.551275 15899 leader_election.cc:304] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:26.574015 16154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad97b67157f44c618ddfebbc1c25588b" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:26.574640 16154 raft_consensus.cc:3060] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.575544 16450 raft_consensus.cc:3060] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.572674 16204 tablet_bootstrap.cc:492] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:26.577395 16204 ts_tablet_manager.cc:1403] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.172s	user 0.015s	sys 0.002s
I20260709 13:16:26.561609 15898 leader_election.cc:304] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:26.579548 15974 leader_election.cc:304] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:26.579710 16386 raft_consensus.cc:2804] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:26.580386 16386 raft_consensus.cc:697] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:26.581424 16386 consensus_queue.cc:237] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.583446 16154 raft_consensus.cc:2468] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:26.585111 16204 raft_consensus.cc:359] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.585963 16204 raft_consensus.cc:385] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:16:26.589565 16119 leader_election.cc:343] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.572186 16077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad97b67157f44c618ddfebbc1c25588b" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:26.595131 16202 ts_tablet_manager.cc:1434] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.101s	user 0.009s	sys 0.000s
I20260709 13:16:26.596345 16202 tablet_bootstrap.cc:492] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:26.586282 16204 raft_consensus.cc:740] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.598912 16204 consensus_queue.cc:260] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.603740 16204 ts_tablet_manager.cc:1434] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:16:26.604923 16204 tablet_bootstrap.cc:492] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:26.602312 16362 raft_consensus.cc:2749] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:16:26.607905 15972 leader_election.cc:343] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.614677 15974 leader_election.cc:304] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:26.616531 16362 raft_consensus.cc:2804] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:26.631408 16362 raft_consensus.cc:697] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:26.601646 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28cbf604c51f48da99cb7e3089651236" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:26.639824 16204 tablet_bootstrap.cc:654] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.640033 16203 raft_consensus.cc:359] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:26.640969 16203 raft_consensus.cc:385] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.641309 16203 raft_consensus.cc:740] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.643232 16203 consensus_queue.cc:260] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:26.649243 16203 ts_tablet_manager.cc:1434] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.102s	user 0.008s	sys 0.000s
I20260709 13:16:26.642218 16149 raft_consensus.cc:3060] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.632282 16362 consensus_queue.cc:237] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.651355 16203 tablet_bootstrap.cc:492] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:26.672232 15830 catalog_manager.cc:5697] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:26.682211 16202 tablet_bootstrap.cc:654] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.683171 16203 tablet_bootstrap.cc:654] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.684470 16534 consensus_queue.cc:1048] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:26.685302 16450 raft_consensus.cc:2749] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:16:26.694252 15898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:32879, user_credentials={real_user=slave}} blocked reactor thread for 68332.1us
I20260709 13:16:26.723757 16201 tablet_bootstrap.cc:492] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:26.724380 16201 ts_tablet_manager.cc:1403] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.269s	user 0.016s	sys 0.014s
W20260709 13:16:26.716840 16120 leader_election.cc:343] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.736050 16120 leader_election.cc:304] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:26.737066 16293 raft_consensus.cc:2749] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:26.751317 16539 raft_consensus.cc:493] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.751420 16538 consensus_queue.cc:1048] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:26.751808 16539 raft_consensus.cc:515] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.753870 16539 leader_election.cc:290] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:26.755168 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b83d43d98074cb09deac765b8b035e6" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:26.773298 16121 leader_election.cc:343] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.779495 16203 tablet_bootstrap.cc:492] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:26.779969 16203 ts_tablet_manager.cc:1403] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.130s	user 0.012s	sys 0.026s
I20260709 13:16:26.779301 16077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b83d43d98074cb09deac765b8b035e6" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:26.782545 16203 raft_consensus.cc:359] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.783183 16203 raft_consensus.cc:385] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.783470 16203 raft_consensus.cc:740] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.784173 16203 consensus_queue.cc:260] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.786286 16203 ts_tablet_manager.cc:1434] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:16:26.787186 16203 tablet_bootstrap.cc:492] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
W20260709 13:16:26.795142 15879 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:26.806720 16149 raft_consensus.cc:2468] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
W20260709 13:16:26.822999 16120 leader_election.cc:343] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.823696 16120 leader_election.cc:304] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:26.824728 16539 raft_consensus.cc:2749] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:26.832443 15829 catalog_manager.cc:5697] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:26.842985 16204 tablet_bootstrap.cc:492] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:26.843700 16204 ts_tablet_manager.cc:1403] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.239s	user 0.014s	sys 0.011s
I20260709 13:16:26.851254 16202 tablet_bootstrap.cc:492] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:26.857436 16203 tablet_bootstrap.cc:654] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.848915 16201 raft_consensus.cc:359] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.866840 16201 raft_consensus.cc:385] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.859443 16202 ts_tablet_manager.cc:1403] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.263s	user 0.011s	sys 0.018s
I20260709 13:16:26.867174 16201 raft_consensus.cc:740] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.865516 16204 raft_consensus.cc:359] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:26.870157 16204 raft_consensus.cc:385] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.871140 16204 raft_consensus.cc:740] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.871779 16204 consensus_queue.cc:260] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:26.875404 16201 consensus_queue.cc:260] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.877581 16201 ts_tablet_manager.cc:1434] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.153s	user 0.006s	sys 0.003s
I20260709 13:16:26.877449 16275 consensus_queue.cc:1048] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:26.878525 16201 tablet_bootstrap.cc:492] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:26.880298 16204 ts_tablet_manager.cc:1434] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.003s
I20260709 13:16:26.881193 16204 tablet_bootstrap.cc:492] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:26.886569 15830 catalog_manager.cc:5697] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:26.893748 16201 tablet_bootstrap.cc:654] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.903508 16386 raft_consensus.cc:493] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.904066 16386 raft_consensus.cc:515] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:26.891027 16202 raft_consensus.cc:359] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.905371 16203 tablet_bootstrap.cc:492] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:26.906333 16203 ts_tablet_manager.cc:1403] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.119s	user 0.012s	sys 0.003s
I20260709 13:16:26.908025 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd99621a84f4fe994996601dae8988c" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:26.908681 16076 raft_consensus.cc:2468] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:26.908895 16203 raft_consensus.cc:359] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.909297 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd99621a84f4fe994996601dae8988c" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:26.909586 16203 raft_consensus.cc:385] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.909979 16203 raft_consensus.cc:740] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.910734 15897 leader_election.cc:304] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:26.910732 16203 consensus_queue.cc:260] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.911724 16450 raft_consensus.cc:2804] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:26.912093 16450 raft_consensus.cc:493] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.912436 16450 raft_consensus.cc:3060] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:26.914459 16386 leader_election.cc:290] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:26.905872 16202 raft_consensus.cc:385] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.916167 16202 raft_consensus.cc:740] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
W20260709 13:16:26.916420 15898 leader_election.cc:343] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:26.917289 16469 consensus_queue.cc:1048] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:26.913595 16203 ts_tablet_manager.cc:1434] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:26.920723 16203 tablet_bootstrap.cc:492] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:26.924088 16204 tablet_bootstrap.cc:654] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.925817 15831 catalog_manager.cc:5697] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } }
I20260709 13:16:26.927713 16535 consensus_queue.cc:1048] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:26.922693 16202 consensus_queue.cc:260] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.934245 16202 ts_tablet_manager.cc:1434] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.066s	user 0.003s	sys 0.005s
I20260709 13:16:26.935453 16202 tablet_bootstrap.cc:492] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:26.943547 16201 tablet_bootstrap.cc:492] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:26.944207 16201 ts_tablet_manager.cc:1403] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.066s	user 0.004s	sys 0.005s
I20260709 13:16:26.949911 16203 tablet_bootstrap.cc:654] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.951769 16201 raft_consensus.cc:359] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.952699 16201 raft_consensus.cc:385] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.953032 16201 raft_consensus.cc:740] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.960831 16204 tablet_bootstrap.cc:492] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:26.961421 16204 ts_tablet_manager.cc:1403] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.080s	user 0.007s	sys 0.003s
I20260709 13:16:26.962532 15827 catalog_manager.cc:5697] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:26.964820 16204 raft_consensus.cc:359] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.970513 16204 raft_consensus.cc:385] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:26.970779 16204 raft_consensus.cc:740] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:26.971405 16204 consensus_queue.cc:260] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.965785 16201 consensus_queue.cc:260] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.972424 16202 tablet_bootstrap.cc:654] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:26.980212 16275 consensus_queue.cc:1048] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:26.980963 16201 ts_tablet_manager.cc:1434] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 13:16:26.981878 16201 tablet_bootstrap.cc:492] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:26.988360 16538 consensus_queue.cc:1048] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:26.989166 16293 raft_consensus.cc:493] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.989653 16293 raft_consensus.cc:515] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:26.987795 16386 raft_consensus.cc:493] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.990581 16386 raft_consensus.cc:515] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:26.992558 16386 leader_election.cc:290] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:26.993042 16386 raft_consensus.cc:493] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.993485 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:26.993816 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a3eeeeab11406e90e1fd10d63228e8" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:26.994417 16156 raft_consensus.cc:2468] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:26.994864 16293 leader_election.cc:290] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:26.997355 16275 raft_consensus.cc:493] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.997963 16078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a3eeeeab11406e90e1fd10d63228e8" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:26.997833 16275 raft_consensus.cc:515] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:26.998652 16078 raft_consensus.cc:2468] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:26.999305 15899 leader_election.cc:304] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:27.000129 16538 raft_consensus.cc:2804] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:16:27.000267 16119 leader_election.cc:343] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.000578 16538 raft_consensus.cc:493] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.001026 16538 raft_consensus.cc:3060] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.002942 16204 ts_tablet_manager.cc:1434] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.002s
I20260709 13:16:27.004005 16204 tablet_bootstrap.cc:492] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:27.007543 16275 leader_election.cc:290] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:27.007577 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad97b67157f44c618ddfebbc1c25588b" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:27.008494 16156 raft_consensus.cc:2393] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4d6e13a8f4a473098351ada96fd7dab in current term 1: Already voted for candidate 903b3435239241eab01ea54ffb015ee3 in this term.
I20260709 13:16:27.009497 16204 tablet_bootstrap.cc:654] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.009349 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:27.008332 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad97b67157f44c618ddfebbc1c25588b" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:27.011734 16201 tablet_bootstrap.cc:654] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:27.011883 16121 leader_election.cc:343] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.012342 16121 leader_election.cc:304] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:26.993521 16386 raft_consensus.cc:515] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:27.016603 16386 leader_election.cc:290] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:27.016885 16538 raft_consensus.cc:515] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:27.018208 16584 raft_consensus.cc:2749] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:27.020228 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e6891e9cab4a7298664c3c431ce36d" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:27.024883 16156 raft_consensus.cc:2468] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:27.022882 16078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a3eeeeab11406e90e1fd10d63228e8" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:27.025475 16078 raft_consensus.cc:3060] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.027376 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e6891e9cab4a7298664c3c431ce36d" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:27.028005 16005 raft_consensus.cc:2468] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:27.030937 15898 leader_election.cc:304] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:27.031884 16047 leader_election.cc:304] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:27.032636 16562 raft_consensus.cc:2804] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:27.033074 16562 raft_consensus.cc:493] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.033200 16469 raft_consensus.cc:3060] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.033463 16562 raft_consensus.cc:3060] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.037948 16078 raft_consensus.cc:2468] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:27.040211 15897 leader_election.cc:304] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:26.956122 16450 raft_consensus.cc:515] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.045737 16386 raft_consensus.cc:2804] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.046479 16386 raft_consensus.cc:697] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:27.047346 16386 consensus_queue.cc:237] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:27.048465 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd99621a84f4fe994996601dae8988c" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:27.049039 16076 raft_consensus.cc:3060] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.045277 16469 raft_consensus.cc:2749] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:27.071354 16450 leader_election.cc:290] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:27.088186 16538 leader_election.cc:290] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:27.076700 16562 raft_consensus.cc:515] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:27.103600 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a3eeeeab11406e90e1fd10d63228e8" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:27.104414 16156 raft_consensus.cc:3060] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.109030 16204 tablet_bootstrap.cc:492] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:27.109650 16204 ts_tablet_manager.cc:1403] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.106s	user 0.009s	sys 0.013s
I20260709 13:16:27.113381 16202 tablet_bootstrap.cc:492] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:27.113842 16202 ts_tablet_manager.cc:1403] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.179s	user 0.016s	sys 0.015s
I20260709 13:16:27.116865 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd99621a84f4fe994996601dae8988c" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:27.119532 16201 tablet_bootstrap.cc:492] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:27.119982 16201 ts_tablet_manager.cc:1403] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.138s	user 0.017s	sys 0.004s
I20260709 13:16:27.122673 16076 raft_consensus.cc:2468] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:27.122565 16201 raft_consensus.cc:359] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.123201 16201 raft_consensus.cc:385] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.123474 16201 raft_consensus.cc:740] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.091857 16237 consensus_queue.cc:1048] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.034s
I20260709 13:16:27.124135 16201 consensus_queue.cc:260] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
W20260709 13:16:27.124876 15898 leader_election.cc:343] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.126235 16201 ts_tablet_manager.cc:1434] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:16:27.126497 16562 leader_election.cc:290] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:27.127945 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e6891e9cab4a7298664c3c431ce36d" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:27.128533 16006 raft_consensus.cc:3060] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.130784 16201 tablet_bootstrap.cc:492] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:27.132388 15897 leader_election.cc:304] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3
I20260709 13:16:27.134562 16202 raft_consensus.cc:359] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:27.138491 16202 raft_consensus.cc:385] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.138865 16202 raft_consensus.cc:740] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.140136 16202 consensus_queue.cc:260] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:27.136987 16201 tablet_bootstrap.cc:654] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.155378 16202 ts_tablet_manager.cc:1434] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 13:16:27.156354 16202 tablet_bootstrap.cc:492] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:27.158880 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e6891e9cab4a7298664c3c431ce36d" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:27.159543 16151 raft_consensus.cc:3060] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.161919 16562 raft_consensus.cc:2804] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.164726 16562 raft_consensus.cc:697] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:27.157848 16204 raft_consensus.cc:359] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.167147 16204 raft_consensus.cc:385] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.174649 16204 raft_consensus.cc:740] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.175360 16204 consensus_queue.cc:260] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.175895 16203 tablet_bootstrap.cc:492] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:27.176573 16203 ts_tablet_manager.cc:1403] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.256s	user 0.033s	sys 0.028s
I20260709 13:16:27.188077 16156 raft_consensus.cc:2468] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:27.193550 16204 ts_tablet_manager.cc:1434] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.084s	user 0.010s	sys 0.001s
I20260709 13:16:27.194702 16204 tablet_bootstrap.cc:492] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:27.165547 16562 consensus_queue.cc:237] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.211566 16375 consensus_queue.cc:1048] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:27.216506 16202 tablet_bootstrap.cc:654] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.219947 16151 raft_consensus.cc:2468] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:27.221815 15899 leader_election.cc:304] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:27.222759 16520 raft_consensus.cc:2804] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.223208 16520 raft_consensus.cc:697] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:27.228502 16520 consensus_queue.cc:237] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:27.237782 16203 raft_consensus.cc:359] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:27.238794 16203 raft_consensus.cc:385] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.239122 16203 raft_consensus.cc:740] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.240217 16203 consensus_queue.cc:260] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:27.246861 16362 raft_consensus.cc:493] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.247383 16362 raft_consensus.cc:515] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:27.249239 16203 ts_tablet_manager.cc:1434] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.072s	user 0.005s	sys 0.003s
I20260709 13:16:27.253082 16006 raft_consensus.cc:2468] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:27.257061 16362 leader_election.cc:290] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:27.257688 16077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ddd2107d2e4e73a47f9e4f6bdc73fb" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:27.258846 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ddd2107d2e4e73a47f9e4f6bdc73fb" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:27.264111 16077 raft_consensus.cc:2468] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:27.265221 16149 raft_consensus.cc:2468] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:27.268898 15972 leader_election.cc:304] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:27.271370 16542 raft_consensus.cc:2804] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:27.272338 16542 raft_consensus.cc:493] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.272660 16542 raft_consensus.cc:3060] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.274153 16204 tablet_bootstrap.cc:654] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.275396 15829 catalog_manager.cc:5697] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:27.288470 16203 tablet_bootstrap.cc:492] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:27.290254 16201 tablet_bootstrap.cc:492] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:27.290750 16201 ts_tablet_manager.cc:1403] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.160s	user 0.016s	sys 0.005s
I20260709 13:16:27.298697 16201 raft_consensus.cc:359] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.299350 16201 raft_consensus.cc:385] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.299609 16201 raft_consensus.cc:740] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.300335 16201 consensus_queue.cc:260] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.305634 16201 ts_tablet_manager.cc:1434] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:16:27.306579 16201 tablet_bootstrap.cc:492] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:27.303653 16203 tablet_bootstrap.cc:654] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.307437 16375 consensus_queue.cc:1048] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:27.323885 15830 catalog_manager.cc:5697] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:27.326014 15830 catalog_manager.cc:5697] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:27.327346 16202 tablet_bootstrap.cc:492] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:27.327860 16202 ts_tablet_manager.cc:1403] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.172s	user 0.016s	sys 0.001s
I20260709 13:16:27.334263 16202 raft_consensus.cc:359] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:27.334913 16202 raft_consensus.cc:385] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.335187 16202 raft_consensus.cc:740] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.335774 16202 consensus_queue.cc:260] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:27.337687 16201 tablet_bootstrap.cc:654] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.352285 16202 ts_tablet_manager.cc:1434] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:16:27.354024 16202 tablet_bootstrap.cc:492] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:27.361133 16542 raft_consensus.cc:515] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:27.383147 16079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ddd2107d2e4e73a47f9e4f6bdc73fb" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:27.383977 16079 raft_consensus.cc:3060] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.388847 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ddd2107d2e4e73a47f9e4f6bdc73fb" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:27.389443 16153 raft_consensus.cc:3060] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.391605 16542 leader_election.cc:290] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:27.396718 16153 raft_consensus.cc:2468] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:27.397933 15974 leader_election.cc:304] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:27.398813 16542 raft_consensus.cc:2804] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.399308 16542 raft_consensus.cc:697] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:27.400137 16542 consensus_queue.cc:237] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:27.417727 16202 tablet_bootstrap.cc:654] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.436615 16203 tablet_bootstrap.cc:492] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:27.437165 16203 ts_tablet_manager.cc:1403] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.149s	user 0.049s	sys 0.025s
I20260709 13:16:27.445164 16584 raft_consensus.cc:493] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.445676 16584 raft_consensus.cc:515] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:27.448810 16584 leader_election.cc:290] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:27.449360 16079 raft_consensus.cc:2468] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:27.450492 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58eccbbe71e465188784170045cdb34" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:27.454772 16077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58eccbbe71e465188784170045cdb34" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:27.456911 16203 raft_consensus.cc:359] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
W20260709 13:16:27.457175 16120 leader_election.cc:343] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.457652 16203 raft_consensus.cc:385] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.458007 16203 raft_consensus.cc:740] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.458688 16203 consensus_queue.cc:260] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
W20260709 13:16:27.458678 16119 leader_election.cc:343] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.460907 16119 leader_election.cc:304] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:27.462536 16203 ts_tablet_manager.cc:1434] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.002s
I20260709 13:16:27.462515 16584 raft_consensus.cc:2749] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:27.463686 16203 tablet_bootstrap.cc:492] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:27.463894 16201 tablet_bootstrap.cc:492] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:27.464378 16201 ts_tablet_manager.cc:1403] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.158s	user 0.011s	sys 0.012s
I20260709 13:16:27.466449 16204 tablet_bootstrap.cc:492] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:27.466928 16204 ts_tablet_manager.cc:1403] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.272s	user 0.044s	sys 0.051s
I20260709 13:16:27.469827 16201 raft_consensus.cc:359] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:27.470455 16201 raft_consensus.cc:385] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.470708 16201 raft_consensus.cc:740] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
W20260709 13:16:27.567525 15972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:32795, user_credentials={real_user=slave}} blocked reactor thread for 58799.9us
I20260709 13:16:27.473301 16204 raft_consensus.cc:359] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:27.570117 16204 raft_consensus.cc:385] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.570514 16204 raft_consensus.cc:740] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.571257 16204 consensus_queue.cc:260] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:27.573457 16204 ts_tablet_manager.cc:1434] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.106s	user 0.013s	sys 0.000s
I20260709 13:16:27.578555 15826 catalog_manager.cc:5697] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:27.474081 16201 consensus_queue.cc:260] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:27.570219 16203 tablet_bootstrap.cc:654] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.579457 16204 tablet_bootstrap.cc:492] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:27.586238 16528 raft_consensus.cc:493] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.586637 16237 raft_consensus.cc:493] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.586777 16528 raft_consensus.cc:515] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.587123 16237 raft_consensus.cc:515] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.588744 16528 leader_election.cc:290] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:27.589815 16237 leader_election.cc:290] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:27.590025 16582 raft_consensus.cc:493] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.590575 16582 raft_consensus.cc:515] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:27.592816 16201 ts_tablet_manager.cc:1434] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.128s	user 0.015s	sys 0.023s
I20260709 13:16:27.593760 16201 tablet_bootstrap.cc:492] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:27.594532 16542 raft_consensus.cc:493] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.595059 16202 tablet_bootstrap.cc:492] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:27.594990 16542 raft_consensus.cc:515] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:27.595543 16202 ts_tablet_manager.cc:1403] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.242s	user 0.015s	sys 0.004s
I20260709 13:16:27.597329 16542 leader_election.cc:290] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:27.601425 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca16f452c2a4b19a53a12119268b92d" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:27.602061 16005 raft_consensus.cc:2468] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:27.602928 16582 leader_election.cc:290] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:27.603767 15898 leader_election.cc:304] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:27.605472 16451 raft_consensus.cc:2804] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:27.605545 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca16f452c2a4b19a53a12119268b92d" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:27.605906 16451 raft_consensus.cc:493] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.606266 16451 raft_consensus.cc:3060] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.606855 16203 tablet_bootstrap.cc:492] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:27.607839 16203 ts_tablet_manager.cc:1403] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.144s	user 0.014s	sys 0.001s
I20260709 13:16:27.626623 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b83d43d98074cb09deac765b8b035e6" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:27.627239 16150 raft_consensus.cc:2468] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
I20260709 13:16:27.628805 16528 raft_consensus.cc:493] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.629279 16528 raft_consensus.cc:515] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.598327 16202 raft_consensus.cc:359] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:27.631038 16202 raft_consensus.cc:385] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.631295 16202 raft_consensus.cc:740] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.632352 16528 leader_election.cc:290] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:27.613409 16451 raft_consensus.cc:515] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.619253 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4332ee7a936341baad672bfc16703cc7" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:27.632848 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4332ee7a936341baad672bfc16703cc7" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:27.633391 15932 raft_consensus.cc:2393] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 903b3435239241eab01ea54ffb015ee3 in current term 1: Already voted for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in this term.
I20260709 13:16:27.634881 16202 consensus_queue.cc:260] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:27.635676 16451 leader_election.cc:290] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
W20260709 13:16:27.637043 16121 leader_election.cc:343] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.639336 16202 ts_tablet_manager.cc:1434] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 13:16:27.641985 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca16f452c2a4b19a53a12119268b92d" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:27.642553 16005 raft_consensus.cc:3060] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.644477 15974 leader_election.cc:304] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:27.646090 16204 tablet_bootstrap.cc:654] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.646914 16202 tablet_bootstrap.cc:492] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8: Bootstrap starting.
I20260709 13:16:27.647778 16375 raft_consensus.cc:2804] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:27.648188 16375 raft_consensus.cc:493] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.648524 16375 raft_consensus.cc:3060] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.649706 16005 raft_consensus.cc:2468] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:27.652251 15898 leader_election.cc:304] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:27.653316 16451 raft_consensus.cc:2804] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.653741 16451 raft_consensus.cc:697] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:27.654392 16203 raft_consensus.cc:359] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:27.654528 16451 consensus_queue.cc:237] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.655079 16203 raft_consensus.cc:385] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.655352 16203 raft_consensus.cc:740] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.656068 16203 consensus_queue.cc:260] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:27.657090 16375 raft_consensus.cc:515] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
W20260709 13:16:27.664125 16119 leader_election.cc:343] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.664702 16119 leader_election.cc:304] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:27.670077 16375 leader_election.cc:290] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:27.671211 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd99621a84f4fe994996601dae8988c" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:27.673189 16609 raft_consensus.cc:2749] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:27.682757 16078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca16f452c2a4b19a53a12119268b92d" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:27.683341 16078 raft_consensus.cc:2468] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:27.684939 16078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca16f452c2a4b19a53a12119268b92d" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:27.685421 16078 raft_consensus.cc:3060] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.689332 16451 raft_consensus.cc:493] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.689803 16451 raft_consensus.cc:515] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:27.691071 16201 tablet_bootstrap.cc:654] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.692221 16451 leader_election.cc:290] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:27.693213 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d7d4e0bc194613bb410e793fd7bd39" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:27.694451 15898 leader_election.cc:343] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.696421 16203 ts_tablet_manager.cc:1434] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.073s	user 0.001s	sys 0.018s
I20260709 13:16:27.697440 16082 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b83d43d98074cb09deac765b8b035e6" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:27.698149 16079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca16f452c2a4b19a53a12119268b92d" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:27.698740 16203 tablet_bootstrap.cc:492] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:27.712025 16079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd99621a84f4fe994996601dae8988c" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:27.712738 16079 raft_consensus.cc:2393] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 903b3435239241eab01ea54ffb015ee3 in current term 1: Already voted for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in this term.
I20260709 13:16:27.716857 16079 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b83d43d98074cb09deac765b8b035e6" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:27.719626 16150 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b83d43d98074cb09deac765b8b035e6" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:27.720494 16150 raft_consensus.cc:3060] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.724284 16203 tablet_bootstrap.cc:654] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.726981 16562 consensus_queue.cc:1048] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:27.731405 16237 consensus_queue.cc:1048] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:27.734531 16005 raft_consensus.cc:3060] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.746886 16520 consensus_queue.cc:1048] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:27.750697 15826 catalog_manager.cc:5697] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:27.755631 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d7d4e0bc194613bb410e793fd7bd39" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:27.756217 16156 raft_consensus.cc:2468] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:27.760296 16150 raft_consensus.cc:2468] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:27.761405 15974 leader_election.cc:304] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:27.762184 16375 raft_consensus.cc:2804] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:27.762684 16375 raft_consensus.cc:697] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:27.763576 16375 consensus_queue.cc:237] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
W20260709 13:16:27.774937 15972 leader_election.cc:343] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.776885 15972 leader_election.cc:304] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:27.777664 16542 raft_consensus.cc:2749] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:27.779964 16451 consensus_queue.cc:1048] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:27.780663 16520 consensus_queue.cc:1048] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:27.783643 16078 raft_consensus.cc:2468] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:27.788367 16237 consensus_queue.cc:1048] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:27.789559 15899 leader_election.cc:304] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3
I20260709 13:16:27.790529 16450 raft_consensus.cc:2804] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:27.791029 16450 raft_consensus.cc:493] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.791450 16450 raft_consensus.cc:3060] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.804517 16201 tablet_bootstrap.cc:492] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:27.805039 16201 ts_tablet_manager.cc:1403] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.211s	user 0.039s	sys 0.049s
I20260709 13:16:27.806083 16450 raft_consensus.cc:515] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:27.807538 16201 raft_consensus.cc:359] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:27.808213 16201 raft_consensus.cc:385] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.808468 16201 raft_consensus.cc:740] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.809223 16450 leader_election.cc:290] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:27.809090 16201 consensus_queue.cc:260] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:27.810233 15972 leader_election.cc:304] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:27.811502 16375 raft_consensus.cc:2749] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:27.812587 15831 catalog_manager.cc:5697] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:27.829123 16201 ts_tablet_manager.cc:1434] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.004s
I20260709 13:16:27.830302 16201 tablet_bootstrap.cc:492] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:27.860111 16203 tablet_bootstrap.cc:492] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:27.860610 16203 ts_tablet_manager.cc:1403] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.162s	user 0.014s	sys 0.016s
I20260709 13:16:27.860877 16202 tablet_bootstrap.cc:654] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.846661 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d7d4e0bc194613bb410e793fd7bd39" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
W20260709 13:16:27.872021 15879 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:27.903759 16203 raft_consensus.cc:359] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.904736 16203 raft_consensus.cc:385] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.905064 16203 raft_consensus.cc:740] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.905859 16203 consensus_queue.cc:260] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.874493 16005 raft_consensus.cc:3060] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.867460 16607 raft_consensus.cc:493] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.872727 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d7d4e0bc194613bb410e793fd7bd39" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
W20260709 13:16:27.927140 15972 leader_election.cc:343] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:27.927765 16204 tablet_bootstrap.cc:492] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:27.928411 16204 ts_tablet_manager.cc:1403] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.349s	user 0.022s	sys 0.033s
I20260709 13:16:27.937713 16005 raft_consensus.cc:2468] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:27.939901 16368 raft_consensus.cc:493] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.940356 16368 raft_consensus.cc:515] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:27.938798 16204 raft_consensus.cc:359] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.942296 16204 raft_consensus.cc:385] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:27.942692 16204 raft_consensus.cc:740] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:27.942986 16201 tablet_bootstrap.cc:654] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:27.943562 16204 consensus_queue.cc:260] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:27.952335 16204 ts_tablet_manager.cc:1434] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:16:27.953462 16203 ts_tablet_manager.cc:1434] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.093s	user 0.010s	sys 0.000s
I20260709 13:16:27.954123 16204 tablet_bootstrap.cc:492] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:27.944172 16149 raft_consensus.cc:3060] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:27.959056 16203 tablet_bootstrap.cc:492] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:27.944482 16607 raft_consensus.cc:515] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:27.963817 15898 leader_election.cc:304] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:28.003170 16469 raft_consensus.cc:493] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:28.005393 15972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:32795, user_credentials={real_user=slave}} blocked reactor thread for 76745.5us
I20260709 13:16:28.022162 16520 raft_consensus.cc:2804] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:16:28.027274 15898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:32879, user_credentials={real_user=slave}} blocked reactor thread for 69987.3us
I20260709 13:16:28.028735 16362 consensus_queue.cc:1048] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:16:28.033769 16203 tablet_bootstrap.cc:654] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.039433 16149 raft_consensus.cc:2468] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:28.051960 16204 tablet_bootstrap.cc:654] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.073405 16562 raft_consensus.cc:493] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.075548 16202 tablet_bootstrap.cc:492] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8: No bootstrap required, opened a new log
I20260709 13:16:28.076196 16202 ts_tablet_manager.cc:1403] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8: Time spent bootstrapping tablet: real 0.429s	user 0.011s	sys 0.018s
I20260709 13:16:28.017789 16469 raft_consensus.cc:515] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:28.088167 16469 leader_election.cc:290] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:28.096004 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3132182819ce40fdb691736db3dd1061" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:28.098657 16237 consensus_queue.cc:1048] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:28.029177 16520 raft_consensus.cc:697] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:28.112192 16202 raft_consensus.cc:359] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:28.113147 16202 raft_consensus.cc:385] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.113487 16202 raft_consensus.cc:740] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.073944 16562 raft_consensus.cc:515] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:28.121510 16202 consensus_queue.cc:260] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:28.124022 16154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3132182819ce40fdb691736db3dd1061" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
W20260709 13:16:28.137544 16045 leader_election.cc:343] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:28.141867 16154 raft_consensus.cc:2468] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:28.143234 16048 leader_election.cc:304] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:28.141316 16156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d7d4e0bc194613bb410e793fd7bd39" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:28.145448 16156 raft_consensus.cc:2468] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:28.166380 16611 raft_consensus.cc:2804] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:28.170542 16202 ts_tablet_manager.cc:1434] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8: Time spent starting tablet: real 0.094s	user 0.009s	sys 0.001s
I20260709 13:16:28.169610 16611 raft_consensus.cc:493] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.175577 16607 leader_election.cc:290] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:28.179950 16611 raft_consensus.cc:3060] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.178295 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3424ea4fd86e4453a5d9fde62fc43e7f" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:28.178438 16368 leader_election.cc:290] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:28.250808 16201 tablet_bootstrap.cc:492] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:28.251266 16201 ts_tablet_manager.cc:1403] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.421s	user 0.032s	sys 0.014s
I20260709 13:16:28.169721 16520 consensus_queue.cc:237] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:28.256161 16203 tablet_bootstrap.cc:492] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:28.256592 16203 ts_tablet_manager.cc:1403] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.298s	user 0.008s	sys 0.014s
I20260709 13:16:28.261283 16469 raft_consensus.cc:493] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.261817 16469 raft_consensus.cc:515] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:28.263820 16469 leader_election.cc:290] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:28.242693 16646 raft_consensus.cc:493] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.265973 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d7d4e0bc194613bb410e793fd7bd39" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
W20260709 13:16:28.241457 16119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45841, user_credentials={real_user=slave}} blocked reactor thread for 76253.8us
I20260709 13:16:28.277222 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6cabb92769049d088d88c54ec482c85" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:28.278159 16562 leader_election.cc:290] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:28.307353 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6cabb92769049d088d88c54ec482c85" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:28.308022 15932 raft_consensus.cc:2468] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:28.310755 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6cabb92769049d088d88c54ec482c85" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:28.314705 16121 leader_election.cc:343] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:28.314733 16073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3424ea4fd86e4453a5d9fde62fc43e7f" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
W20260709 13:16:28.333798 16120 leader_election.cc:343] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:28.334281 16120 leader_election.cc:304] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:28.273249 16646 raft_consensus.cc:515] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:28.318946 16078 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6cabb92769049d088d88c54ec482c85" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:28.325644 16632 consensus_queue.cc:1048] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:16:28.350970 16607 raft_consensus.cc:2749] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:28.348620 16078 raft_consensus.cc:2468] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:28.416803 16646 leader_election.cc:290] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:28.419884 16203 raft_consensus.cc:359] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:28.420608 16203 raft_consensus.cc:385] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.420859 16203 raft_consensus.cc:740] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.421456 16203 consensus_queue.cc:260] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:28.351512 16611 raft_consensus.cc:515] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:28.429664 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a7a7d029fb040b881d5165c98065a12" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:28.430490 16611 leader_election.cc:290] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:28.360883 16045 leader_election.cc:304] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
W20260709 13:16:28.411777 15972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:32795, user_credentials={real_user=slave}} blocked reactor thread for 46068.3us
I20260709 13:16:28.443368 16203 ts_tablet_manager.cc:1434] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.187s	user 0.006s	sys 0.001s
I20260709 13:16:28.451241 16614 raft_consensus.cc:2804] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:28.451730 16614 raft_consensus.cc:493] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.452059 16614 raft_consensus.cc:3060] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:28.451287 16045 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:45841, user_credentials={real_user=slave}} blocked reactor thread for 99231.9us
W20260709 13:16:28.523744 16120 leader_election.cc:343] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:28.524652 16201 raft_consensus.cc:359] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:28.525259 16201 raft_consensus.cc:385] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.525484 16201 raft_consensus.cc:740] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.528172 16201 consensus_queue.cc:260] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
W20260709 13:16:28.487207 15898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:32879, user_credentials={real_user=slave}} blocked reactor thread for 146141us
I20260709 13:16:28.532773 16201 ts_tablet_manager.cc:1434] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.281s	user 0.007s	sys 0.007s
I20260709 13:16:28.539407 16201 tablet_bootstrap.cc:492] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:28.547111 16204 tablet_bootstrap.cc:492] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:28.547624 16204 ts_tablet_manager.cc:1403] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.595s	user 0.041s	sys 0.002s
I20260709 13:16:28.550112 16204 raft_consensus.cc:359] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:28.550786 16204 raft_consensus.cc:385] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.551051 16204 raft_consensus.cc:740] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.552729 16201 tablet_bootstrap.cc:654] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.553913 15897 leader_election.cc:304] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:28.552011 16204 consensus_queue.cc:260] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:28.558066 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3132182819ce40fdb691736db3dd1061" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:28.560328 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3132182819ce40fdb691736db3dd1061" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:28.563581 16614 raft_consensus.cc:515] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:28.565112 15932 raft_consensus.cc:3060] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.567873 16153 raft_consensus.cc:3060] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.591617 15932 raft_consensus.cc:2468] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:28.598035 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6cabb92769049d088d88c54ec482c85" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:28.595906 16614 leader_election.cc:290] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:28.598881 16008 raft_consensus.cc:3060] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.601887 15974 leader_election.cc:304] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:28.608264 16203 tablet_bootstrap.cc:492] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:28.614104 16626 raft_consensus.cc:2804] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:28.614563 16626 raft_consensus.cc:493] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.614905 16626 raft_consensus.cc:3060] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:28.635025 15971 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45841, user_credentials={real_user=slave}} blocked reactor thread for 93043.4us
I20260709 13:16:28.636445 16203 tablet_bootstrap.cc:654] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:28.638366 15898 leader_election.cc:343] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:28.639140 15898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:32879, user_credentials={real_user=slave}} blocked reactor thread for 47859.8us
I20260709 13:16:28.645534 16153 raft_consensus.cc:2468] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:28.646750 16048 leader_election.cc:304] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
W20260709 13:16:28.589113 16654 log.cc:927] Time spent T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e: Append to log took a long time: real 0.187s	user 0.010s	sys 0.000s
I20260709 13:16:28.647511 16614 raft_consensus.cc:2804] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:28.647985 16614 raft_consensus.cc:697] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:28.648772 16634 consensus_queue.cc:1048] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.075s
I20260709 13:16:28.649049 16614 consensus_queue.cc:237] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:28.655375 16008 raft_consensus.cc:2468] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
W20260709 13:16:28.662767 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.68:36069, user_credentials={real_user=slave}} blocked reactor thread for 60944.4us
I20260709 13:16:28.663064 16637 raft_consensus.cc:2804] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:28.663448 16637 raft_consensus.cc:493] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.663892 16637 raft_consensus.cc:3060] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.683809 16368 consensus_queue.cc:1048] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:28.684511 16006 raft_consensus.cc:2376] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1. Re-sending same reply.
I20260709 13:16:28.683673 16626 raft_consensus.cc:515] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
W20260709 13:16:28.650573 16045 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45841, user_credentials={real_user=slave}} blocked reactor thread for 61210.3us
I20260709 13:16:28.696631 16626 leader_election.cc:290] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:28.705346 16204 ts_tablet_manager.cc:1434] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.157s	user 0.007s	sys 0.003s
I20260709 13:16:28.711236 16204 tablet_bootstrap.cc:492] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:28.727378 16201 tablet_bootstrap.cc:492] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:28.727991 16201 ts_tablet_manager.cc:1403] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.189s	user 0.026s	sys 0.024s
W20260709 13:16:28.710853 15972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:32795, user_credentials={real_user=slave}} blocked reactor thread for 169352us
I20260709 13:16:28.731922 16081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6cabb92769049d088d88c54ec482c85" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:28.732959 16081 raft_consensus.cc:2393] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bdeb23ab98c34a269f5b2e71b03c5e4e in current term 1: Already voted for candidate d4d6e13a8f4a473098351ada96fd7dab in this term.
I20260709 13:16:28.733567 16203 tablet_bootstrap.cc:492] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:28.734077 16203 ts_tablet_manager.cc:1403] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.126s	user 0.007s	sys 0.015s
I20260709 13:16:28.738247 16047 leader_election.cc:304] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:28.741734 16204 tablet_bootstrap.cc:654] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.748699 16612 raft_consensus.cc:2804] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:28.751993 16612 raft_consensus.cc:697] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:28.753655 16612 consensus_queue.cc:237] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:28.760248 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6cabb92769049d088d88c54ec482c85" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:28.761030 15933 raft_consensus.cc:2393] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d4d6e13a8f4a473098351ada96fd7dab in current term 1: Already voted for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in this term.
I20260709 13:16:28.760280 16203 raft_consensus.cc:359] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:28.762914 16203 raft_consensus.cc:385] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.763175 16203 raft_consensus.cc:740] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.775897 16203 consensus_queue.cc:260] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:28.799042 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a7a7d029fb040b881d5165c98065a12" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:28.800668 15831 catalog_manager.cc:5697] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.814483 16203 ts_tablet_manager.cc:1434] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.000s
I20260709 13:16:28.806120 16614 raft_consensus.cc:493] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.819075 16614 raft_consensus.cc:515] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:28.821031 16637 raft_consensus.cc:515] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:28.821702 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58eccbbe71e465188784170045cdb34" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:28.822436 16151 raft_consensus.cc:2468] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:28.831027 16201 raft_consensus.cc:359] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:28.831923 16201 raft_consensus.cc:385] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:28.832259 16201 raft_consensus.cc:740] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:28.833071 16201 consensus_queue.cc:260] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:28.847411 16203 tablet_bootstrap.cc:492] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:28.854000 16201 ts_tablet_manager.cc:1434] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.126s	user 0.007s	sys 0.001s
W20260709 13:16:28.805349 15898 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:32879, user_credentials={real_user=slave}} blocked reactor thread for 58789.9us
I20260709 13:16:28.821090 16614 leader_election.cc:290] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:28.876154 15972 consensus_peers.cc:597] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 -> Peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Couldn't send request to peer d4d6e13a8f4a473098351ada96fd7dab. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:28.864061 16119 leader_election.cc:343] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:28.886365 16119 leader_election.cc:304] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:28.887250 16582 raft_consensus.cc:2749] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:28.887974 15830 catalog_manager.cc:5697] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:28.907086 16048 leader_election.cc:304] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:28.909240 16237 consensus_queue.cc:1048] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:28.921501 16204 tablet_bootstrap.cc:492] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:28.921973 16204 ts_tablet_manager.cc:1403] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.211s	user 0.009s	sys 0.026s
I20260709 13:16:28.920701 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58eccbbe71e465188784170045cdb34" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:28.924479 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d7d4e0bc194613bb410e793fd7bd39" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:28.918586 16649 raft_consensus.cc:2804] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:28.939263 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6cabb92769049d088d88c54ec482c85" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:28.940052 16008 raft_consensus.cc:2393] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bdeb23ab98c34a269f5b2e71b03c5e4e in current term 1: Already voted for candidate d4d6e13a8f4a473098351ada96fd7dab in this term.
I20260709 13:16:28.925094 16155 raft_consensus.cc:3060] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:28.957631 16203 tablet_bootstrap.cc:654] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:28.931358 16649 raft_consensus.cc:493] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.958601 16649 raft_consensus.cc:3060] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:28.963749 16201 tablet_bootstrap.cc:492] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:28.977190 15929 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d7d4e0bc194613bb410e793fd7bd39" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:28.977658 16637 leader_election.cc:290] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:28.980741 15898 leader_election.cc:304] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:28.981557 16450 raft_consensus.cc:2749] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:16:28.983685 16666 log.cc:927] Time spent T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:16:28.990969 16006 raft_consensus.cc:1240] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Rejecting Update request from peer bdeb23ab98c34a269f5b2e71b03c5e4e for earlier term 1. Current term is 2. Ops: []
W20260709 13:16:28.994823 16045 leader_election.cc:343] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:28.997867 16201 tablet_bootstrap.cc:654] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.003167 16155 raft_consensus.cc:2468] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 2.
I20260709 13:16:29.003741 16204 raft_consensus.cc:359] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:29.010008 16204 raft_consensus.cc:385] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.010426 16204 raft_consensus.cc:740] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.005167 16649 raft_consensus.cc:515] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:29.011530 16204 consensus_queue.cc:260] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:29.018978 15974 leader_election.cc:304] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:29.021807 16204 ts_tablet_manager.cc:1434] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.100s	user 0.008s	sys 0.000s
I20260709 13:16:29.023021 16204 tablet_bootstrap.cc:492] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:29.020146 16528 raft_consensus.cc:2804] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:29.015563 16149 raft_consensus.cc:1240] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Rejecting Update request from peer bdeb23ab98c34a269f5b2e71b03c5e4e for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:16:29.025568 16528 raft_consensus.cc:697] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:29.026669 16528 consensus_queue.cc:237] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.028554 16237 consensus_queue.cc:1059] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:29.031834 15879 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:29.037853 16237 raft_consensus.cc:3055] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:29.038213 16237 raft_consensus.cc:740] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:29.048534 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58eccbbe71e465188784170045cdb34" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:29.049325 16149 raft_consensus.cc:3060] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.044989 16237 consensus_queue.cc:260] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.058895 16237 raft_consensus.cc:3060] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:29.062390 16520 consensus_queue.cc:1175] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "903b3435239241eab01ea54ffb015ee3" responder_term: 2 status { last_received { term: 0 index: 0 } last_committed_idx: 0 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer bdeb23ab98c34a269f5b2e71b03c5e4e for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:16:29.044044 16681 log.cc:927] Time spent T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:16:29.070071 16204 tablet_bootstrap.cc:654] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.080817 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58eccbbe71e465188784170045cdb34" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:29.081379 15933 raft_consensus.cc:3060] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.083911 16649 leader_election.cc:290] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:29.095098 15933 raft_consensus.cc:2468] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:29.099328 16203 tablet_bootstrap.cc:492] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:29.099783 16203 ts_tablet_manager.cc:1403] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.264s	user 0.013s	sys 0.010s
I20260709 13:16:29.101984 16651 raft_consensus.cc:493] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.102471 16651 raft_consensus.cc:515] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:29.104447 16651 leader_election.cc:290] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
W20260709 13:16:29.105795 15972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:32795, user_credentials={real_user=slave}} blocked reactor thread for 52245.6us
I20260709 13:16:29.105885 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3424ea4fd86e4453a5d9fde62fc43e7f" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:29.106613 16152 raft_consensus.cc:2468] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:29.108203 16045 leader_election.cc:304] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:29.110639 16048 leader_election.cc:304] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:29.112169 16149 raft_consensus.cc:2468] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:29.112375 16653 raft_consensus.cc:2804] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:29.116087 16653 raft_consensus.cc:697] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:29.117005 16653 consensus_queue.cc:237] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:29.112856 16614 raft_consensus.cc:2804] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:29.118712 16614 raft_consensus.cc:493] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.119097 16614 raft_consensus.cc:3060] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.136394 16201 tablet_bootstrap.cc:492] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:29.137072 16201 ts_tablet_manager.cc:1403] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.174s	user 0.031s	sys 0.045s
I20260709 13:16:29.143668 16204 tablet_bootstrap.cc:492] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:29.144381 16204 ts_tablet_manager.cc:1403] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.122s	user 0.015s	sys 0.004s
I20260709 13:16:29.153916 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3424ea4fd86e4453a5d9fde62fc43e7f" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:29.147569 16201 raft_consensus.cc:359] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:29.163267 16201 raft_consensus.cc:385] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.163698 16201 raft_consensus.cc:740] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.163892 16204 raft_consensus.cc:359] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:29.164675 16204 raft_consensus.cc:385] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.165391 16609 raft_consensus.cc:493] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.165935 16609 raft_consensus.cc:515] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.165114 16201 consensus_queue.cc:260] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:29.168124 16609 leader_election.cc:290] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:29.169353 15933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:29.182013 16006 raft_consensus.cc:2468] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:29.171375 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:29.183295 16119 leader_election.cc:343] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:29.185075 16121 leader_election.cc:343] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:29.185545 16121 leader_election.cc:304] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:29.114912 16203 raft_consensus.cc:359] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:29.186306 16203 raft_consensus.cc:385] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.186690 16609 raft_consensus.cc:2749] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:29.186717 16203 raft_consensus.cc:740] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.164960 16204 raft_consensus.cc:740] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.188387 16203 consensus_queue.cc:260] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:29.189896 16201 ts_tablet_manager.cc:1434] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.002s
I20260709 13:16:29.190934 16614 raft_consensus.cc:515] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:29.191784 16204 consensus_queue.cc:260] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:29.194571 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3424ea4fd86e4453a5d9fde62fc43e7f" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:29.195335 16006 raft_consensus.cc:3060] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.205035 16637 raft_consensus.cc:493] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.205633 16637 raft_consensus.cc:515] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.210546 16637 leader_election.cc:290] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:29.210749 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4332ee7a936341baad672bfc16703cc7" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:29.211127 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4332ee7a936341baad672bfc16703cc7" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:29.211735 16155 raft_consensus.cc:2468] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 0.
W20260709 13:16:29.212167 15971 leader_election.cc:343] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:29.214056 15974 leader_election.cc:304] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:29.215685 16662 raft_consensus.cc:2804] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:29.216032 16662 raft_consensus.cc:493] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.216354 16662 raft_consensus.cc:3060] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.217444 16204 ts_tablet_manager.cc:1434] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.073s	user 0.004s	sys 0.003s
I20260709 13:16:29.218453 16204 tablet_bootstrap.cc:492] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:29.224524 16204 tablet_bootstrap.cc:654] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.232810 16201 tablet_bootstrap.cc:492] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:29.240252 16203 ts_tablet_manager.cc:1434] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.140s	user 0.004s	sys 0.004s
I20260709 13:16:29.241107 16203 tablet_bootstrap.cc:492] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:29.227939 16662 raft_consensus.cc:515] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.244742 16662 leader_election.cc:290] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:29.245083 16006 raft_consensus.cc:2468] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:29.246325 16047 leader_election.cc:304] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:29.246445 16614 leader_election.cc:290] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:29.247046 16651 raft_consensus.cc:2804] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:29.247684 16651 raft_consensus.cc:697] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:29.248546 16651 consensus_queue.cc:237] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:29.246958 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3424ea4fd86e4453a5d9fde62fc43e7f" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:29.250994 15930 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4332ee7a936341baad672bfc16703cc7" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:29.251299 16155 raft_consensus.cc:3060] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:29.252421 15971 leader_election.cc:343] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:29.253676 16201 tablet_bootstrap.cc:654] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.260856 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4332ee7a936341baad672bfc16703cc7" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:29.261466 16152 raft_consensus.cc:3060] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.262077 16203 tablet_bootstrap.cc:654] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.263648 16155 raft_consensus.cc:2468] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:29.292886 16152 raft_consensus.cc:2468] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:29.293258 16203 tablet_bootstrap.cc:492] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:29.267597 16653 consensus_queue.cc:1048] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 13:16:29.294274 16649 consensus_queue.cc:1048] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:29.295176 16204 tablet_bootstrap.cc:492] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:29.295637 16204 ts_tablet_manager.cc:1403] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.077s	user 0.011s	sys 0.000s
I20260709 13:16:29.294605 16203 ts_tablet_manager.cc:1403] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.054s	user 0.010s	sys 0.007s
I20260709 13:16:29.297055 15974 leader_election.cc:304] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:29.298368 16637 raft_consensus.cc:2804] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:29.298851 16637 raft_consensus.cc:697] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:29.298851 16204 raft_consensus.cc:359] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.299548 16204 raft_consensus.cc:385] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.299597 16203 raft_consensus.cc:359] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:29.299885 16204 raft_consensus.cc:740] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.300263 16203 raft_consensus.cc:385] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.299911 16637 consensus_queue.cc:237] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.300540 16203 raft_consensus.cc:740] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.303807 16203 consensus_queue.cc:260] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:29.303027 16204 consensus_queue.cc:260] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.322783 16203 ts_tablet_manager.cc:1434] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.001s
I20260709 13:16:29.324554 16204 ts_tablet_manager.cc:1434] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:16:29.324646 16203 tablet_bootstrap.cc:492] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab: Bootstrap starting.
I20260709 13:16:29.325666 16204 tablet_bootstrap.cc:492] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:29.331743 16237 raft_consensus.cc:493] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.332232 16237 raft_consensus.cc:515] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.333998 16237 leader_election.cc:290] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:29.352429 16204 tablet_bootstrap.cc:654] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.353399 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:29.355118 16201 tablet_bootstrap.cc:492] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:29.355583 16201 ts_tablet_manager.cc:1403] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.123s	user 0.032s	sys 0.006s
W20260709 13:16:29.355774 15898 leader_election.cc:343] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:29.350841 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:29.335800 16614 consensus_queue.cc:1048] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 13:16:29.362162 16155 raft_consensus.cc:2468] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:29.364075 16201 raft_consensus.cc:359] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:29.364917 16201 raft_consensus.cc:385] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.365171 16201 raft_consensus.cc:740] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.365599 15899 leader_election.cc:304] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3
I20260709 13:16:29.365885 16201 consensus_queue.cc:260] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:29.368001 16201 ts_tablet_manager.cc:1434] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:16:29.368860 16201 tablet_bootstrap.cc:492] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3: Bootstrap starting.
I20260709 13:16:29.371603 16076 raft_consensus.cc:3060] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.372448 16670 raft_consensus.cc:2804] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:29.372872 16670 raft_consensus.cc:493] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.373196 16670 raft_consensus.cc:3060] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.376986 16203 tablet_bootstrap.cc:654] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.415803 15830 catalog_manager.cc:5697] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:29.424943 15830 catalog_manager.cc:5697] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:29.426472 15830 catalog_manager.cc:5697] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:29.442034 16469 consensus_queue.cc:1048] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:16:29.462618 16155 raft_consensus.cc:1275] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Refusing update from remote peer 903b3435239241eab01ea54ffb015ee3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:29.461651 15831 catalog_manager.cc:5697] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:29.465287 16637 consensus_queue.cc:1048] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:29.465056 15831 catalog_manager.cc:5697] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:29.469002 15930 raft_consensus.cc:1275] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Refusing update from remote peer 903b3435239241eab01ea54ffb015ee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:16:29.469373 15930 pending_rounds.cc:85] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e: Aborting all ops after (but not including) 0
I20260709 13:16:29.469589 15930 pending_rounds.cc:107] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:16:29.469877 15930 raft_consensus.cc:2889] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:29.473564 16642 consensus_queue.cc:1048] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:29.481375 16201 tablet_bootstrap.cc:654] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.487110 16203 tablet_bootstrap.cc:492] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab: No bootstrap required, opened a new log
I20260709 13:16:29.487769 16203 ts_tablet_manager.cc:1403] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab: Time spent bootstrapping tablet: real 0.164s	user 0.011s	sys 0.006s
I20260709 13:16:29.491308 16203 raft_consensus.cc:359] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:29.492178 16203 raft_consensus.cc:385] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.492511 16203 raft_consensus.cc:740] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.494915 16203 consensus_queue.cc:260] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:29.504426 16203 ts_tablet_manager.cc:1434] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:16:29.513005 16670 raft_consensus.cc:515] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.516112 16670 leader_election.cc:290] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:29.518095 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:29.521732 16155 raft_consensus.cc:3060] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.522410 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
W20260709 13:16:29.525153 15898 leader_election.cc:343] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:29.539767 16204 tablet_bootstrap.cc:492] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:29.540241 16204 ts_tablet_manager.cc:1403] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.215s	user 0.016s	sys 0.001s
I20260709 13:16:29.541807 16651 raft_consensus.cc:493] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 903b3435239241eab01ea54ffb015ee3)
I20260709 13:16:29.542377 16651 raft_consensus.cc:515] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:29.543179 16204 raft_consensus.cc:359] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.543937 16204 raft_consensus.cc:385] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.557188 16204 raft_consensus.cc:740] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.558601 16204 consensus_queue.cc:260] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.564183 16204 ts_tablet_manager.cc:1434] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:16:29.546689 16155 raft_consensus.cc:2468] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:29.568523 16204 tablet_bootstrap.cc:492] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:29.574550 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b83d43d98074cb09deac765b8b035e6" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:29.591261 16651 leader_election.cc:290] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:29.592347 16201 tablet_bootstrap.cc:492] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3: No bootstrap required, opened a new log
I20260709 13:16:29.593048 16201 ts_tablet_manager.cc:1403] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3: Time spent bootstrapping tablet: real 0.224s	user 0.011s	sys 0.000s
I20260709 13:16:29.595456 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b83d43d98074cb09deac765b8b035e6" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:29.596429 16201 raft_consensus.cc:359] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.597378 16048 leader_election.cc:304] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:29.598963 15899 leader_election.cc:304] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3
I20260709 13:16:29.600066 16237 raft_consensus.cc:2804] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:29.600695 16237 raft_consensus.cc:697] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:29.601366 16651 raft_consensus.cc:2749] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:29.601768 16670 raft_consensus.cc:493] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.602236 16670 raft_consensus.cc:515] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:29.597571 16201 raft_consensus.cc:385] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.601425 16237 consensus_queue.cc:237] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.609450 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9719c47150e74128b537e918110116e0" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:29.610101 16003 raft_consensus.cc:2468] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:29.610558 16670 leader_election.cc:290] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:29.611944 16077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9719c47150e74128b537e918110116e0" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:29.612552 16077 raft_consensus.cc:2468] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 0.
I20260709 13:16:29.613539 15898 leader_election.cc:304] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:29.614885 16622 raft_consensus.cc:2804] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:29.615257 16622 raft_consensus.cc:493] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.615566 16622 raft_consensus.cc:3060] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.617311 16204 tablet_bootstrap.cc:654] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:29.604698 16201 raft_consensus.cc:740] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.632028 16201 consensus_queue.cc:260] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.635305 16201 ts_tablet_manager.cc:1434] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.000s
I20260709 13:16:29.666795 16622 raft_consensus.cc:515] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:29.671541 16077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9719c47150e74128b537e918110116e0" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:29.672111 16077 raft_consensus.cc:3060] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.673254 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9719c47150e74128b537e918110116e0" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:29.673862 16003 raft_consensus.cc:3060] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.679517 16622 leader_election.cc:290] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:29.679905 16653 consensus_queue.cc:1048] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:29.693694 16077 raft_consensus.cc:2468] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:29.697597 16469 consensus_queue.cc:1048] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:29.705189 16003 raft_consensus.cc:2468] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 1.
I20260709 13:16:29.705708 16611 consensus_queue.cc:1048] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:29.710332 15898 leader_election.cc:304] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:29.711596 16204 tablet_bootstrap.cc:492] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e: No bootstrap required, opened a new log
I20260709 13:16:29.712055 16204 ts_tablet_manager.cc:1403] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.144s	user 0.007s	sys 0.009s
I20260709 13:16:29.712417 16709 raft_consensus.cc:2804] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:29.713137 16709 raft_consensus.cc:697] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:29.713980 16709 consensus_queue.cc:237] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:29.719163 16204 raft_consensus.cc:359] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:29.719792 16204 raft_consensus.cc:385] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:29.720081 16204 raft_consensus.cc:740] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:29.717480 15831 catalog_manager.cc:5697] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:29.726588 16204 consensus_queue.cc:260] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:29.728621 16204 ts_tablet_manager.cc:1434] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.000s
I20260709 13:16:29.759620 16469 raft_consensus.cc:493] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.760289 16469 raft_consensus.cc:515] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:29.768627 16614 consensus_queue.cc:1048] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:29.779263 16709 raft_consensus.cc:493] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.779753 16709 raft_consensus.cc:515] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.781397 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a7a7d029fb040b881d5165c98065a12" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:29.782205 16152 raft_consensus.cc:2468] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:29.782582 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4332ee7a936341baad672bfc16703cc7" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:29.783460 16151 raft_consensus.cc:2393] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bdeb23ab98c34a269f5b2e71b03c5e4e in current term 1: Already voted for candidate 903b3435239241eab01ea54ffb015ee3 in this term.
I20260709 13:16:29.785928 16709 leader_election.cc:290] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:29.786136 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4332ee7a936341baad672bfc16703cc7" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:29.787135 15932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a7a7d029fb040b881d5165c98065a12" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:29.785650 15831 catalog_manager.cc:5697] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 0 to 1, leader changed from <none> to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:29.788228 16048 leader_election.cc:304] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:29.788516 15898 leader_election.cc:304] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:29.789058 16651 raft_consensus.cc:2804] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:29.787715 15932 raft_consensus.cc:2468] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 0.
I20260709 13:16:29.789541 16651 raft_consensus.cc:493] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.789965 16651 raft_consensus.cc:3060] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.790307 16709 raft_consensus.cc:3060] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.804260 16469 leader_election.cc:290] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:29.831934 16709 raft_consensus.cc:2749] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:29.850728 16647 consensus_queue.cc:1048] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:29.842167 16651 raft_consensus.cc:515] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:29.870659 16651 leader_election.cc:290] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:29.871766 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a7a7d029fb040b881d5165c98065a12" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
I20260709 13:16:29.872424 15931 raft_consensus.cc:3060] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.870169 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a7a7d029fb040b881d5165c98065a12" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:29.882202 16151 raft_consensus.cc:3060] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:29.882810 16662 consensus_queue.cc:1048] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:29.895180 15931 raft_consensus.cc:2468] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:29.896530 16151 raft_consensus.cc:2468] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:29.896785 16045 leader_election.cc:304] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:29.899111 16651 raft_consensus.cc:2804] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:29.899732 16651 raft_consensus.cc:697] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:29.900802 16651 consensus_queue.cc:237] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:29.922052 15830 catalog_manager.cc:5697] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 0 to 1, leader changed from <none> to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 1 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:29.990494 16375 raft_consensus.cc:493] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.991073 16375 raft_consensus.cc:515] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:29.993014 16375 leader_election.cc:290] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:29.997026 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:29.997931 16151 raft_consensus.cc:2393] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 903b3435239241eab01ea54ffb015ee3 in current term 1: Already voted for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in this term.
I20260709 13:16:29.998183 15931 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:30.010566 15974 leader_election.cc:304] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:30.011341 16662 raft_consensus.cc:3060] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:30.019207 16662 raft_consensus.cc:2749] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:30.171769 16709 consensus_queue.cc:1048] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:30.192494 16710 consensus_queue.cc:1048] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:30.209563 16710 consensus_queue.cc:1048] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:30.235450 16520 consensus_queue.cc:1048] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:30.319470 16611 consensus_queue.cc:1048] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:30.331610 16611 consensus_queue.cc:1048] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:30.946586 15972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:32795, user_credentials={real_user=slave}} blocked reactor thread for 65171.3us
I20260709 13:16:31.888609  6937 tablet_server.cc:179] TabletServer@127.6.198.65:0 shutting down...
W20260709 13:16:31.908516 16045 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:31.932909 16119 consensus_peers.cc:597] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:31.935286 16045 consensus_peers.cc:597] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:31.946231 16045 consensus_peers.cc:597] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:31.956990 16045 consensus_peers.cc:597] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:31.958033 15971 consensus_peers.cc:597] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:31.975526 16045 consensus_peers.cc:597] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:31.991722  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:16:31.994391 16045 consensus_peers.cc:597] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:31.996395  6937 tablet_replica.cc:333] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:31.997349  6937 raft_consensus.cc:2243] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:31.998612  6937 raft_consensus.cc:2272] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.002362  6937 tablet_replica.cc:333] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.003185  6937 raft_consensus.cc:2243] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.004134  6937 raft_consensus.cc:2272] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.006785  6937 tablet_replica.cc:333] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.007418  6937 raft_consensus.cc:2243] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.008304  6937 raft_consensus.cc:2272] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.010818  6937 tablet_replica.cc:333] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.011587  6937 raft_consensus.cc:2243] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.012207  6937 raft_consensus.cc:2272] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:32.016724 15971 consensus_peers.cc:597] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.017992  6937 tablet_replica.cc:333] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.019042  6937 raft_consensus.cc:2243] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.019655  6937 raft_consensus.cc:2272] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.024250  6937 tablet_replica.cc:333] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
W20260709 13:16:32.024537 15971 consensus_peers.cc:597] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.025352  6937 raft_consensus.cc:2243] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.026700  6937 raft_consensus.cc:2272] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:32.029196 16045 consensus_peers.cc:597] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.030974  6937 tablet_replica.cc:333] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.032006  6937 raft_consensus.cc:2243] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.032981  6937 raft_consensus.cc:2272] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:32.035240 15971 consensus_peers.cc:597] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.035982  6937 tablet_replica.cc:333] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.036801  6937 raft_consensus.cc:2243] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.037297  6937 raft_consensus.cc:2272] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.040010  6937 tablet_replica.cc:333] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.040828  6937 raft_consensus.cc:2243] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.041440  6937 raft_consensus.cc:2272] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.044922  6937 tablet_replica.cc:333] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.045861  6937 raft_consensus.cc:2243] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.047039  6937 raft_consensus.cc:2272] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.050583  6937 tablet_replica.cc:333] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.051424  6937 raft_consensus.cc:2243] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.052078  6937 raft_consensus.cc:2272] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.055244  6937 tablet_replica.cc:333] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.056010  6937 raft_consensus.cc:2243] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.056582  6937 raft_consensus.cc:2272] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.059711  6937 tablet_replica.cc:333] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.060519  6937 raft_consensus.cc:2243] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.061158  6937 raft_consensus.cc:2272] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.064185  6937 tablet_replica.cc:333] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.065155  6937 raft_consensus.cc:2243] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.066404  6937 raft_consensus.cc:2272] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.069248  6937 tablet_replica.cc:333] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.070225  6937 raft_consensus.cc:2243] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.070858  6937 raft_consensus.cc:2272] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.073851  6937 tablet_replica.cc:333] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.074643  6937 raft_consensus.cc:2243] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.075206  6937 raft_consensus.cc:2272] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.078436  6937 tablet_replica.cc:333] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.079355  6937 raft_consensus.cc:2243] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.080626  6937 raft_consensus.cc:2272] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.083039  6937 tablet_replica.cc:333] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.083667  6937 raft_consensus.cc:2243] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.084169  6937 raft_consensus.cc:2272] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.086913  6937 tablet_replica.cc:333] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.087502  6937 raft_consensus.cc:2243] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.088091  6937 raft_consensus.cc:2272] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.090778  6937 tablet_replica.cc:333] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.091585  6937 raft_consensus.cc:2243] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.092336  6937 raft_consensus.cc:2272] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.095474  6937 tablet_replica.cc:333] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.096050  6937 raft_consensus.cc:2243] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.097016  6937 raft_consensus.cc:2272] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.100052  6937 tablet_replica.cc:333] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.100755  6937 raft_consensus.cc:2243] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.101925  6937 raft_consensus.cc:2272] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:32.102746 16045 consensus_peers.cc:597] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.105481  6937 tablet_replica.cc:333] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.106415  6937 raft_consensus.cc:2243] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.106946  6937 raft_consensus.cc:2272] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.109313  6937 tablet_replica.cc:333] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
W20260709 13:16:32.109406 15971 consensus_peers.cc:597] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.109974  6937 raft_consensus.cc:2243] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.110627  6937 raft_consensus.cc:2272] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.113209  6937 tablet_replica.cc:333] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.113966  6937 raft_consensus.cc:2243] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.114749  6937 raft_consensus.cc:2272] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.118005  6937 tablet_replica.cc:333] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.118745  6937 raft_consensus.cc:2243] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.120198  6937 raft_consensus.cc:2272] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.124600  6937 tablet_replica.cc:333] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.125859  6937 raft_consensus.cc:2243] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.126621  6937 raft_consensus.cc:2272] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:32.127213 15971 consensus_peers.cc:597] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.129619  6937 tablet_replica.cc:333] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.130275  6937 raft_consensus.cc:2243] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.131393  6937 raft_consensus.cc:2272] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:32.131690 16045 consensus_peers.cc:597] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.133782  6937 tablet_replica.cc:333] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.134677  6937 raft_consensus.cc:2243] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.135318  6937 raft_consensus.cc:2272] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.138603  6937 tablet_replica.cc:333] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.139410  6937 raft_consensus.cc:2243] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.140022  6937 raft_consensus.cc:2272] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:32.141132 16045 consensus_peers.cc:597] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.143857  6937 tablet_replica.cc:333] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.144893  6937 raft_consensus.cc:2243] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.145510  6937 raft_consensus.cc:2272] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.148699  6937 tablet_replica.cc:333] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.149667  6937 raft_consensus.cc:2243] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.150279  6937 raft_consensus.cc:2272] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:32.152369 16119 consensus_peers.cc:597] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.153322  6937 tablet_replica.cc:333] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.154198  6937 raft_consensus.cc:2243] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.154794  6937 raft_consensus.cc:2272] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.157244  6937 tablet_replica.cc:333] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.157786  6937 raft_consensus.cc:2243] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.158214  6937 raft_consensus.cc:2272] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.160199  6937 tablet_replica.cc:333] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.161123  6937 raft_consensus.cc:2243] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.162197  6937 raft_consensus.cc:2272] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.165333  6937 tablet_replica.cc:333] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
W20260709 13:16:32.165618 16045 consensus_peers.cc:597] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.166288  6937 raft_consensus.cc:2243] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.167017  6937 raft_consensus.cc:2272] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.170092  6937 tablet_replica.cc:333] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.170935  6937 raft_consensus.cc:2243] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.172202  6937 raft_consensus.cc:2272] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.175225  6937 tablet_replica.cc:333] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.175971  6937 raft_consensus.cc:2243] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.176604  6937 raft_consensus.cc:2272] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:32.177786 15971 consensus_peers.cc:597] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:32.180241 15971 consensus_peers.cc:597] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.184167  6937 tablet_replica.cc:333] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.184988  6937 raft_consensus.cc:2243] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.185515  6937 raft_consensus.cc:2272] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.188691  6937 tablet_replica.cc:333] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.189427  6937 raft_consensus.cc:2243] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.190507  6937 raft_consensus.cc:2272] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.194649  6937 tablet_replica.cc:333] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.195361  6937 raft_consensus.cc:2243] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.195920  6937 raft_consensus.cc:2272] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.199887  6937 tablet_replica.cc:333] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.200649  6937 raft_consensus.cc:2243] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.201233  6937 raft_consensus.cc:2272] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.204593  6937 tablet_replica.cc:333] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.205312  6937 raft_consensus.cc:2243] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.206429  6937 raft_consensus.cc:2272] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:32.207518 15971 consensus_peers.cc:597] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.209587  6937 tablet_replica.cc:333] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.212841  6937 raft_consensus.cc:2243] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:32.213444  6937 raft_consensus.cc:2272] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:32.216820  6937 tablet_replica.cc:333] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e: stopping tablet replica
I20260709 13:16:32.217595  6937 raft_consensus.cc:2243] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:32.223037  6937 raft_consensus.cc:2272] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:32.257416 15971 consensus_peers.cc:597] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:32.275185 16045 consensus_peers.cc:597] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:32.308938 16119 consensus_peers.cc:597] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:32.324273 16119 connection.cc:573] client connection to 127.6.198.65:45841 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:16:32.325083 16119 consensus_peers.cc:597] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:32.353149 16119 consensus_peers.cc:597] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.355723  6937 tablet_server.cc:196] TabletServer@127.6.198.65:0 shutdown complete.
W20260709 13:16:32.392874 15971 consensus_peers.cc:597] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:32.412758 16045 consensus_peers.cc:597] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:32.589349  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:32.598731 16744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:32.610885 16745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:32.726640  6937 server_base.cc:1061] running on GCE node
W20260709 13:16:32.732213 16748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:32.744884  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:32.745322  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:32.745613  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783602992745591 us; error 0 us; skew 500 ppm
I20260709 13:16:32.746730  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:32.771477  6937 webserver.cc:533] Webserver started at http://127.6.198.65:43789/ using document root <none> and password file <none>
I20260709 13:16:32.772185  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:32.772418  6937 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:16:32.778991  6937 fs_manager.cc:714] Time spent opening directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 13:16:32.792284 16753 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:32.793215  6937 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.001s
I20260709 13:16:32.793678  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-0-root
uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
format_stamp: "Formatted at 2026-07-09 13:16:18 on dist-test-slave-tqzw"
I20260709 13:16:32.795550  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602718985547-6937-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:32.836405  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:32.839918  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:32.937669 16728 raft_consensus.cc:493] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:32.939265 16728 raft_consensus.cc:515] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:32.950012 16728 leader_election.cc:290] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:32.952064 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e6891e9cab4a7298664c3c431ce36d" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:32.965400 16119 leader_election.cc:336] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:32.966666 16119 leader_election.cc:304] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:32.967696 16728 raft_consensus.cc:2749] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:32.969249  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:32.977576 16761 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 13:16:33.037314 16728 raft_consensus.cc:493] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.038165 16728 raft_consensus.cc:515] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.040491 16728 leader_election.cc:290] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.042059 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f15b348b66b4bffae9c7f9eed8207e2" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:33.045271 16119 leader_election.cc:336] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.045732 16119 leader_election.cc:304] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.046291 16728 raft_consensus.cc:2749] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.058847 16728 raft_consensus.cc:493] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.059501 16728 raft_consensus.cc:515] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.062386 16728 leader_election.cc:290] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.062434 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a39139d4f2b48d09ba9eb320dfcf265" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
W20260709 13:16:33.067201 16119 leader_election.cc:336] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.067612 16119 leader_election.cc:304] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:33.068218 16728 raft_consensus.cc:2749] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.098089 16662 raft_consensus.cc:493] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.098573 16662 raft_consensus.cc:515] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.101023 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e6891e9cab4a7298664c3c431ce36d" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:33.101619 16153 raft_consensus.cc:2468] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:33.103153 15974 leader_election.cc:304] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:33.104245 16740 raft_consensus.cc:2804] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.104632 16740 raft_consensus.cc:493] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.105001 16740 raft_consensus.cc:3060] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:33.110702 15971 leader_election.cc:336] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.110918 16759 raft_consensus.cc:493] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.111591 16759 raft_consensus.cc:515] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.113395 16731 raft_consensus.cc:493] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.113920 16731 raft_consensus.cc:515] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.114404 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9719c47150e74128b537e918110116e0" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:33.116106 16731 leader_election.cc:290] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.118980 16759 leader_election.cc:290] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.120434 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633df69636042288f27156e6084dce9" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:33.130446 16662 leader_election.cc:290] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.131440 16740 raft_consensus.cc:515] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
W20260709 13:16:33.136721 15971 leader_election.cc:336] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.137213 15971 leader_election.cc:304] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:33.138664 16662 raft_consensus.cc:2749] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.142554 16740 leader_election.cc:290] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.152549 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e6891e9cab4a7298664c3c431ce36d" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:33.153091 16152 raft_consensus.cc:3060] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.159777 16152 raft_consensus.cc:2468] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 2.
I20260709 13:16:33.170560 15974 leader_election.cc:304] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:33.171490 16662 raft_consensus.cc:2804] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.172084 16662 raft_consensus.cc:697] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
W20260709 13:16:33.174096 15971 leader_election.cc:336] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:33.164628 16045 leader_election.cc:336] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.177662 16045 leader_election.cc:304] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.178586 16759 raft_consensus.cc:2749] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.175196 16662 consensus_queue.cc:237] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.189538 16759 raft_consensus.cc:493] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.190037 16759 raft_consensus.cc:515] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.192538 16759 leader_election.cc:290] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.195277 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd99621a84f4fe994996601dae8988c" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:33.195757 15830 catalog_manager.cc:5697] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.204420 16728 raft_consensus.cc:493] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.205148 16728 raft_consensus.cc:515] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.207932 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b94b09aec6746f69853f14b417f18e3" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
W20260709 13:16:33.211200 16045 leader_election.cc:336] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.211779 16045 leader_election.cc:304] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.213891 16469 raft_consensus.cc:2749] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.208743 16728 leader_election.cc:290] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:33.221899 16119 leader_election.cc:336] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.222333 16119 leader_election.cc:304] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:33.223531 16728 raft_consensus.cc:2749] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.227967 16728 raft_consensus.cc:493] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.228492 16728 raft_consensus.cc:515] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.232365 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a3eeeeab11406e90e1fd10d63228e8" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:33.234549 16728 leader_election.cc:290] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:33.240775 16119 leader_election.cc:336] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.241461 16469 raft_consensus.cc:493] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.243791 16469 raft_consensus.cc:515] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.245625 16469 leader_election.cc:290] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.248173 16662 raft_consensus.cc:493] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.248313 16119 leader_election.cc:304] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:33.248864 16662 raft_consensus.cc:515] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.249301 16728 raft_consensus.cc:2749] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.252163 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca16f452c2a4b19a53a12119268b92d" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
W20260709 13:16:33.255785 15971 leader_election.cc:336] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.256218 15971 leader_election.cc:304] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:33.256883 16731 raft_consensus.cc:2749] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.257570 16759 raft_consensus.cc:493] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.258153 16759 raft_consensus.cc:515] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.267002 16759 leader_election.cc:290] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.256981 16662 leader_election.cc:290] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:33.270956 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a39139d4f2b48d09ba9eb320dfcf265" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:33.271824 16152 raft_consensus.cc:2468] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:33.273646 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b94b09aec6746f69853f14b417f18e3" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:33.273834 16048 leader_election.cc:304] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:33.274235 16152 raft_consensus.cc:2468] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:33.274713 16759 raft_consensus.cc:2804] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.275125 16759 raft_consensus.cc:493] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.275432 16759 raft_consensus.cc:3060] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.279940 16662 raft_consensus.cc:493] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.282644 16048 leader_election.cc:304] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:33.283496 16469 raft_consensus.cc:2804] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.283891 16469 raft_consensus.cc:493] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.284303 16469 raft_consensus.cc:3060] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.286778 16651 raft_consensus.cc:493] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.287313 16651 raft_consensus.cc:515] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.289039 16651 leader_election.cc:290] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.290268 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca16f452c2a4b19a53a12119268b92d" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:33.280717 16662 raft_consensus.cc:515] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.291183 16006 raft_consensus.cc:2468] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
W20260709 13:16:33.292291 16045 leader_election.cc:336] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.294533 16047 leader_election.cc:304] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:33.294673 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f15b348b66b4bffae9c7f9eed8207e2" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:33.295408 16651 raft_consensus.cc:2804] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.295503 16153 raft_consensus.cc:2468] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:33.295912 16651 raft_consensus.cc:493] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.296350 16651 raft_consensus.cc:3060] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.301157 16661 raft_consensus.cc:493] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.301702 16661 raft_consensus.cc:515] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.302977 16728 raft_consensus.cc:493] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.303740 16728 raft_consensus.cc:515] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.305598 16661 leader_election.cc:290] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.305635 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633df69636042288f27156e6084dce9" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:33.312222 16006 raft_consensus.cc:2468] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 1.
I20260709 13:16:33.312840 16759 raft_consensus.cc:515] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.313565 16651 raft_consensus.cc:515] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.314018 16121 leader_election.cc:304] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:33.315158 16469 raft_consensus.cc:515] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.315335 16661 raft_consensus.cc:2804] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.315735 16759 leader_election.cc:290] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.316004 16661 raft_consensus.cc:493] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.316458 16661 raft_consensus.cc:3060] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.324611 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a39139d4f2b48d09ba9eb320dfcf265" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:33.325361 16153 raft_consensus.cc:3060] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.325491 16661 raft_consensus.cc:515] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.328970 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4633df69636042288f27156e6084dce9" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:33.329138 16469 leader_election.cc:290] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:33.329197 16045 leader_election.cc:336] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.329679 16006 raft_consensus.cc:3060] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.331175 16651 leader_election.cc:290] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.328959 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b94b09aec6746f69853f14b417f18e3" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:33.333348 16155 raft_consensus.cc:3060] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:33.335410 16045 leader_election.cc:336] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:33.336192 16045 leader_election.cc:336] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.338150 15974 leader_election.cc:304] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:33.340358 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ca16f452c2a4b19a53a12119268b92d" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:33.340520 16739 raft_consensus.cc:2804] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.340994 16005 raft_consensus.cc:3060] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.341027 16739 raft_consensus.cc:493] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.341727 16739 raft_consensus.cc:3060] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.347061 16740 raft_consensus.cc:493] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.347604 16740 raft_consensus.cc:515] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.349493 16740 leader_election.cc:290] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.350246 16005 raft_consensus.cc:2468] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 2.
I20260709 13:16:33.350798 16661 leader_election.cc:290] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.351918 16047 leader_election.cc:304] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
W20260709 13:16:33.353026 16045 leader_election.cc:336] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.353883 16469 raft_consensus.cc:2804] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:16:33.354099 16045 leader_election.cc:336] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.354465 16469 raft_consensus.cc:697] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 2 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:33.355257 16469 consensus_queue.cc:237] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.357388 16759 raft_consensus.cc:493] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.358108 16759 raft_consensus.cc:515] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.359351 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:33.360735 16759 leader_election.cc:290] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.361850 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce23588dc52d46338893551f44214a28" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
W20260709 13:16:33.362536 15971 leader_election.cc:336] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:33.366290 16045 leader_election.cc:336] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.366999 16045 leader_election.cc:304] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.368240 16651 raft_consensus.cc:2749] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.369418 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a21a302bc17473392974aeea5a31d60" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:33.372330 15974 leader_election.cc:304] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:33.373795 16740 raft_consensus.cc:2749] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.370925 16155 raft_consensus.cc:2468] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 2.
W20260709 13:16:33.376919 16119 leader_election.cc:336] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.385695 16153 raft_consensus.cc:2468] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 2.
I20260709 13:16:33.386901 16662 leader_election.cc:290] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.359293 16728 leader_election.cc:290] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.386873 16048 leader_election.cc:304] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.389920 16651 raft_consensus.cc:2804] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.390501 16651 raft_consensus.cc:697] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 2 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:33.395121 16651 consensus_queue.cc:237] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.391569 16661 raft_consensus.cc:493] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.400684 16661 raft_consensus.cc:515] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.403723 16661 leader_election.cc:290] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.390100 16739 raft_consensus.cc:515] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.404399 16731 raft_consensus.cc:493] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.404973 16731 raft_consensus.cc:515] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.404706 15830 catalog_manager.cc:5697] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.405898 16739 leader_election.cc:290] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:33.399106 16119 leader_election.cc:336] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.381412 16006 raft_consensus.cc:2468] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 2.
I20260709 13:16:33.407258 16731 leader_election.cc:290] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
W20260709 13:16:33.408547 16119 leader_election.cc:336] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.403645 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc17a40e973a46a18d21e55a0a939417" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:33.408964 16119 leader_election.cc:304] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.410260 16048 leader_election.cc:304] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.410908 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f15b348b66b4bffae9c7f9eed8207e2" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:33.411029 16469 raft_consensus.cc:2804] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.411556 16155 raft_consensus.cc:3060] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:33.409014 15971 leader_election.cc:336] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.414404 16077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd99621a84f4fe994996601dae8988c" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:33.415174 16077 raft_consensus.cc:2468] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:33.416559 16469 raft_consensus.cc:697] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 2 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:33.426919 16728 raft_consensus.cc:493] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.428853 16121 leader_election.cc:304] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.428771 16728 raft_consensus.cc:515] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.430963 16728 leader_election.cc:290] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.431165 16155 raft_consensus.cc:2468] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 2.
I20260709 13:16:33.431478 16728 raft_consensus.cc:2804] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.431988 16728 raft_consensus.cc:697] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 2 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
I20260709 13:16:33.417035 15972 leader_election.cc:304] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:33.445112 15974 leader_election.cc:304] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:33.419131 16661 raft_consensus.cc:2749] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.448378 16469 consensus_queue.cc:237] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.450779 16662 raft_consensus.cc:2804] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.427870 16731 raft_consensus.cc:493] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.453294 16731 raft_consensus.cc:515] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.446821 16739 raft_consensus.cc:493] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.457212 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a21a302bc17473392974aeea5a31d60" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:33.458194 16006 raft_consensus.cc:2468] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 1.
I20260709 13:16:33.456177 16731 leader_election.cc:290] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.451356 16662 raft_consensus.cc:697] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:33.468916 16731 raft_consensus.cc:493] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.469508 16731 raft_consensus.cc:515] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.471686 16731 leader_election.cc:290] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.473016 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28cbf604c51f48da99cb7e3089651236" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:33.457358 16739 raft_consensus.cc:515] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.476306 16739 leader_election.cc:290] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:33.477383 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce23588dc52d46338893551f44214a28" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:33.478520 16076 raft_consensus.cc:2468] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:33.479808 15972 leader_election.cc:304] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:33.480333 16073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5922376de62747aabde22ad134c86f36" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:33.481071 16749 raft_consensus.cc:493] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.481703 16749 raft_consensus.cc:515] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.481775 16728 consensus_queue.cc:237] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.480942 16121 leader_election.cc:304] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:33.492316 16774 raft_consensus.cc:2804] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.492961 16774 raft_consensus.cc:493] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.493424 16774 raft_consensus.cc:3060] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.458050 16740 raft_consensus.cc:2804] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.484175 16749 leader_election.cc:290] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.511083 16739 raft_consensus.cc:2804] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.511554 16739 raft_consensus.cc:493] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.490830 16662 consensus_queue.cc:237] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.511915 16739 raft_consensus.cc:3060] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.542402 16737 raft_consensus.cc:493] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.542793 16651 raft_consensus.cc:493] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
W20260709 13:16:33.545086 15972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:32795, user_credentials={real_user=slave}} blocked reactor thread for 55976.1us
I20260709 13:16:33.543488 16651 raft_consensus.cc:515] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.512436 16740 raft_consensus.cc:493] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.549034 16768 raft_consensus.cc:493] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.549615 16651 leader_election.cc:290] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.549692 16768 raft_consensus.cc:515] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.551255 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc17a40e973a46a18d21e55a0a939417" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:33.551908 16155 raft_consensus.cc:2468] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:33.552019 16768 leader_election.cc:290] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:33.553107 16048 leader_election.cc:304] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:33.555239 16651 raft_consensus.cc:2804] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.555572 16651 raft_consensus.cc:493] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.555874 16651 raft_consensus.cc:3060] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:33.556573 16119 leader_election.cc:336] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.560266 16119 leader_election.cc:304] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
W20260709 13:16:33.561335 16119 leader_election.cc:336] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.558540 16740 raft_consensus.cc:3060] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.572009 15830 catalog_manager.cc:5697] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.543066 16737 raft_consensus.cc:515] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.573261 16728 raft_consensus.cc:2749] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:33.573623 16121 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:32879, user_credentials={real_user=slave}} blocked reactor thread for 88624.8us
I20260709 13:16:33.588146 16469 raft_consensus.cc:493] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.595582 16469 raft_consensus.cc:515] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.597676 16469 leader_election.cc:290] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.598099 15831 catalog_manager.cc:5697] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.600878 15831 catalog_manager.cc:5697] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.602119 16775 raft_consensus.cc:493] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
W20260709 13:16:33.602969 16119 leader_election.cc:336] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.604681 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0c303e2dd466caf0e684c6ea83284" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:33.608028 16651 raft_consensus.cc:515] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.609442 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a3eeeeab11406e90e1fd10d63228e8" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:33.610159 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28cbf604c51f48da99cb7e3089651236" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:33.610198 16155 raft_consensus.cc:2468] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:33.610872 16005 raft_consensus.cc:2468] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 1.
I20260709 13:16:33.612336 16778 raft_consensus.cc:493] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.612376 16048 leader_election.cc:304] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:33.612931 16778 raft_consensus.cc:515] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.616004 16737 leader_election.cc:290] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.616156 16739 raft_consensus.cc:515] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.616955 16773 raft_consensus.cc:493] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
W20260709 13:16:33.617496 16045 leader_election.cc:336] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.617429 16773 raft_consensus.cc:515] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.618518 16770 raft_consensus.cc:2804] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.618880 16121 leader_election.cc:304] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.618081 16739 leader_election.cc:290] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.592178 16782 raft_consensus.cc:493] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.621304 16651 leader_election.cc:290] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.621508 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc17a40e973a46a18d21e55a0a939417" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:33.603250 16775 raft_consensus.cc:515] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.626739 16153 raft_consensus.cc:2468] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
I20260709 13:16:33.628327 16775 leader_election.cc:290] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.618916 16770 raft_consensus.cc:493] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.629750 16770 raft_consensus.cc:3060] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.630142 16773 leader_election.cc:290] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.623921 16077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9719c47150e74128b537e918110116e0" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:33.634582 16778 leader_election.cc:290] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:33.634853 16077 raft_consensus.cc:2468] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:33.625486 16782 raft_consensus.cc:515] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.631578 16774 raft_consensus.cc:515] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.632241 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0c303e2dd466caf0e684c6ea83284" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:33.636570 16749 raft_consensus.cc:2804] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.636914 16076 raft_consensus.cc:2468] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 1.
I20260709 13:16:33.637231 16782 leader_election.cc:290] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.623379 16155 raft_consensus.cc:3060] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.637024 16749 raft_consensus.cc:493] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.638550 16749 raft_consensus.cc:3060] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.633266 16048 leader_election.cc:304] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
W20260709 13:16:33.641294 15971 leader_election.cc:336] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.642755 16120 leader_election.cc:304] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
W20260709 13:16:33.644353 15971 leader_election.cc:336] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.645047 16737 raft_consensus.cc:2804] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.645488 16737 raft_consensus.cc:493] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.645857 16737 raft_consensus.cc:3060] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:33.650045 16119 leader_election.cc:336] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.650867 16661 raft_consensus.cc:2804] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:16:33.651746 15971 leader_election.cc:336] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.648142 16740 raft_consensus.cc:515] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
W20260709 13:16:33.651845 16045 leader_election.cc:336] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:33.653468 16045 leader_election.cc:336] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:33.652454 15971 leader_election.cc:336] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.651284 16661 raft_consensus.cc:493] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.654531 15971 leader_election.cc:304] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:33.655108 16155 raft_consensus.cc:2468] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 2.
I20260709 13:16:33.659469 16048 leader_election.cc:304] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:33.668246 16469 raft_consensus.cc:2804] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:16:33.673158 15971 leader_election.cc:336] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.673834 15971 leader_election.cc:304] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
W20260709 13:16:33.674899 15971 leader_election.cc:336] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.675472 16778 raft_consensus.cc:2749] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.656147 16661 raft_consensus.cc:3060] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.677426 16774 leader_election.cc:290] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.679669 16768 raft_consensus.cc:2749] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.683616 16770 raft_consensus.cc:515] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.685819 16770 leader_election.cc:290] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:33.687901 15971 leader_election.cc:336] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:33.691182 16045 leader_election.cc:336] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:33.692147 16119 leader_election.cc:336] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:33.696511 15971 leader_election.cc:336] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:33.697439 15971 leader_election.cc:336] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.697649 16737 raft_consensus.cc:515] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.693734 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a21a302bc17473392974aeea5a31d60" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:33.703819 16005 raft_consensus.cc:3060] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:33.705307 15971 leader_election.cc:336] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.687047 16749 raft_consensus.cc:515] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.711928 16749 leader_election.cc:290] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.713816 16737 leader_election.cc:290] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.714183 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28cbf604c51f48da99cb7e3089651236" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:33.714807 16006 raft_consensus.cc:3060] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.715989 16005 raft_consensus.cc:2468] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 2.
W20260709 13:16:33.717114 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:36069, user_credentials={real_user=slave}} blocked reactor thread for 110684us
I20260709 13:16:33.717430 16469 raft_consensus.cc:697] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 2 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:33.718307 16469 consensus_queue.cc:237] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.718662 16121 leader_election.cc:304] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.719163 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce23588dc52d46338893551f44214a28" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:33.719707 16076 raft_consensus.cc:3060] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:33.719983 16119 leader_election.cc:336] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:33.721459 16048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:36069, user_credentials={real_user=slave}} blocked reactor thread for 52232.8us
I20260709 13:16:33.721582 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:33.722249 16153 raft_consensus.cc:2468] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:33.692757 16740 leader_election.cc:290] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
W20260709 13:16:33.696638 16045 leader_election.cc:336] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.725227 15831 catalog_manager.cc:5697] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:33.727097 16077 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6aa884c0889444ea8aa23dbe7bcfe9de" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab" is_pre_election: true
I20260709 13:16:33.729403 16759 raft_consensus.cc:493] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.731725 16149 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6a3eeeeab11406e90e1fd10d63228e8" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:33.732949 16149 raft_consensus.cc:3060] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.732818 16759 raft_consensus.cc:515] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.734056 16077 raft_consensus.cc:2468] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 1.
I20260709 13:16:33.734223 16787 raft_consensus.cc:2804] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.734859 16787 raft_consensus.cc:697] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 2 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
I20260709 13:16:33.737517 16759 leader_election.cc:290] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.740433 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0c303e2dd466caf0e684c6ea83284" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:33.741468 16149 raft_consensus.cc:2468] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 2.
I20260709 13:16:33.699987 16661 raft_consensus.cc:515] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.743480 16155 raft_consensus.cc:2393] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d4d6e13a8f4a473098351ada96fd7dab in current term 2: Already voted for candidate c95f0816c41142699d80a2e326e17ab8 in this term.
I20260709 13:16:33.743907 15974 leader_election.cc:304] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.748157 16048 leader_election.cc:304] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:33.752142 16651 raft_consensus.cc:2804] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.752830 16651 raft_consensus.cc:697] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 2 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:33.752890 16081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd99621a84f4fe994996601dae8988c" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:33.753772 16081 raft_consensus.cc:3060] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.753952 16651 consensus_queue.cc:237] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
W20260709 13:16:33.752139 16119 leader_election.cc:336] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.735687 16787 consensus_queue.cc:237] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.768888 16740 raft_consensus.cc:2804] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.746194 16006 raft_consensus.cc:2468] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 2.
I20260709 13:16:33.769631 16740 raft_consensus.cc:493] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.770274 16740 raft_consensus.cc:3060] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.773139  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:16:33.773512  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.798s	user 0.005s	sys 0.004s
I20260709 13:16:33.773855  6937 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:16:33.776727 15972 leader_election.cc:304] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.776929 16081 raft_consensus.cc:2468] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 2.
I20260709 13:16:33.777976 16661 leader_election.cc:290] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.749998 16076 raft_consensus.cc:2468] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 2.
I20260709 13:16:33.779736 16768 raft_consensus.cc:2804] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.780200 16768 raft_consensus.cc:493] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.780594 16768 raft_consensus.cc:3060] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.789752 16768 raft_consensus.cc:515] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
W20260709 13:16:33.790776 15971 consensus_peers.cc:597] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:33.791963 16768 leader_election.cc:290] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:33.799130 15972 leader_election.cc:304] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.800664 15972 leader_election.cc:304] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.801767 16772 raft_consensus.cc:2804] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.802206 16772 raft_consensus.cc:493] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.802568 16772 raft_consensus.cc:3060] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.802726 16784 raft_consensus.cc:2804] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:16:33.800606 16120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:32795, user_credentials={real_user=slave}} blocked reactor thread for 148530us
I20260709 13:16:33.803273 16784 raft_consensus.cc:697] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:33.804190 16784 consensus_queue.cc:237] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.801822 15972 leader_election.cc:304] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.808264 16777 raft_consensus.cc:2804] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.808764 16777 raft_consensus.cc:697] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:33.809374 16121 leader_election.cc:304] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.809577 16777 consensus_queue.cc:237] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:33.810256 16728 raft_consensus.cc:2804] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.810947 16728 raft_consensus.cc:697] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 2 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
I20260709 13:16:33.811556 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9719c47150e74128b537e918110116e0" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:33.812166 16076 raft_consensus.cc:3060] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.811797 16728 consensus_queue.cc:237] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.817567 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5922376de62747aabde22ad134c86f36" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:33.818285 16003 raft_consensus.cc:2468] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 1.
W20260709 13:16:33.823277 15971 leader_election.cc:336] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.823675 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6aa884c0889444ea8aa23dbe7bcfe9de" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:33.828176 16081 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0c303e2dd466caf0e684c6ea83284" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:33.828966 16081 raft_consensus.cc:2393] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c95f0816c41142699d80a2e326e17ab8 in current term 2: Already voted for candidate d4d6e13a8f4a473098351ada96fd7dab in this term.
I20260709 13:16:33.831372 16047 leader_election.cc:304] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.832865 16737 raft_consensus.cc:2804] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:33.833256 16737 raft_consensus.cc:493] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Starting leader election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:33.833616 16737 raft_consensus.cc:3060] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.834501 16120 leader_election.cc:304] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:33.835863 16749 raft_consensus.cc:2749] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:16:33.837613 16772 raft_consensus.cc:515] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.862277 16005 raft_consensus.cc:2393] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d4d6e13a8f4a473098351ada96fd7dab in current term 2: Already voted for candidate 903b3435239241eab01ea54ffb015ee3 in this term.
W20260709 13:16:33.836527 16045 leader_election.cc:336] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.841566 16761 tablet_bootstrap.cc:492] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:33.848078 16155 raft_consensus.cc:1275] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Refusing update from remote peer 903b3435239241eab01ea54ffb015ee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:33.842554 16076 raft_consensus.cc:2468] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 2.
I20260709 13:16:33.850796 16737 raft_consensus.cc:515] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.867810 16737 leader_election.cc:290] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:33.868202 16740 raft_consensus.cc:515] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.869072 15830 catalog_manager.cc:5697] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.870527 15972 leader_election.cc:304] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.870890 15830 catalog_manager.cc:5697] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.871680 16784 raft_consensus.cc:2804] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.872285 16784 raft_consensus.cc:697] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:33.873106 16784 consensus_queue.cc:237] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.873628 16740 leader_election.cc:290] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:33.876231 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:33.876808 16152 raft_consensus.cc:3060] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.877100 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5922376de62747aabde22ad134c86f36" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:33.877645 16005 raft_consensus.cc:3060] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:33.883795 15971 leader_election.cc:336] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.884796 16005 raft_consensus.cc:2468] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 2.
W20260709 13:16:33.864187 16045 leader_election.cc:336] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.893107 16045 leader_election.cc:304] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:33.889094 16777 consensus_queue.cc:1048] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:33.895633 16047 leader_election.cc:304] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:33.896008 16469 raft_consensus.cc:2749] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:16:33.896726 15971 leader_election.cc:336] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.897338 16003 raft_consensus.cc:1275] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Refusing update from remote peer c95f0816c41142699d80a2e326e17ab8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:33.897447 16469 raft_consensus.cc:2804] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.897981 16469 raft_consensus.cc:697] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [term 2 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:33.885104 16152 raft_consensus.cc:2468] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 2.
I20260709 13:16:33.899598 15974 leader_election.cc:304] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.899454 16469 consensus_queue.cc:237] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
W20260709 13:16:33.900744 16119 consensus_peers.cc:597] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:33.903124 16740 raft_consensus.cc:2804] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.903748 16740 raft_consensus.cc:697] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:33.852519 15831 catalog_manager.cc:5697] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.892560 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6aa884c0889444ea8aa23dbe7bcfe9de" candidate_uuid: "903b3435239241eab01ea54ffb015ee3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:33.904747 16076 raft_consensus.cc:3060] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:33.906508 16787 consensus_queue.cc:1048] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:33.905252 16740 consensus_queue.cc:237] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:33.911988 16076 raft_consensus.cc:2468] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 903b3435239241eab01ea54ffb015ee3 in term 2.
I20260709 13:16:33.913232 15972 leader_election.cc:304] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.912295 16005 raft_consensus.cc:1275] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Refusing update from remote peer d4d6e13a8f4a473098351ada96fd7dab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:33.920121 16772 leader_election.cc:290] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [CANDIDATE]: Term 2 election: Requested vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:33.921845 16784 raft_consensus.cc:2804] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:33.922399 16737 consensus_queue.cc:1048] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:33.922669 16784 raft_consensus.cc:697] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Becoming Leader. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:33.924196 16784 consensus_queue.cc:237] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
W20260709 13:16:33.929891 16045 leader_election.cc:336] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.938872 16045 leader_election.cc:304] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d4d6e13a8f4a473098351ada96fd7dab; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:33.922257 15827 catalog_manager.cc:5697] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:33.943503 16155 raft_consensus.cc:1275] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Refusing update from remote peer d4d6e13a8f4a473098351ada96fd7dab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:33.949540 16651 consensus_queue.cc:1048] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:33.964804 16789 raft_consensus.cc:2749] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:33.971212 16045 leader_election.cc:336] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:33.986981 16761 tablet_bootstrap.cc:492] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:33.988018 16761 tablet_bootstrap.cc:492] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:33.990574 16761 ts_tablet_manager.cc:1403] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.149s	user 0.011s	sys 0.020s
I20260709 13:16:33.995060 16153 raft_consensus.cc:1275] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Refusing update from remote peer d4d6e13a8f4a473098351ada96fd7dab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:33.995589 16761 raft_consensus.cc:359] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:33.996273 16761 raft_consensus.cc:740] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:33.996892 16761 consensus_queue.cc:260] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
W20260709 13:16:34.007292 16045 consensus_peers.cc:597] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:34.007704 16045 consensus_peers.cc:597] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:34.019097 16153 raft_consensus.cc:1275] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Refusing update from remote peer 903b3435239241eab01ea54ffb015ee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:34.021754 16777 consensus_queue.cc:1048] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:34.006840 16770 consensus_queue.cc:1048] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:16:34.028148 15831 catalog_manager.cc:5697] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:34.034911 15831 catalog_manager.cc:5697] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 2 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:34.043527 15827 catalog_manager.cc:5697] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:34.044917 16045 consensus_peers.cc:597] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:34.045282 15827 catalog_manager.cc:5697] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:34.051599 15827 catalog_manager.cc:5697] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to 903b3435239241eab01ea54ffb015ee3 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "903b3435239241eab01ea54ffb015ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:34.063359 16761 ts_tablet_manager.cc:1434] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.071s	user 0.015s	sys 0.015s
I20260709 13:16:34.064554 16761 tablet_bootstrap.cc:492] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:34.080111 15830 catalog_manager.cc:5697] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 1 to 2, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:34.095399 15971 consensus_peers.cc:597] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:34.109696 16808 log.cc:927] Time spent T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:16:34.175863 16811 log.cc:927] Time spent T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:16:34.178263 16152 raft_consensus.cc:1275] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Refusing update from remote peer d4d6e13a8f4a473098351ada96fd7dab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:34.181254 16790 consensus_queue.cc:1048] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:34.209919 16081 raft_consensus.cc:1275] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer 903b3435239241eab01ea54ffb015ee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:34.220886 16740 consensus_queue.cc:1048] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:34.232877 15971 consensus_peers.cc:597] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:34.273319 16119 consensus_peers.cc:597] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:34.308787  6937 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:16:34.309232  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.535s	user 0.264s	sys 0.000s
W20260709 13:16:34.311576 15971 consensus_peers.cc:597] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.312279 15971 consensus_peers.cc:597] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.332563 16045 consensus_peers.cc:597] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:34.356643 16155 raft_consensus.cc:1275] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Refusing update from remote peer 903b3435239241eab01ea54ffb015ee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:16:34.356669 16045 consensus_peers.cc:597] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:34.361279 15971 consensus_peers.cc:597] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:34.363420 16152 raft_consensus.cc:1275] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Refusing update from remote peer d4d6e13a8f4a473098351ada96fd7dab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:34.371320 16761 tablet_bootstrap.cc:492] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:34.372288 16761 tablet_bootstrap.cc:492] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:34.372901 16761 ts_tablet_manager.cc:1403] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.309s	user 0.038s	sys 0.083s
I20260709 13:16:34.373395 16789 consensus_queue.cc:1048] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:34.375435 16761 raft_consensus.cc:359] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:34.376125 16761 raft_consensus.cc:740] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.376686 16005 raft_consensus.cc:1275] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Refusing update from remote peer c95f0816c41142699d80a2e326e17ab8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:34.376763 16761 consensus_queue.cc:260] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:34.378927 16761 ts_tablet_manager.cc:1434] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.002s
I20260709 13:16:34.379695 16761 tablet_bootstrap.cc:492] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
W20260709 13:16:34.380131 16045 consensus_peers.cc:597] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:34.387215 16749 consensus_queue.cc:1048] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:34.390710 15971 consensus_peers.cc:597] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:34.391321 15971 consensus_peers.cc:597] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.391852 15971 consensus_peers.cc:597] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:34.393672 16782 consensus_queue.cc:1048] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:34.406582 16761 tablet_bootstrap.cc:492] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:16:34.407015 16119 consensus_peers.cc:597] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.407598 16119 consensus_peers.cc:597] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:34.411098 16005 raft_consensus.cc:1275] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Refusing update from remote peer c95f0816c41142699d80a2e326e17ab8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:34.412416 16749 consensus_queue.cc:1048] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:34.407605 16761 tablet_bootstrap.cc:492] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:34.413492 16081 raft_consensus.cc:1275] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer 903b3435239241eab01ea54ffb015ee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:34.419703 16761 ts_tablet_manager.cc:1403] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.001s
I20260709 13:16:34.420128 16777 consensus_queue.cc:1048] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:34.429104 16003 raft_consensus.cc:1275] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Refusing update from remote peer d4d6e13a8f4a473098351ada96fd7dab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:34.430967 16789 consensus_queue.cc:1048] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:34.455039 16077 raft_consensus.cc:1275] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer 903b3435239241eab01ea54ffb015ee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:34.452744 16761 raft_consensus.cc:359] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:34.463480 16761 raft_consensus.cc:740] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.464427 16761 consensus_queue.cc:260] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:34.471922 16761 ts_tablet_manager.cc:1434] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.001s
I20260709 13:16:34.472930 16761 tablet_bootstrap.cc:492] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:34.490796 16785 consensus_queue.cc:1048] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
W20260709 13:16:34.527369 16045 consensus_peers.cc:597] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.528124 16045 consensus_peers.cc:597] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.538409 15971 consensus_peers.cc:597] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.559846 16822 log.cc:927] Time spent T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 13:16:34.564497 16045 consensus_peers.cc:597] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.583595 15972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:32795, user_credentials={real_user=slave}} blocked reactor thread for 92604.7us
I20260709 13:16:34.636302 16761 tablet_bootstrap.cc:492] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:16:34.650204 16120 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:32795, user_credentials={real_user=slave}} blocked reactor thread for 84146.9us
I20260709 13:16:34.660363 16761 tablet_bootstrap.cc:492] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:34.661449 16761 ts_tablet_manager.cc:1403] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.189s	user 0.019s	sys 0.008s
W20260709 13:16:34.676282 16119 consensus_peers.cc:597] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.679639 16045 consensus_peers.cc:597] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.683517 16800 log.cc:927] Time spent T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3: Append to log took a long time: real 0.174s	user 0.005s	sys 0.000s
W20260709 13:16:34.684269 16045 consensus_peers.cc:597] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:34.691915 16761 raft_consensus.cc:359] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:34.692708 16761 raft_consensus.cc:740] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.693667 16761 consensus_queue.cc:260] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:34.695832 16761 ts_tablet_manager.cc:1434] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 13:16:34.696578 16761 tablet_bootstrap.cc:492] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
W20260709 13:16:34.716498 16045 consensus_peers.cc:597] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.717056 16045 consensus_peers.cc:597] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.717464 16045 consensus_peers.cc:597] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:34.766916 16761 tablet_bootstrap.cc:492] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:34.767867 16761 tablet_bootstrap.cc:492] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:34.768472 16761 ts_tablet_manager.cc:1403] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.072s	user 0.015s	sys 0.000s
W20260709 13:16:34.765025 15971 consensus_peers.cc:597] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.781787 15971 consensus_peers.cc:597] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.783152 15972 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:32795, user_credentials={real_user=slave}} blocked reactor thread for 164369us
W20260709 13:16:34.797883 15971 consensus_peers.cc:597] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.798398 15971 consensus_peers.cc:597] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:34.799150 16761 raft_consensus.cc:359] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:34.799961 16761 raft_consensus.cc:740] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.800753 16761 consensus_queue.cc:260] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:34.802074 16076 raft_consensus.cc:1275] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer 903b3435239241eab01ea54ffb015ee3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:34.807714 16761 ts_tablet_manager.cc:1434] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 13:16:34.808590 16761 tablet_bootstrap.cc:492] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
W20260709 13:16:34.811252 15971 consensus_peers.cc:597] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:34.813444 15971 consensus_peers.cc:597] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:34.844400 16785 consensus_queue.cc:1048] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:34.847798 16761 tablet_bootstrap.cc:492] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:34.848799 16761 tablet_bootstrap.cc:492] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:34.849426 16761 ts_tablet_manager.cc:1403] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.001s
I20260709 13:16:34.851969 16761 raft_consensus.cc:359] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:34.852635 16761 raft_consensus.cc:740] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.863453 16761 consensus_queue.cc:260] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:34.875804 16761 ts_tablet_manager.cc:1434] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:16:34.876725 16761 tablet_bootstrap.cc:492] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
W20260709 13:16:34.890867 15971 consensus_peers.cc:597] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:34.937745 16761 tablet_bootstrap.cc:492] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:34.938776 16761 tablet_bootstrap.cc:492] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:34.939468 16761 ts_tablet_manager.cc:1403] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.010s
I20260709 13:16:34.942163 16761 raft_consensus.cc:359] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:34.942914 16761 raft_consensus.cc:740] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.943804 16761 consensus_queue.cc:260] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:34.945910 16761 ts_tablet_manager.cc:1434] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.002s
I20260709 13:16:34.946734 16761 tablet_bootstrap.cc:492] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:34.980084 16761 tablet_bootstrap.cc:492] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:34.981079 16761 tablet_bootstrap.cc:492] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:34.981731 16761 ts_tablet_manager.cc:1403] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.009s
I20260709 13:16:34.984331 16761 raft_consensus.cc:359] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:34.985037 16761 raft_consensus.cc:740] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:34.985831 16761 consensus_queue.cc:260] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:34.988047 16761 ts_tablet_manager.cc:1434] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:16:34.988849 16761 tablet_bootstrap.cc:492] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.006558 16761 tablet_bootstrap.cc:492] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.007535 16761 tablet_bootstrap.cc:492] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.008173 16761 ts_tablet_manager.cc:1403] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 13:16:35.010788 16761 raft_consensus.cc:359] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:35.011498 16761 raft_consensus.cc:740] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.012530 16761 consensus_queue.cc:260] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:35.014675 16761 ts_tablet_manager.cc:1434] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:16:35.015444 16761 tablet_bootstrap.cc:492] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.038175 16761 tablet_bootstrap.cc:492] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.039179 16761 tablet_bootstrap.cc:492] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.039827 16761 ts_tablet_manager.cc:1403] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.009s
I20260709 13:16:35.042570 16761 raft_consensus.cc:359] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:35.043507 16761 raft_consensus.cc:740] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.044176 16761 consensus_queue.cc:260] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:35.046411 16761 ts_tablet_manager.cc:1434] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:16:35.047156 16761 tablet_bootstrap.cc:492] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.245965 16761 tablet_bootstrap.cc:492] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.254403 16761 tablet_bootstrap.cc:492] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.260800 16761 ts_tablet_manager.cc:1403] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.214s	user 0.018s	sys 0.011s
I20260709 13:16:35.278612 16761 raft_consensus.cc:359] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.279525 16761 raft_consensus.cc:740] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.280578 16761 consensus_queue.cc:260] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.285351 16761 ts_tablet_manager.cc:1434] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:16:35.286469 16761 tablet_bootstrap.cc:492] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
W20260709 13:16:35.316416 15971 consensus_peers.cc:597] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:35.316872 16761 tablet_bootstrap.cc:492] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.318076 16761 tablet_bootstrap.cc:492] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.318941 16761 ts_tablet_manager.cc:1403] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.011s
I20260709 13:16:35.322897  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.65:45841
I20260709 13:16:35.322549 16761 raft_consensus.cc:359] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.323261 16906 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.65:45841 every 8 connection(s)
I20260709 13:16:35.323443 16761 raft_consensus.cc:740] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.324313 16761 consensus_queue.cc:260] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.327682 16761 ts_tablet_manager.cc:1434] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:16:35.328722 16761 tablet_bootstrap.cc:492] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.401966 16761 tablet_bootstrap.cc:492] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.403067 16761 tablet_bootstrap.cc:492] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.403851 16761 ts_tablet_manager.cc:1403] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.075s	user 0.022s	sys 0.001s
I20260709 13:16:35.406783 16761 raft_consensus.cc:359] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:35.407604 16761 raft_consensus.cc:740] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.408504 16761 consensus_queue.cc:260] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:35.409806 16907 heartbeater.cc:344] Connected to a master server at 127.6.198.126:41247
I20260709 13:16:35.410265 16907 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:35.411229 16907 heartbeater.cc:507] Master 127.6.198.126:41247 requested a full tablet report, sending...
I20260709 13:16:35.411345 16761 ts_tablet_manager.cc:1434] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:16:35.412292 16761 tablet_bootstrap.cc:492] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.436566 15827 ts_manager.cc:194] Re-registered known tserver with Master: bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:35.441118 16872 raft_consensus.cc:3060] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.444860 16761 tablet_bootstrap.cc:492] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.445900 16761 tablet_bootstrap.cc:492] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.446727 16761 ts_tablet_manager.cc:1403] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.000s
I20260709 13:16:35.449811 16761 raft_consensus.cc:359] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.450519 16761 raft_consensus.cc:740] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.451428 16761 consensus_queue.cc:260] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.453583 16761 ts_tablet_manager.cc:1434] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:16:35.454421 16761 tablet_bootstrap.cc:492] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.465097 16873 raft_consensus.cc:3060] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.499894 16761 tablet_bootstrap.cc:492] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.500406 16872 raft_consensus.cc:3060] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.500921 16761 tablet_bootstrap.cc:492] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.503891 16761 ts_tablet_manager.cc:1403] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.050s	user 0.014s	sys 0.013s
I20260709 13:16:35.504343 15827 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55080
I20260709 13:16:35.506593 16761 raft_consensus.cc:359] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:35.507340 16761 raft_consensus.cc:740] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.508132 16761 consensus_queue.cc:260] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:35.509028 16907 heartbeater.cc:499] Master 127.6.198.126:41247 was elected leader, sending a full tablet report...
I20260709 13:16:35.510480 16761 ts_tablet_manager.cc:1434] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:35.511341 16761 tablet_bootstrap.cc:492] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.532218 16761 tablet_bootstrap.cc:492] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.533147 16761 tablet_bootstrap.cc:492] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.533810 16761 ts_tablet_manager.cc:1403] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.000s
I20260709 13:16:35.537544 16761 raft_consensus.cc:359] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.538169 16761 raft_consensus.cc:740] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.538954 16761 consensus_queue.cc:260] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.541711 16761 ts_tablet_manager.cc:1434] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:16:35.542757 16761 tablet_bootstrap.cc:492] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.557518 16761 tablet_bootstrap.cc:492] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.558429 16761 tablet_bootstrap.cc:492] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.559048 16761 ts_tablet_manager.cc:1403] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.004s
I20260709 13:16:35.561749 16761 raft_consensus.cc:359] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.562552 16761 raft_consensus.cc:740] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.563377 16761 consensus_queue.cc:260] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.564002 16807 raft_consensus.cc:493] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.573726 16807 raft_consensus.cc:515] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:35.574790 16761 ts_tablet_manager.cc:1434] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.008s
I20260709 13:16:35.575662 16761 tablet_bootstrap.cc:492] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.582679 16807 leader_election.cc:290] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:35.611814 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9df103185604f988d3abe94ae731141" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:35.620894 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9df103185604f988d3abe94ae731141" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "903b3435239241eab01ea54ffb015ee3" is_pre_election: true
I20260709 13:16:35.622839 16756 leader_election.cc:304] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:35.624078 16807 raft_consensus.cc:2749] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:35.626984 16761 tablet_bootstrap.cc:492] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.627727 16761 tablet_bootstrap.cc:492] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.628327 16761 ts_tablet_manager.cc:1403] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.014s
I20260709 13:16:35.630534 16761 raft_consensus.cc:359] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:35.631070 16761 raft_consensus.cc:740] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.631702 16761 consensus_queue.cc:260] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:35.633419 16761 ts_tablet_manager.cc:1434] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:16:35.634251 16761 tablet_bootstrap.cc:492] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.652781 16761 tablet_bootstrap.cc:492] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.653625 16761 tablet_bootstrap.cc:492] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.654232 16761 ts_tablet_manager.cc:1403] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.000s
I20260709 13:16:35.656658 16761 raft_consensus.cc:359] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:35.657300 16761 raft_consensus.cc:740] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.658067 16761 consensus_queue.cc:260] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:35.660043 16761 ts_tablet_manager.cc:1434] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:16:35.661288 16761 tablet_bootstrap.cc:492] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.676256 16761 tablet_bootstrap.cc:492] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.677243 16761 tablet_bootstrap.cc:492] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.677852 16761 ts_tablet_manager.cc:1403] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.000s
I20260709 13:16:35.680060 16761 raft_consensus.cc:359] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.680694 16761 raft_consensus.cc:740] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.681648 16761 consensus_queue.cc:260] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.683614 16761 ts_tablet_manager.cc:1434] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:16:35.689563 16761 tablet_bootstrap.cc:492] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.707614 16761 tablet_bootstrap.cc:492] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.708496 16761 tablet_bootstrap.cc:492] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.709079 16898 raft_consensus.cc:493] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.709118 16761 ts_tablet_manager.cc:1403] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.003s
I20260709 13:16:35.709602 16898 raft_consensus.cc:515] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.711488 16898 leader_election.cc:290] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:35.711984 16761 raft_consensus.cc:359] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:35.712368 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0c303e2dd466caf0e684c6ea83284" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8" is_pre_election: true
I20260709 13:16:35.712700 16761 raft_consensus.cc:740] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.712898 16155 raft_consensus.cc:2468] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 2.
I20260709 13:16:35.713569 16761 consensus_queue.cc:260] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:35.713857 16048 leader_election.cc:304] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:35.714648 16898 raft_consensus.cc:2804] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:16:35.715088 16898 raft_consensus.cc:493] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:35.715480 16898 raft_consensus.cc:3060] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:35.716831 16761 ts_tablet_manager.cc:1434] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:16:35.717698 16761 tablet_bootstrap.cc:492] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.726701 16898 raft_consensus.cc:515] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.728715 16898 leader_election.cc:290] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 3 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
I20260709 13:16:35.732262 16155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0c303e2dd466caf0e684c6ea83284" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:35.733321 16155 raft_consensus.cc:3060] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:35.733677 16761 tablet_bootstrap.cc:492] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.734613 16761 tablet_bootstrap.cc:492] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.735206 16761 ts_tablet_manager.cc:1403] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.003s
I20260709 13:16:35.737754 16761 raft_consensus.cc:359] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.738418 16761 raft_consensus.cc:740] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.739066 16761 consensus_queue.cc:260] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.741087 16155 raft_consensus.cc:2468] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4d6e13a8f4a473098351ada96fd7dab in term 3.
I20260709 13:16:35.741987 16761 ts_tablet_manager.cc:1434] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:16:35.742847 16761 tablet_bootstrap.cc:492] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.743553 16048 leader_election.cc:304] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:35.744647 16469 raft_consensus.cc:2804] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:35.745107 16469 raft_consensus.cc:697] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 3 LEADER]: Becoming Leader. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:35.745844 16469 consensus_queue.cc:237] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:35.769368 16761 tablet_bootstrap.cc:492] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.770298 16761 tablet_bootstrap.cc:492] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.770063 15830 catalog_manager.cc:5697] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab reported cstate change: term changed from 1 to 3, leader changed from bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65) to d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67). New cstate: current_term: 3 leader_uuid: "d4d6e13a8f4a473098351ada96fd7dab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:35.771027 16761 ts_tablet_manager.cc:1403] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.000s
I20260709 13:16:35.773700 16761 raft_consensus.cc:359] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:35.774322 16761 raft_consensus.cc:740] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.775125 16761 consensus_queue.cc:260] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:35.777182 16761 ts_tablet_manager.cc:1434] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.004s
I20260709 13:16:35.777879 16761 tablet_bootstrap.cc:492] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.800421 16761 tablet_bootstrap.cc:492] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:35.801434 16761 tablet_bootstrap.cc:492] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:35.802067 16761 ts_tablet_manager.cc:1403] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.008s
I20260709 13:16:35.804724 16761 raft_consensus.cc:359] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:35.805379 16761 raft_consensus.cc:740] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:35.806321 16761 consensus_queue.cc:260] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:35.821370 16873 raft_consensus.cc:3060] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.837450 16876 raft_consensus.cc:3060] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.845896 16871 raft_consensus.cc:3060] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.868311 16875 raft_consensus.cc:3060] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:35.878677 16872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0c303e2dd466caf0e684c6ea83284" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" is_pre_election: true
I20260709 13:16:35.880086 16874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0c303e2dd466caf0e684c6ea83284" candidate_uuid: "d4d6e13a8f4a473098351ada96fd7dab" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
W20260709 13:16:35.881461 16045 consensus_peers.cc:597] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:35.908048 16045 leader_election.cc:343] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:35.916374 16045 leader_election.cc:343] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:35.954614 16761 ts_tablet_manager.cc:1434] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.152s	user 0.012s	sys 0.001s
I20260709 13:16:35.955741 16761 tablet_bootstrap.cc:492] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:35.993705 16874 raft_consensus.cc:3060] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:36.106190 16761 tablet_bootstrap.cc:492] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.107456 16761 tablet_bootstrap.cc:492] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.108101 16761 ts_tablet_manager.cc:1403] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.153s	user 0.014s	sys 0.010s
W20260709 13:16:36.109540 16119 consensus_peers.cc:597] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:36.110441 16761 raft_consensus.cc:359] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:36.111404 16761 raft_consensus.cc:740] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.112293 16761 consensus_queue.cc:260] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
W20260709 13:16:36.121986 16119 consensus_peers.cc:597] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:36.114450 16761 ts_tablet_manager.cc:1434] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:16:36.123824 16761 tablet_bootstrap.cc:492] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.162719 16761 tablet_bootstrap.cc:492] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.163657 16761 tablet_bootstrap.cc:492] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.164287 16761 ts_tablet_manager.cc:1403] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.041s	user 0.019s	sys 0.001s
I20260709 13:16:36.166733 16761 raft_consensus.cc:359] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.167408 16761 raft_consensus.cc:740] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.168036 16761 consensus_queue.cc:260] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.170298 16761 ts_tablet_manager.cc:1434] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:36.171090 16761 tablet_bootstrap.cc:492] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
W20260709 13:16:36.189404 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:36069, user_credentials={real_user=slave}} blocked reactor thread for 72949.4us
I20260709 13:16:36.204279 16761 tablet_bootstrap.cc:492] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.205231 16761 tablet_bootstrap.cc:492] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.205861 16761 ts_tablet_manager.cc:1403] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.008s
I20260709 13:16:36.208356 16761 raft_consensus.cc:359] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:36.209787 16761 raft_consensus.cc:740] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.210495 16761 consensus_queue.cc:260] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:36.213099 16761 ts_tablet_manager.cc:1434] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:16:36.213997 16761 tablet_bootstrap.cc:492] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.247169 16761 tablet_bootstrap.cc:492] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.248126 16761 tablet_bootstrap.cc:492] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.248775 16761 ts_tablet_manager.cc:1403] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.008s
I20260709 13:16:36.252274 16761 raft_consensus.cc:359] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:36.252893 16761 raft_consensus.cc:740] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.253583 16761 consensus_queue.cc:260] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:36.255528 16761 ts_tablet_manager.cc:1434] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:16:36.256454 16761 tablet_bootstrap.cc:492] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.259799 16874 raft_consensus.cc:3060] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:36.266274 16155 raft_consensus.cc:1275] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Refusing update from remote peer d4d6e13a8f4a473098351ada96fd7dab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:36.275444 16943 consensus_queue.cc:1048] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:36.351225 16871 raft_consensus.cc:3060] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:16:36.374315 16871 raft_consensus.cc:1275] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Refusing update from remote peer d4d6e13a8f4a473098351ada96fd7dab: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:16:36.381913 15971 consensus_peers.cc:597] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:36.387612 16943 consensus_queue.cc:1048] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:36.412528 16761 tablet_bootstrap.cc:492] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.413468 16761 tablet_bootstrap.cc:492] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.419152 16761 ts_tablet_manager.cc:1403] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.163s	user 0.033s	sys 0.064s
I20260709 13:16:36.421700 16761 raft_consensus.cc:359] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:36.422365 16761 raft_consensus.cc:740] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.426374 16761 consensus_queue.cc:260] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:36.428552 16761 ts_tablet_manager.cc:1434] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.000s
I20260709 13:16:36.429296 16761 tablet_bootstrap.cc:492] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.490803 16761 tablet_bootstrap.cc:492] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.492159 16761 tablet_bootstrap.cc:492] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.492885 16761 ts_tablet_manager.cc:1403] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.064s	user 0.020s	sys 0.006s
I20260709 13:16:36.495620 16761 raft_consensus.cc:359] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:36.496368 16761 raft_consensus.cc:740] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.497323 16761 consensus_queue.cc:260] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:36.499890 16761 ts_tablet_manager.cc:1434] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:16:36.500725 16761 tablet_bootstrap.cc:492] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.531301 16761 tablet_bootstrap.cc:492] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.532063 16761 tablet_bootstrap.cc:492] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.532678 16761 ts_tablet_manager.cc:1403] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.006s
I20260709 13:16:36.534663 16761 raft_consensus.cc:359] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.535285 16761 raft_consensus.cc:740] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.535970 16761 consensus_queue.cc:260] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.542392 16761 ts_tablet_manager.cc:1434] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:16:36.543035 16761 tablet_bootstrap.cc:492] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.560174 16761 tablet_bootstrap.cc:492] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.561110 16761 tablet_bootstrap.cc:492] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.561748 16761 ts_tablet_manager.cc:1403] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.001s
I20260709 13:16:36.564298 16761 raft_consensus.cc:359] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.565004 16761 raft_consensus.cc:740] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.569432 16761 consensus_queue.cc:260] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.571448 16761 ts_tablet_manager.cc:1434] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 13:16:36.573601 16761 tablet_bootstrap.cc:492] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.587634 16761 tablet_bootstrap.cc:492] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.588481 16761 tablet_bootstrap.cc:492] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.589165 16761 ts_tablet_manager.cc:1403] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.003s
I20260709 13:16:36.591616 16761 raft_consensus.cc:359] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:36.592301 16761 raft_consensus.cc:740] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.592962 16761 consensus_queue.cc:260] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:36.594619 16761 ts_tablet_manager.cc:1434] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:16:36.596051 16761 tablet_bootstrap.cc:492] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.619846 16761 tablet_bootstrap.cc:492] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.620741 16761 tablet_bootstrap.cc:492] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.621369 16761 ts_tablet_manager.cc:1403] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.004s
I20260709 13:16:36.623944 16761 raft_consensus.cc:359] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:36.624632 16761 raft_consensus.cc:740] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.625344 16761 consensus_queue.cc:260] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:36.627561 16761 ts_tablet_manager.cc:1434] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:16:36.628720 16761 tablet_bootstrap.cc:492] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.643572 16761 tablet_bootstrap.cc:492] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.644460 16761 tablet_bootstrap.cc:492] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.645119 16761 ts_tablet_manager.cc:1403] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.017s	user 0.006s	sys 0.008s
I20260709 13:16:36.647619 16761 raft_consensus.cc:359] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:36.648284 16761 raft_consensus.cc:740] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.648939 16761 consensus_queue.cc:260] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:36.651084 16761 ts_tablet_manager.cc:1434] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:16:36.652381 16761 tablet_bootstrap.cc:492] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.653107 16874 raft_consensus.cc:3060] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:36.677914 16761 tablet_bootstrap.cc:492] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.679936 16761 tablet_bootstrap.cc:492] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.680598 16761 ts_tablet_manager.cc:1403] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.009s
I20260709 13:16:36.682945 16761 raft_consensus.cc:359] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.683642 16761 raft_consensus.cc:740] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.684320 16761 consensus_queue.cc:260] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.686757 16761 ts_tablet_manager.cc:1434] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:16:36.687573 16761 tablet_bootstrap.cc:492] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
W20260709 13:16:36.702165 15971 consensus_peers.cc:597] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:16:36.711840 16874 raft_consensus.cc:3060] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:36.713479 16761 tablet_bootstrap.cc:492] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=1 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.714442 16761 tablet_bootstrap.cc:492] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.715279 16761 ts_tablet_manager.cc:1403] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.007s
I20260709 13:16:36.717804 16761 raft_consensus.cc:359] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.718467 16761 raft_consensus.cc:740] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.720312 16761 consensus_queue.cc:260] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.738721 16761 ts_tablet_manager.cc:1434] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.023s	user 0.014s	sys 0.007s
I20260709 13:16:36.739706 16761 tablet_bootstrap.cc:492] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.771198 16761 tablet_bootstrap.cc:492] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.772102 16761 tablet_bootstrap.cc:492] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.772693 16761 ts_tablet_manager.cc:1403] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.014s
I20260709 13:16:36.774998 16761 raft_consensus.cc:359] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.775619 16761 raft_consensus.cc:740] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.776182 16761 consensus_queue.cc:260] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.778093 16761 ts_tablet_manager.cc:1434] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.001s
I20260709 13:16:36.779474 16761 tablet_bootstrap.cc:492] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.779572 16871 raft_consensus.cc:3060] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:36.823439 16761 tablet_bootstrap.cc:492] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.824236 16761 tablet_bootstrap.cc:492] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.824891 16761 ts_tablet_manager.cc:1403] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.013s
I20260709 13:16:36.827872 16761 raft_consensus.cc:359] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:36.828528 16761 raft_consensus.cc:740] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.829144 16761 consensus_queue.cc:260] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:36.831316 16761 ts_tablet_manager.cc:1434] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:16:36.832259 16761 tablet_bootstrap.cc:492] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.849761 16761 tablet_bootstrap.cc:492] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.850643 16761 tablet_bootstrap.cc:492] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.851238 16761 ts_tablet_manager.cc:1403] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.007s
I20260709 13:16:36.853538 16761 raft_consensus.cc:359] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.860217 16761 raft_consensus.cc:740] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.861013 16761 consensus_queue.cc:260] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.862815 16761 ts_tablet_manager.cc:1434] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:16:36.863626 16761 tablet_bootstrap.cc:492] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.898840 16761 tablet_bootstrap.cc:492] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.900128 16761 tablet_bootstrap.cc:492] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.900724 16761 ts_tablet_manager.cc:1403] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.037s	user 0.021s	sys 0.002s
I20260709 13:16:36.902730 16761 raft_consensus.cc:359] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:36.903298 16761 raft_consensus.cc:740] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.903959 16761 consensus_queue.cc:260] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:36.908545 16761 ts_tablet_manager.cc:1434] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:16:36.910461 16761 tablet_bootstrap.cc:492] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.929587 16761 tablet_bootstrap.cc:492] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.930486 16761 tablet_bootstrap.cc:492] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.931106 16761 ts_tablet_manager.cc:1403] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.006s
I20260709 13:16:36.933331 16761 raft_consensus.cc:359] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:36.933779 16761 raft_consensus.cc:740] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.934310 16761 consensus_queue.cc:260] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:36.936060 16761 ts_tablet_manager.cc:1434] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.001s
I20260709 13:16:36.937016 16761 tablet_bootstrap.cc:492] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.963702 16761 tablet_bootstrap.cc:492] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.964619 16761 tablet_bootstrap.cc:492] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.965242 16761 ts_tablet_manager.cc:1403] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.000s
I20260709 13:16:36.967700 16761 raft_consensus.cc:359] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.968333 16761 raft_consensus.cc:740] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.969017 16761 consensus_queue.cc:260] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:36.971176 16761 ts_tablet_manager.cc:1434] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:16:36.972296 16761 tablet_bootstrap.cc:492] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:36.986759 16761 tablet_bootstrap.cc:492] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:36.987610 16761 tablet_bootstrap.cc:492] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:36.988097 16761 ts_tablet_manager.cc:1403] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 13:16:36.991397 16761 raft_consensus.cc:359] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:36.991973 16761 raft_consensus.cc:740] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:36.992690 16761 consensus_queue.cc:260] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:36.994638 16761 ts_tablet_manager.cc:1434] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:16:36.995560 16761 tablet_bootstrap.cc:492] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap starting.
I20260709 13:16:37.011116 16761 tablet_bootstrap.cc:492] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:16:37.011798 16761 tablet_bootstrap.cc:492] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e: Bootstrap complete.
I20260709 13:16:37.012296 16761 ts_tablet_manager.cc:1403] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 13:16:37.014613 16761 raft_consensus.cc:359] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:37.015102 16761 raft_consensus.cc:740] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Initialized, Role: FOLLOWER
I20260709 13:16:37.015854 16761 consensus_queue.cc:260] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:37.017367 16761 ts_tablet_manager.cc:1434] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:16:37.028787 16871 raft_consensus.cc:3060] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:37.119822 16871 raft_consensus.cc:3060] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:37.177583 16876 raft_consensus.cc:3060] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:37.261080 16871 raft_consensus.cc:3060] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.384145  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:45.436514 16005 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c53d28c984a4016b8461698b47d3300"
dest_uuid: "903b3435239241eab01ea54ffb015ee3"
mode: GRACEFUL
new_leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:53726
I20260709 13:16:45.437249 16005 raft_consensus.cc:606] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Received request to transfer leadership to TS bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:45.444110 16005 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74"
dest_uuid: "903b3435239241eab01ea54ffb015ee3"
mode: GRACEFUL
new_leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:53726
I20260709 13:16:45.445173 16005 raft_consensus.cc:606] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Received request to transfer leadership to TS bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:45.462404 16073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cca9877952a413a9112ea61481bc582"
dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
mode: GRACEFUL
new_leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:41666
I20260709 13:16:45.463068 16073 raft_consensus.cc:606] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Received request to transfer leadership to TS bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:45.466156 16005 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9719c47150e74128b537e918110116e0"
dest_uuid: "903b3435239241eab01ea54ffb015ee3"
mode: GRACEFUL
new_leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:53726
I20260709 13:16:45.466715 16005 raft_consensus.cc:606] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Received request to transfer leadership to TS bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:45.470541 16005 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5e6891e9cab4a7298664c3c431ce36d"
dest_uuid: "903b3435239241eab01ea54ffb015ee3"
mode: GRACEFUL
new_leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:53726
I20260709 13:16:45.471199 16005 raft_consensus.cc:606] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Received request to transfer leadership to TS bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:45.473980 16005 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9df103185604f988d3abe94ae731141"
dest_uuid: "903b3435239241eab01ea54ffb015ee3"
mode: GRACEFUL
new_leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:53726
I20260709 13:16:45.474560 16005 raft_consensus.cc:606] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Received request to transfer leadership to TS bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:45.475695  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:16:45.477514  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:45.510221 17004 raft_consensus.cc:993] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab: : Instructing follower bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:45.510875 17011 raft_consensus.cc:1081] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Signalling peer bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:45.512328 16871 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7cca9877952a413a9112ea61481bc582"
dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:49510
I20260709 13:16:45.512841 16871 raft_consensus.cc:493] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:45.513212 16871 raft_consensus.cc:3060] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.521068 16871 raft_consensus.cc:515] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:45.523059 16871 leader_election.cc:290] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:45.524650 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cca9877952a413a9112ea61481bc582" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:45.525394 16152 raft_consensus.cc:3060] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.531169 16152 raft_consensus.cc:2468] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:45.532487 16757 leader_election.cc:304] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:45.537672 17016 raft_consensus.cc:2804] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:45.551081 17010 raft_consensus.cc:993] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3: : Instructing follower bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:45.551704 17002 raft_consensus.cc:1081] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Signalling peer bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:45.554418 16876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74"
dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:49708
I20260709 13:16:45.554910 16876 raft_consensus.cc:493] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:45.555269 16876 raft_consensus.cc:3060] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.555182 17016 raft_consensus.cc:697] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:45.557806 17016 consensus_queue.cc:237] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:45.566398 17010 raft_consensus.cc:993] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3: : Instructing follower bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:45.566457 16073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cca9877952a413a9112ea61481bc582" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:45.567026 17010 raft_consensus.cc:1081] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Signalling peer bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:45.567215 16073 raft_consensus.cc:3055] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:45.567579 16073 raft_consensus.cc:740] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:45.568358 16073 consensus_queue.cc:260] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:45.568796 16871 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5e6891e9cab4a7298664c3c431ce36d"
dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:49708
I20260709 13:16:45.569355 16871 raft_consensus.cc:493] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:45.569681 16871 raft_consensus.cc:3060] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.569693 16073 raft_consensus.cc:3060] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.572587 16876 raft_consensus.cc:515] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:45.574807 16876 leader_election.cc:290] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 3 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:45.575400 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:45.575953 16152 raft_consensus.cc:3060] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.578495 15829 catalog_manager.cc:5697] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 1 to 2, leader changed from d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67) to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 2 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:45.580798 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4712e5b3e765415b916c9bf89f5b0d74" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:45.586203 16005 raft_consensus.cc:3055] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:45.586553 16005 raft_consensus.cc:740] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:45.587327 16005 consensus_queue.cc:260] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:45.588752 16005 raft_consensus.cc:3060] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.582314 16073 raft_consensus.cc:2468] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:45.583434 16152 raft_consensus.cc:2468] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 3.
I20260709 13:16:45.594524 16871 raft_consensus.cc:515] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:45.595278 16757 leader_election.cc:304] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:45.596069 17016 raft_consensus.cc:2804] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:45.597239 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e6891e9cab4a7298664c3c431ce36d" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:45.597882 16152 raft_consensus.cc:3060] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.598155 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5e6891e9cab4a7298664c3c431ce36d" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:45.598764 16003 raft_consensus.cc:3055] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:45.599059 16003 raft_consensus.cc:740] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:45.599160 16871 leader_election.cc:290] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 3 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:45.599993 16003 consensus_queue.cc:260] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:45.600850 16003 raft_consensus.cc:3060] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.604007 17016 raft_consensus.cc:697] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:45.605005 17016 consensus_queue.cc:237] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:45.606938 16152 raft_consensus.cc:2468] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 3.
I20260709 13:16:45.609156 16757 leader_election.cc:304] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:45.609517 16003 raft_consensus.cc:2468] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 3.
I20260709 13:16:45.610107 17017 raft_consensus.cc:2804] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:45.612511 16005 raft_consensus.cc:2468] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 3.
I20260709 13:16:45.624699 17017 raft_consensus.cc:697] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:45.626238 17017 consensus_queue.cc:237] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:45.651744 17010 raft_consensus.cc:993] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3: : Instructing follower bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:45.652379 17010 raft_consensus.cc:1081] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Signalling peer bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:45.656836 16876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9df103185604f988d3abe94ae731141"
dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:49708
I20260709 13:16:45.657492 16876 raft_consensus.cc:493] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:45.657924 16876 raft_consensus.cc:3060] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.658610 17002 raft_consensus.cc:993] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3: : Instructing follower bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:45.659132 17002 raft_consensus.cc:1081] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Signalling peer bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:45.660923 16874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9719c47150e74128b537e918110116e0"
dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:49708
I20260709 13:16:45.661535 16874 raft_consensus.cc:493] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:45.661991 16874 raft_consensus.cc:3060] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.670564 16874 raft_consensus.cc:515] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:45.673372 16874 leader_election.cc:290] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 3 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:45.673538 16073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9719c47150e74128b537e918110116e0" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:45.674137 16073 raft_consensus.cc:3060] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.675092 16876 raft_consensus.cc:515] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:45.677496 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9df103185604f988d3abe94ae731141" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:45.677829 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9719c47150e74128b537e918110116e0" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:45.678025 16152 raft_consensus.cc:3060] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.681474 16073 raft_consensus.cc:2468] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 3.
I20260709 13:16:45.683908 16876 leader_election.cc:290] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:45.684185 16755 leader_election.cc:304] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:45.684938 16003 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9df103185604f988d3abe94ae731141" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:45.685084 17017 raft_consensus.cc:2804] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:45.685560 16003 raft_consensus.cc:3055] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:45.685756 17017 raft_consensus.cc:697] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:45.685968 16003 raft_consensus.cc:740] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:45.686806 17017 consensus_queue.cc:237] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:45.687130 16003 consensus_queue.cc:260] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:45.688933 16003 raft_consensus.cc:3060] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.689960 16152 raft_consensus.cc:2468] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:45.691104 16757 leader_election.cc:304] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:45.692054 17023 raft_consensus.cc:2804] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:45.692526 17023 raft_consensus.cc:697] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:45.693466 17023 consensus_queue.cc:237] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:45.704790 16003 raft_consensus.cc:2468] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:45.678398 16005 raft_consensus.cc:3055] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:45.706055 16005 raft_consensus.cc:740] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:45.706972 16005 consensus_queue.cc:260] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:45.708102 16005 raft_consensus.cc:3060] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:45.715325 16005 raft_consensus.cc:2468] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 3.
I20260709 13:16:45.727584 15827 catalog_manager.cc:5697] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 2 to 3, leader changed from 903b3435239241eab01ea54ffb015ee3 (127.6.198.66) to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 3 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:45.728818 15827 catalog_manager.cc:5697] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 2 to 3, leader changed from 903b3435239241eab01ea54ffb015ee3 (127.6.198.66) to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 3 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:45.729725 15827 catalog_manager.cc:5697] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 2 to 3, leader changed from 903b3435239241eab01ea54ffb015ee3 (127.6.198.66) to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 3 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:45.730684 15827 catalog_manager.cc:5697] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 1 to 2, leader changed from 903b3435239241eab01ea54ffb015ee3 (127.6.198.66) to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 2 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:45.824090 17025 raft_consensus.cc:993] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3: : Instructing follower bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:45.824654 17020 raft_consensus.cc:1081] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Signalling peer bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:45.826025 16874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c53d28c984a4016b8461698b47d3300"
dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:49708
I20260709 13:16:45.826597 16874 raft_consensus.cc:493] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:45.826964 16874 raft_consensus.cc:3060] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.832538 16874 raft_consensus.cc:515] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:45.834187 16874 leader_election.cc:290] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:45.834697 16005 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c53d28c984a4016b8461698b47d3300" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:45.835304 16073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c53d28c984a4016b8461698b47d3300" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:45.835311 16005 raft_consensus.cc:3055] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:45.835912 16073 raft_consensus.cc:3060] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.835932 16005 raft_consensus.cc:740] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:45.836913 16005 consensus_queue.cc:260] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:45.837994 16005 raft_consensus.cc:3060] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:45.842309 16073 raft_consensus.cc:2468] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:45.843227 16755 leader_election.cc:304] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:45.843909 17023 raft_consensus.cc:2804] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:45.844288 17023 raft_consensus.cc:697] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:45.844318 16005 raft_consensus.cc:2468] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:45.845273 17023 consensus_queue.cc:237] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:45.855137 15827 catalog_manager.cc:5697] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 1 to 2, leader changed from 903b3435239241eab01ea54ffb015ee3 (127.6.198.66) to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 2 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:45.981546 16152 raft_consensus.cc:1275] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:45.982780 17023 consensus_queue.cc:1048] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:46.006567 16073 raft_consensus.cc:1275] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:46.008741 17023 consensus_queue.cc:1048] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:46.030462 16152 raft_consensus.cc:1275] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:46.031620 17023 consensus_queue.cc:1048] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:46.047068 16003 raft_consensus.cc:1275] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:46.050403 17023 consensus_queue.cc:1048] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:46.098891 16152 raft_consensus.cc:1275] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:46.100749 17023 consensus_queue.cc:1048] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:46.165679 16155 raft_consensus.cc:1275] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:46.167357 17023 consensus_queue.cc:1048] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:46.553965 16119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45841, user_credentials={real_user=slave}} blocked reactor thread for 202131us
W20260709 13:16:46.689783 16003 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.007s	sys 0.011s
W20260709 13:16:46.692348 16003 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.008s	sys 0.011s
I20260709 13:16:46.739374 16077 raft_consensus.cc:1275] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:46.741536 16076 raft_consensus.cc:1275] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 3 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:46.742030 17023 consensus_queue.cc:1048] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:46.745553 17023 consensus_queue.cc:1048] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:46.790630 16005 raft_consensus.cc:1275] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:46.794183 16005 raft_consensus.cc:1275] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:46.795356 16003 raft_consensus.cc:1275] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:46.798484 16003 raft_consensus.cc:1275] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:16:46.813165 15974 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:36069, user_credentials={real_user=slave}} blocked reactor thread for 77347.6us
I20260709 13:16:46.916458 17073 consensus_queue.cc:1048] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:46.920967 17073 consensus_queue.cc:1048] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:46.929148 17071 consensus_queue.cc:1048] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:46.932647 17071 consensus_queue.cc:1048] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:47.074014  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:47.129908  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:47.174719 16076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c2dba979b4b4cfdb4eb1159d36ae158"
dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
mode: GRACEFUL
new_leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:41666
I20260709 13:16:47.175263 16076 raft_consensus.cc:606] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Received request to transfer leadership to TS bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:47.179908 16008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bd99621a84f4fe994996601dae8988c"
dest_uuid: "903b3435239241eab01ea54ffb015ee3"
mode: GRACEFUL
new_leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:53726
I20260709 13:16:47.180436 16008 raft_consensus.cc:606] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Received request to transfer leadership to TS bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:47.188285 16008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "68a3e44fcd5b438486e1ec0241ef1e8d"
dest_uuid: "903b3435239241eab01ea54ffb015ee3"
mode: GRACEFUL
new_leader_uuid: "c95f0816c41142699d80a2e326e17ab8"
 from {username='slave'} at 127.0.0.1:53726
I20260709 13:16:47.188761 16008 raft_consensus.cc:606] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Received request to transfer leadership to TS c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:47.192209 16008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "69d7d4e0bc194613bb410e793fd7bd39"
dest_uuid: "903b3435239241eab01ea54ffb015ee3"
mode: GRACEFUL
new_leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:53726
I20260709 13:16:47.192689 16008 raft_consensus.cc:606] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Received request to transfer leadership to TS bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:47.195914 16008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7ddd2107d2e4e73a47f9e4f6bdc73fb"
dest_uuid: "903b3435239241eab01ea54ffb015ee3"
mode: GRACEFUL
new_leader_uuid: "c95f0816c41142699d80a2e326e17ab8"
 from {username='slave'} at 127.0.0.1:53726
I20260709 13:16:47.196461 16008 raft_consensus.cc:606] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Received request to transfer leadership to TS c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:47.199652 16008 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e38e5c73946b4d14825079217a21a21a"
dest_uuid: "903b3435239241eab01ea54ffb015ee3"
mode: GRACEFUL
new_leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:53726
I20260709 13:16:47.200258 16008 raft_consensus.cc:606] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Received request to transfer leadership to TS bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:47.201632  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:16:47.203198  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:47.216869 17057 raft_consensus.cc:993] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3: : Instructing follower c95f0816c41142699d80a2e326e17ab8 to start an election
I20260709 13:16:47.217406 17057 raft_consensus.cc:1081] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Signalling peer c95f0816c41142699d80a2e326e17ab8 to start an election
I20260709 13:16:47.219175 16153 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7ddd2107d2e4e73a47f9e4f6bdc73fb"
dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
 from {username='slave'} at 127.0.0.1:34466
I20260709 13:16:47.219703 16153 raft_consensus.cc:493] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:47.220077 16153 raft_consensus.cc:3060] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.227100 16153 raft_consensus.cc:515] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:47.229810 16153 leader_election.cc:290] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:47.230487 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ddd2107d2e4e73a47f9e4f6bdc73fb" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:47.230758 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7ddd2107d2e4e73a47f9e4f6bdc73fb" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:47.231180 16008 raft_consensus.cc:3055] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:47.231380 16076 raft_consensus.cc:3060] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.231580 16008 raft_consensus.cc:740] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:47.232525 16008 consensus_queue.cc:260] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:47.235008 16008 raft_consensus.cc:3060] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.237459 16076 raft_consensus.cc:2468] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 2.
I20260709 13:16:47.238713 16120 leader_election.cc:304] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:47.239363 17005 raft_consensus.cc:2804] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:47.239814 17005 raft_consensus.cc:697] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 2 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
I20260709 13:16:47.240564 17005 consensus_queue.cc:237] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:47.242813 16008 raft_consensus.cc:2468] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 2.
I20260709 13:16:47.298358 15831 catalog_manager.cc:5697] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 1 to 2, leader changed from 903b3435239241eab01ea54ffb015ee3 (127.6.198.66) to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:47.305856 17057 raft_consensus.cc:993] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3: : Instructing follower c95f0816c41142699d80a2e326e17ab8 to start an election
I20260709 13:16:47.306423 17057 raft_consensus.cc:1081] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Signalling peer c95f0816c41142699d80a2e326e17ab8 to start an election
I20260709 13:16:47.308492 16152 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "68a3e44fcd5b438486e1ec0241ef1e8d"
dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
 from {username='slave'} at 127.0.0.1:34466
I20260709 13:16:47.309000 16152 raft_consensus.cc:493] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:47.309350 16152 raft_consensus.cc:3060] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.318923 16152 raft_consensus.cc:515] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:47.320750 16152 leader_election.cc:290] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:47.329393 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a3e44fcd5b438486e1ec0241ef1e8d" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:47.329962 16008 raft_consensus.cc:3055] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:47.330076 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68a3e44fcd5b438486e1ec0241ef1e8d" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:47.330292 16008 raft_consensus.cc:740] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:47.330680 16076 raft_consensus.cc:3060] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.332368 16008 consensus_queue.cc:260] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:47.333681 16008 raft_consensus.cc:3060] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.337205 16076 raft_consensus.cc:2468] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 2.
I20260709 13:16:47.338161 16120 leader_election.cc:304] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:47.339238 17005 raft_consensus.cc:2804] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:47.339671 17005 raft_consensus.cc:697] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [term 2 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
I20260709 13:16:47.340965 17005 consensus_queue.cc:237] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:47.341171 16008 raft_consensus.cc:2468] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 2.
I20260709 13:16:47.349853 15827 catalog_manager.cc:5697] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 1 to 2, leader changed from 903b3435239241eab01ea54ffb015ee3 (127.6.198.66) to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:47.548646 17058 raft_consensus.cc:993] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3: : Instructing follower bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:47.549082 17060 raft_consensus.cc:1081] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Signalling peer bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:47.550426 16873 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "69d7d4e0bc194613bb410e793fd7bd39"
dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:49708
I20260709 13:16:47.550992 16873 raft_consensus.cc:493] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:47.551288 16873 raft_consensus.cc:3060] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:47.556677 16873 raft_consensus.cc:515] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:47.558943 16873 leader_election.cc:290] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 3 election: Requested vote from peers c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
I20260709 13:16:47.559005 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d7d4e0bc194613bb410e793fd7bd39" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:47.559283 16008 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "69d7d4e0bc194613bb410e793fd7bd39" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:47.559620 16152 raft_consensus.cc:3060] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:47.561462 16008 raft_consensus.cc:3055] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:47.561738 16008 raft_consensus.cc:740] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:47.562479 16008 consensus_queue.cc:260] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:47.563685 16008 raft_consensus.cc:3060] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:47.566174 17058 raft_consensus.cc:993] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3: : Instructing follower bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:47.566705 17058 raft_consensus.cc:1081] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Signalling peer bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:47.569658 16008 raft_consensus.cc:2468] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 3.
I20260709 13:16:47.566666 16152 raft_consensus.cc:2468] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 3.
I20260709 13:16:47.569907 16873 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5bd99621a84f4fe994996601dae8988c"
dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:49708
I20260709 13:16:47.570638 16873 raft_consensus.cc:493] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:47.570900 16756 leader_election.cc:304] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:47.571035 16873 raft_consensus.cc:3060] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:47.571822 17072 raft_consensus.cc:2804] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:47.572274 17072 raft_consensus.cc:697] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:47.573175 17072 consensus_queue.cc:237] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:47.577425 16873 raft_consensus.cc:515] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:47.579936 16873 leader_election.cc:290] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 3 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:47.580574 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd99621a84f4fe994996601dae8988c" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:47.580650 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bd99621a84f4fe994996601dae8988c" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:47.581337 16007 raft_consensus.cc:3055] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:16:47.581453 16076 raft_consensus.cc:3060] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:47.581727 16007 raft_consensus.cc:740] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:47.582549 16007 consensus_queue.cc:260] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:47.583393 16007 raft_consensus.cc:3060] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:16:47.585716 16076 raft_consensus.cc:2468] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 3.
I20260709 13:16:47.586521 16755 leader_election.cc:304] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:47.587190 17072 raft_consensus.cc:2804] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:16:47.587625 17072 raft_consensus.cc:697] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:47.588526 16007 raft_consensus.cc:2468] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 3.
I20260709 13:16:47.588354 17072 consensus_queue.cc:237] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:47.593763 15827 catalog_manager.cc:5697] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 2 to 3, leader changed from 903b3435239241eab01ea54ffb015ee3 (127.6.198.66) to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 3 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:47.607076 15827 catalog_manager.cc:5697] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 2 to 3, leader changed from 903b3435239241eab01ea54ffb015ee3 (127.6.198.66) to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 3 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:47.633875 17058 raft_consensus.cc:993] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3: : Instructing follower bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:47.634315 17060 raft_consensus.cc:1081] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Signalling peer bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:47.635679 16873 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e38e5c73946b4d14825079217a21a21a"
dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:49708
I20260709 13:16:47.636127 16873 raft_consensus.cc:493] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:47.636516 16873 raft_consensus.cc:3060] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.642544 16873 raft_consensus.cc:515] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:47.644068 16873 leader_election.cc:290] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:47.644891 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38e5c73946b4d14825079217a21a21a" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:47.644938 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e38e5c73946b4d14825079217a21a21a" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:47.645521 16007 raft_consensus.cc:3055] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:47.645583 16076 raft_consensus.cc:3060] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.645956 16007 raft_consensus.cc:740] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 903b3435239241eab01ea54ffb015ee3, State: Running, Role: LEADER
I20260709 13:16:47.646780 16007 consensus_queue.cc:260] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:47.648056 16007 raft_consensus.cc:3060] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.651310 16076 raft_consensus.cc:2468] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:47.652652 16755 leader_election.cc:304] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:47.653101 16007 raft_consensus.cc:2468] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:47.653373 17072 raft_consensus.cc:2804] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:47.653924 17072 raft_consensus.cc:697] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:47.654822 17072 consensus_queue.cc:237] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:47.662552 17059 raft_consensus.cc:993] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab: : Instructing follower bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:47.663153 17050 raft_consensus.cc:1081] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Signalling peer bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:47.664609 16873 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c2dba979b4b4cfdb4eb1159d36ae158"
dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:49510
I20260709 13:16:47.665242 16873 raft_consensus.cc:493] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:47.664685 15827 catalog_manager.cc:5697] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 1 to 2, leader changed from 903b3435239241eab01ea54ffb015ee3 (127.6.198.66) to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 2 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:47.665620 16873 raft_consensus.cc:3060] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.672529 16873 raft_consensus.cc:515] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:47.674854 16873 leader_election.cc:290] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:47.675239 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2dba979b4b4cfdb4eb1159d36ae158" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:47.675796 16152 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c2dba979b4b4cfdb4eb1159d36ae158" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:47.675851 16076 raft_consensus.cc:3055] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:47.676486 16152 raft_consensus.cc:3060] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.676497 16076 raft_consensus.cc:740] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:47.677536 16076 consensus_queue.cc:260] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:47.679046 16076 raft_consensus.cc:3060] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:47.680914 16073 raft_consensus.cc:1275] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer c95f0816c41142699d80a2e326e17ab8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:47.683157 16152 raft_consensus.cc:2468] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:47.684340 16757 leader_election.cc:304] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:47.685226 17072 raft_consensus.cc:2804] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:47.685374 16076 raft_consensus.cc:2468] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:47.685496 17005 consensus_queue.cc:1048] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:47.685721 17072 raft_consensus.cc:697] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:47.687112 17072 consensus_queue.cc:237] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:47.707195 16007 raft_consensus.cc:1275] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Refusing update from remote peer c95f0816c41142699d80a2e326e17ab8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:47.708567 17005 consensus_queue.cc:1048] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:47.711547 15831 catalog_manager.cc:5697] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 1 to 2, leader changed from d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67) to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 2 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:47.730732 16081 raft_consensus.cc:1275] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer c95f0816c41142699d80a2e326e17ab8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:47.733858 17045 consensus_queue.cc:1048] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:16:47.734654 16007 raft_consensus.cc:1275] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Refusing update from remote peer c95f0816c41142699d80a2e326e17ab8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:47.738417 17045 consensus_queue.cc:1048] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:16:47.836872 16103 debug-util.cc:398] Leaking SignalData structure 0x7b0800133940 after lost signal to thread 6940
W20260709 13:16:47.838116 16103 debug-util.cc:398] Leaking SignalData structure 0x7b08003385e0 after lost signal to thread 15865
W20260709 13:16:47.839432 16103 debug-util.cc:398] Leaking SignalData structure 0x7b08001a45e0 after lost signal to thread 16032
W20260709 13:16:47.840509 16103 debug-util.cc:398] Leaking SignalData structure 0x7b08000e8e60 after lost signal to thread 16106
W20260709 13:16:47.841434 16103 debug-util.cc:398] Leaking SignalData structure 0x7b08000f0540 after lost signal to thread 16180
W20260709 13:16:47.842319 16103 debug-util.cc:398] Leaking SignalData structure 0x7b080015e6e0 after lost signal to thread 16906
I20260709 13:16:48.029211 16007 raft_consensus.cc:1275] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:48.030478 17072 consensus_queue.cc:1048] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.048888 16153 raft_consensus.cc:1275] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:16:48.051573 16073 raft_consensus.cc:1275] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.050287 17073 consensus_queue.cc:1048] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:16:48.053416 17101 consensus_queue.cc:1048] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.103052 16007 raft_consensus.cc:1275] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.105523 17103 consensus_queue.cc:1048] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.119197 16073 raft_consensus.cc:1275] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.121659 17073 consensus_queue.cc:1048] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:48.144354 16153 raft_consensus.cc:1275] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:48.147769 17101 consensus_queue.cc:1048] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:48.174499 16008 raft_consensus.cc:1275] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:48.175830 17106 consensus_queue.cc:1048] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:16:48.203979  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:48.211439 16076 raft_consensus.cc:1275] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 3 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:16:48.214480 17073 consensus_queue.cc:1048] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:16:48.699635  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:48.763875 16076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "034918f94907492ca9a964c053edd50e"
dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
mode: GRACEFUL
new_leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:41666
I20260709 13:16:48.764382 16076 raft_consensus.cc:606] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Received request to transfer leadership to TS bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:48.766789 16076 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bd2e0e0244294ec18481adf638d57ff0"
dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
mode: GRACEFUL
new_leader_uuid: "c95f0816c41142699d80a2e326e17ab8"
 from {username='slave'} at 127.0.0.1:41666
I20260709 13:16:48.767351 16076 raft_consensus.cc:606] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Received request to transfer leadership to TS c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:48.772300 16073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dadf33b9db824fdcbb63864c9e975c6b"
dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
mode: GRACEFUL
new_leader_uuid: "c95f0816c41142699d80a2e326e17ab8"
 from {username='slave'} at 127.0.0.1:41666
I20260709 13:16:48.772825 16073 raft_consensus.cc:606] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Received request to transfer leadership to TS c95f0816c41142699d80a2e326e17ab8
I20260709 13:16:48.777822 16073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e58eccbbe71e465188784170045cdb34"
dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
mode: GRACEFUL
new_leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:41666
I20260709 13:16:48.778383 16073 raft_consensus.cc:606] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Received request to transfer leadership to TS bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:48.781354 16073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5565bfb243b4eeab5cbb5549d639378"
dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
mode: GRACEFUL
new_leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:41666
I20260709 13:16:48.781862 16073 raft_consensus.cc:606] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Received request to transfer leadership to TS bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:48.783828  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:48.784942  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:48.828159 17120 raft_consensus.cc:993] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab: : Instructing follower bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:48.828743 17050 raft_consensus.cc:1081] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Signalling peer bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:48.830302 16871 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e58eccbbe71e465188784170045cdb34"
dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:49510
I20260709 13:16:48.830886 16871 raft_consensus.cc:493] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:48.831235 16871 raft_consensus.cc:3060] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.838701 16871 raft_consensus.cc:515] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:48.841241 16073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58eccbbe71e465188784170045cdb34" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:48.841794 16073 raft_consensus.cc:3055] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:48.842084 16073 raft_consensus.cc:740] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:48.842629 16871 leader_election.cc:290] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:48.843103 16073 consensus_queue.cc:260] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:48.842015 16153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58eccbbe71e465188784170045cdb34" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:48.844026 16153 raft_consensus.cc:3060] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.845515 16073 raft_consensus.cc:3060] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.850538 16153 raft_consensus.cc:2468] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:48.851794 16757 leader_election.cc:304] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:48.852605 17072 raft_consensus.cc:2804] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:48.853122 17072 raft_consensus.cc:697] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:48.854171 17059 raft_consensus.cc:993] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab: : Instructing follower c95f0816c41142699d80a2e326e17ab8 to start an election
I20260709 13:16:48.854038 17072 consensus_queue.cc:237] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:48.854755 17059 raft_consensus.cc:1081] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Signalling peer c95f0816c41142699d80a2e326e17ab8 to start an election
I20260709 13:16:48.856259 16151 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dadf33b9db824fdcbb63864c9e975c6b"
dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
 from {username='slave'} at 127.0.0.1:43776
I20260709 13:16:48.856745 16151 raft_consensus.cc:493] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:48.857091 16151 raft_consensus.cc:3060] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.857197 16073 raft_consensus.cc:2468] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:48.866292 16151 raft_consensus.cc:515] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:48.868953 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dadf33b9db824fdcbb63864c9e975c6b" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:48.869462 16151 leader_election.cc:290] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:48.869521 16007 raft_consensus.cc:3060] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.869243 16073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dadf33b9db824fdcbb63864c9e975c6b" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:48.870767 16073 raft_consensus.cc:3055] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:48.871062 16073 raft_consensus.cc:740] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:48.871803 16073 consensus_queue.cc:260] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:48.872798 16073 raft_consensus.cc:3060] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.878499 16007 raft_consensus.cc:2468] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 2.
I20260709 13:16:48.879141 16073 raft_consensus.cc:2468] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 2.
I20260709 13:16:48.879894 16121 leader_election.cc:304] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:48.881399 17119 raft_consensus.cc:2804] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:48.881949 17119 raft_consensus.cc:697] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 2 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
I20260709 13:16:48.882903 17119 consensus_queue.cc:237] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:48.888943 15827 catalog_manager.cc:5697] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 1 to 2, leader changed from d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67) to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 2 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:48.902804 15829 catalog_manager.cc:5697] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 1 to 2, leader changed from d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67) to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:48.937384 17120 raft_consensus.cc:993] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab: : Instructing follower bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:48.937866 17050 raft_consensus.cc:1081] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Signalling peer bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:48.939347 16871 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f5565bfb243b4eeab5cbb5549d639378"
dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:49510
I20260709 13:16:48.939849 16871 raft_consensus.cc:493] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:48.940196 16871 raft_consensus.cc:3060] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.945581 16871 raft_consensus.cc:515] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:48.947372 16871 leader_election.cc:290] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:48.947643 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5565bfb243b4eeab5cbb5549d639378" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:48.947901 16073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5565bfb243b4eeab5cbb5549d639378" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:48.948257 16007 raft_consensus.cc:3060] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.948467 16073 raft_consensus.cc:3055] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:48.948830 16073 raft_consensus.cc:740] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:48.949493 16073 consensus_queue.cc:260] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:48.950459 16073 raft_consensus.cc:3060] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:48.955336 16007 raft_consensus.cc:2468] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:48.958806 16756 leader_election.cc:304] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e; no voters: 
I20260709 13:16:48.959599 17072 raft_consensus.cc:2804] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:48.959610 16073 raft_consensus.cc:2468] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:48.960178 17072 raft_consensus.cc:697] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:48.961143 17072 consensus_queue.cc:237] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:48.971982 15829 catalog_manager.cc:5697] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 1 to 2, leader changed from d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67) to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 2 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:49.149492 17050 raft_consensus.cc:993] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab: : Instructing follower bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:49.150645 17050 raft_consensus.cc:1081] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Signalling peer bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:49.152079 16870 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "034918f94907492ca9a964c053edd50e"
dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:49510
I20260709 13:16:49.152558 16870 raft_consensus.cc:493] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:49.152868 16870 raft_consensus.cc:3060] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.161836 16870 raft_consensus.cc:515] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:49.163967 16870 leader_election.cc:290] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Requested vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), c95f0816c41142699d80a2e326e17ab8 (127.6.198.68:36069)
I20260709 13:16:49.164853 16151 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034918f94907492ca9a964c053edd50e" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
I20260709 13:16:49.165446 16073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034918f94907492ca9a964c053edd50e" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:49.165968 16151 raft_consensus.cc:3060] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.166085 16073 raft_consensus.cc:3055] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:49.166563 16073 raft_consensus.cc:740] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:49.167443 16073 consensus_queue.cc:260] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:49.168799 16073 raft_consensus.cc:3060] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.171375 16151 raft_consensus.cc:2468] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:49.172232 16757 leader_election.cc:304] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:49.173035 17101 raft_consensus.cc:2804] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:49.173564 17101 raft_consensus.cc:697] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:49.176411 16073 raft_consensus.cc:2468] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:49.176442 17101 consensus_queue.cc:237] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:49.216073 15829 catalog_manager.cc:5697] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 1 to 2, leader changed from d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67) to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 2 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:49.258663 17120 raft_consensus.cc:993] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab: : Instructing follower c95f0816c41142699d80a2e326e17ab8 to start an election
I20260709 13:16:49.259218 17129 raft_consensus.cc:1081] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Signalling peer c95f0816c41142699d80a2e326e17ab8 to start an election
I20260709 13:16:49.261376 16153 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bd2e0e0244294ec18481adf638d57ff0"
dest_uuid: "c95f0816c41142699d80a2e326e17ab8"
 from {username='slave'} at 127.0.0.1:43776
I20260709 13:16:49.261909 16153 raft_consensus.cc:493] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:49.262257 16153 raft_consensus.cc:3060] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.279397 16153 raft_consensus.cc:515] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:49.282145 16073 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2e0e0244294ec18481adf638d57ff0" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:49.282186 16006 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd2e0e0244294ec18481adf638d57ff0" candidate_uuid: "c95f0816c41142699d80a2e326e17ab8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:49.282845 16073 raft_consensus.cc:3055] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:49.282948 16006 raft_consensus.cc:3060] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.283234 16073 raft_consensus.cc:740] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:49.284020 16073 consensus_queue.cc:260] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:49.285167 16073 raft_consensus.cc:3060] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:49.288785 16007 raft_consensus.cc:1275] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Refusing update from remote peer c95f0816c41142699d80a2e326e17ab8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.290138 16006 raft_consensus.cc:2468] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 2.
I20260709 13:16:49.290256 17094 consensus_queue.cc:1048] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:49.292655 16121 leader_election.cc:304] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 903b3435239241eab01ea54ffb015ee3, c95f0816c41142699d80a2e326e17ab8; no voters: 
I20260709 13:16:49.294490 17094 raft_consensus.cc:2804] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:49.295099 17094 raft_consensus.cc:697] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [term 2 LEADER]: Becoming Leader. State: Replica: c95f0816c41142699d80a2e326e17ab8, State: Running, Role: LEADER
I20260709 13:16:49.295955 17094 consensus_queue.cc:237] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:49.300590 16073 raft_consensus.cc:2468] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c95f0816c41142699d80a2e326e17ab8 in term 2.
I20260709 13:16:49.301134 16153 leader_election.cc:290] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:49.314628 15829 catalog_manager.cc:5697] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 reported cstate change: term changed from 1 to 2, leader changed from d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67) to c95f0816c41142699d80a2e326e17ab8 (127.6.198.68). New cstate: current_term: 2 leader_uuid: "c95f0816c41142699d80a2e326e17ab8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:49.323477 16073 raft_consensus.cc:1275] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer c95f0816c41142699d80a2e326e17ab8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.335371 16153 raft_consensus.cc:1275] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.337183 17101 consensus_queue.cc:1048] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:49.337687 17094 consensus_queue.cc:1048] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:49.389189 16076 raft_consensus.cc:1275] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.403134 17140 consensus_queue.cc:1048] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:49.427122 16076 raft_consensus.cc:1275] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.450187 16007 raft_consensus.cc:1275] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.455578 17142 consensus_queue.cc:1048] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:49.454970 17072 consensus_queue.cc:1048] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:16:49.524668 16029 debug-util.cc:398] Leaking SignalData structure 0x7b0800243960 after lost signal to thread 6940
W20260709 13:16:49.525903 16029 debug-util.cc:398] Leaking SignalData structure 0x7b08001f8340 after lost signal to thread 15865
W20260709 13:16:49.527303 16029 debug-util.cc:398] Leaking SignalData structure 0x7b08001f9800 after lost signal to thread 16032
W20260709 13:16:49.528512 16029 debug-util.cc:398] Leaking SignalData structure 0x7b08001edd80 after lost signal to thread 16106
W20260709 13:16:49.529676 16029 debug-util.cc:398] Leaking SignalData structure 0x7b08001afe60 after lost signal to thread 16180
W20260709 13:16:49.530956 16029 debug-util.cc:398] Leaking SignalData structure 0x7b080017b940 after lost signal to thread 16906
I20260709 13:16:49.708212 16151 raft_consensus.cc:1275] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.709519 17101 consensus_queue.cc:1048] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:49.719434 16006 raft_consensus.cc:1275] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Refusing update from remote peer c95f0816c41142699d80a2e326e17ab8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.723515 17094 consensus_queue.cc:1048] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:49.748375 16076 raft_consensus.cc:1275] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer c95f0816c41142699d80a2e326e17ab8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.749967 17119 consensus_queue.cc:1048] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:49.774323 16073 raft_consensus.cc:1275] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:49.785729  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:49.785696 17101 consensus_queue.cc:1048] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:49.898376  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:49.951087 16073 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb4ef2e4f6be44efbf83fb70838b4ae7"
dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
mode: GRACEFUL
new_leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:41666
I20260709 13:16:49.951588 16073 raft_consensus.cc:606] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Received request to transfer leadership to TS bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:49.953860  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:16:49.955569  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:50.333175 17113 raft_consensus.cc:993] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab: : Instructing follower bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:50.333788 17113 raft_consensus.cc:1081] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Signalling peer bdeb23ab98c34a269f5b2e71b03c5e4e to start an election
I20260709 13:16:50.336350 16870 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb4ef2e4f6be44efbf83fb70838b4ae7"
dest_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e"
 from {username='slave'} at 127.0.0.1:49510
I20260709 13:16:50.340003 16870 raft_consensus.cc:493] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:50.340504 16870 raft_consensus.cc:3060] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:50.352624 16870 raft_consensus.cc:515] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:50.356027 16870 leader_election.cc:290] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Requested vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
I20260709 13:16:50.360033 16076 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb4ef2e4f6be44efbf83fb70838b4ae7" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d4d6e13a8f4a473098351ada96fd7dab"
I20260709 13:16:50.360566 16076 raft_consensus.cc:3055] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:50.360832 16076 raft_consensus.cc:740] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d4d6e13a8f4a473098351ada96fd7dab, State: Running, Role: LEADER
I20260709 13:16:50.361564 16076 consensus_queue.cc:260] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:50.362661 16076 raft_consensus.cc:3060] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:50.374291 16076 raft_consensus.cc:2468] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:50.375216 16755 leader_election.cc:304] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab; no voters: 
I20260709 13:16:50.376040 17140 raft_consensus.cc:2804] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:50.376493 17140 raft_consensus.cc:697] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Becoming Leader. State: Replica: bdeb23ab98c34a269f5b2e71b03c5e4e, State: Running, Role: LEADER
I20260709 13:16:50.377310 17140 consensus_queue.cc:237] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } }
I20260709 13:16:50.389081 16007 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb4ef2e4f6be44efbf83fb70838b4ae7" candidate_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "903b3435239241eab01ea54ffb015ee3"
I20260709 13:16:50.389585 16007 raft_consensus.cc:3060] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:50.399134 15829 catalog_manager.cc:5697] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e reported cstate change: term changed from 1 to 2, leader changed from d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67) to bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65). New cstate: current_term: 2 leader_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:50.406379 16007 raft_consensus.cc:2468] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdeb23ab98c34a269f5b2e71b03c5e4e in term 2.
I20260709 13:16:50.790529 16005 raft_consensus.cc:1275] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:50.792156 17140 consensus_queue.cc:1048] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:50.840462 16076 raft_consensus.cc:1275] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Refusing update from remote peer bdeb23ab98c34a269f5b2e71b03c5e4e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:50.845956 17142 consensus_queue.cc:1048] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:50.956553  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:51.177112 16119 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45841, user_credentials={real_user=slave}} blocked reactor thread for 71719.1us
W20260709 13:16:51.208488 16121 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:32879, user_credentials={real_user=slave}} blocked reactor thread for 100552us
W20260709 13:16:51.253095 16045 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:45841, user_credentials={real_user=slave}} blocked reactor thread for 52265.9us
W20260709 13:16:51.254383 16047 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.66:32879, user_credentials={real_user=slave}} blocked reactor thread for 49591us
W20260709 13:16:51.270753 16048 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.68:36069, user_credentials={real_user=slave}} blocked reactor thread for 45660.7us
I20260709 13:16:51.321692  6937 tablet_server.cc:179] TabletServer@127.6.198.65:45841 shutting down...
W20260709 13:16:51.330504 15971 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 255 similar messages]
W20260709 13:16:51.354518 15971 consensus_peers.cc:597] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.358089 16045 consensus_peers.cc:597] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.364655 15971 consensus_peers.cc:597] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.373617 16045 consensus_peers.cc:597] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.378332 15971 consensus_peers.cc:597] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.397199 16045 consensus_peers.cc:597] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.439378 15971 consensus_peers.cc:597] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.443837 16119 consensus_peers.cc:597] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.445773  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:51.448208  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.449483  6937 raft_consensus.cc:2243] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:51.451278  6937 raft_consensus.cc:2272] T b5e6891e9cab4a7298664c3c431ce36d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.455120  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.457370  6937 raft_consensus.cc:2243] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.458132  6937 raft_consensus.cc:2272] T 28cbf604c51f48da99cb7e3089651236 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.463680  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.464913  6937 raft_consensus.cc:2243] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.465636  6937 raft_consensus.cc:2272] T cc17a40e973a46a18d21e55a0a939417 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:51.466235 16045 consensus_peers.cc:597] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.469601  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.471083  6937 raft_consensus.cc:2243] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.471964  6937 raft_consensus.cc:2272] T f7473dd845e447a0b5a453e01dbb4b6b P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.477237  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.478216  6937 raft_consensus.cc:2243] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.478881  6937 raft_consensus.cc:2272] T 1a21a302bc17473392974aeea5a31d60 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.484340  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.485347  6937 raft_consensus.cc:2243] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:51.486907  6937 raft_consensus.cc:2272] T 9719c47150e74128b537e918110116e0 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:51.491943 16119 consensus_peers.cc:597] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.493127  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.494256  6937 raft_consensus.cc:2243] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.494915  6937 raft_consensus.cc:2272] T 62ac7645b9954621a685e4fc9ddde0dd P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.498616  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.499590  6937 raft_consensus.cc:2243] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.500348  6937 raft_consensus.cc:2272] T 4633df69636042288f27156e6084dce9 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:51.501116 16045 consensus_peers.cc:597] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.505888 15971 consensus_peers.cc:597] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.509238 16119 consensus_peers.cc:597] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.510006  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.511291  6937 raft_consensus.cc:2243] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.512092  6937 raft_consensus.cc:2272] T 5922376de62747aabde22ad134c86f36 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.515647  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.516628  6937 raft_consensus.cc:2243] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.517550  6937 raft_consensus.cc:2272] T 8a7a7d029fb040b881d5165c98065a12 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.520877  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.521891  6937 raft_consensus.cc:2243] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.522590  6937 raft_consensus.cc:2272] T f46c65175b29436c8f0704ffa98d4eb7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.527200  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.529012  6937 raft_consensus.cc:2243] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.529676  6937 raft_consensus.cc:2272] T 1708b4e2385e408abc600d9b2f6d42a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:51.535049 16119 consensus_peers.cc:597] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.537444  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.538511  6937 raft_consensus.cc:2243] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.539088  6937 raft_consensus.cc:2272] T 3132182819ce40fdb691736db3dd1061 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.542631  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.543824  6937 raft_consensus.cc:2243] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:51.545090  6937 raft_consensus.cc:2272] T f5565bfb243b4eeab5cbb5549d639378 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.550239  6937 tablet_replica.cc:333] stopping tablet replica
W20260709 13:16:51.545753 16045 consensus_peers.cc:597] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.551587  6937 raft_consensus.cc:2243] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.552335  6937 raft_consensus.cc:2272] T c6319ed0d71e4b4e9608ad673766fb5e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.557030  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.560511  6937 raft_consensus.cc:2243] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:51.561904  6937 raft_consensus.cc:2272] T c9df103185604f988d3abe94ae731141 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.565004  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.565829  6937 raft_consensus.cc:2243] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.566509  6937 raft_consensus.cc:2272] T 31217a26afbc45258ea518a1623dbb01 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.569993  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.570914  6937 raft_consensus.cc:2243] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.572592  6937 raft_consensus.cc:2272] T ce23588dc52d46338893551f44214a28 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:51.582304 15971 consensus_peers.cc:597] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.585201 16045 consensus_peers.cc:597] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.587245 15971 consensus_peers.cc:597] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.590502  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.591423  6937 raft_consensus.cc:2243] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:51.592659  6937 raft_consensus.cc:2272] T 3c53d28c984a4016b8461698b47d3300 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.597076  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.598531  6937 raft_consensus.cc:2243] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:51.599689  6937 raft_consensus.cc:2272] T e58eccbbe71e465188784170045cdb34 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.604569  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.605525  6937 raft_consensus.cc:2243] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.606148  6937 raft_consensus.cc:2272] T c6a3eeeeab11406e90e1fd10d63228e8 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.609532  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.610332  6937 raft_consensus.cc:2243] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:16:51.610651 16119 consensus_peers.cc:597] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.610963  6937 raft_consensus.cc:2272] T d6e83ffadf17413a95b361a2cf2b1b7f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.614898  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.615772  6937 raft_consensus.cc:2243] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:51.616864  6937 raft_consensus.cc:2272] T 034918f94907492ca9a964c053edd50e P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.621138  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.621935  6937 raft_consensus.cc:2243] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.622691  6937 raft_consensus.cc:2272] T 3a39139d4f2b48d09ba9eb320dfcf265 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.625717  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.626488  6937 raft_consensus.cc:2243] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:51.627660  6937 raft_consensus.cc:2272] T 7cca9877952a413a9112ea61481bc582 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.630584  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.631284  6937 raft_consensus.cc:2243] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.631868  6937 raft_consensus.cc:2272] T 8f15b348b66b4bffae9c7f9eed8207e2 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.635082  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.635864  6937 raft_consensus.cc:2243] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.637385  6937 raft_consensus.cc:2272] T ccc0c303e2dd466caf0e684c6ea83284 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.640558  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.641372  6937 raft_consensus.cc:2243] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:51.642555  6937 raft_consensus.cc:2272] T 2c2dba979b4b4cfdb4eb1159d36ae158 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.645547  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.646303  6937 raft_consensus.cc:2243] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.646878  6937 raft_consensus.cc:2272] T bf30209cb5544714bc183c72359c9656 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:51.647779 16045 consensus_peers.cc:597] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.649103 16045 consensus_peers.cc:597] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.650116  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.651347  6937 raft_consensus.cc:2243] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:51.652555  6937 raft_consensus.cc:2272] T 69d7d4e0bc194613bb410e793fd7bd39 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:51.654986 15971 consensus_peers.cc:597] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.655833  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.656752  6937 raft_consensus.cc:2243] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:51.657869  6937 raft_consensus.cc:2272] T 4712e5b3e765415b916c9bf89f5b0d74 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.661237  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.661964  6937 raft_consensus.cc:2243] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:51.662945  6937 raft_consensus.cc:2272] T 5bd99621a84f4fe994996601dae8988c P bdeb23ab98c34a269f5b2e71b03c5e4e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.666039  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.666754  6937 raft_consensus.cc:2243] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.667273  6937 raft_consensus.cc:2272] T 20762eaf5e494270a0e86a0b95c1437f P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.669802  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.670614  6937 raft_consensus.cc:2243] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:51.672701  6937 raft_consensus.cc:2272] T e38e5c73946b4d14825079217a21a21a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:51.674048 15971 consensus_peers.cc:597] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.675421  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.676100  6937 raft_consensus.cc:2243] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.676510  6937 raft_consensus.cc:2272] T d6cabb92769049d088d88c54ec482c85 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.678895  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.679415  6937 raft_consensus.cc:2243] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:51.680182  6937 raft_consensus.cc:2272] T cb4ef2e4f6be44efbf83fb70838b4ae7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.682981  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.683537  6937 raft_consensus.cc:2243] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.684073  6937 raft_consensus.cc:2272] T 6aa884c0889444ea8aa23dbe7bcfe9de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.686764  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.687465  6937 raft_consensus.cc:2243] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.688635  6937 raft_consensus.cc:2272] T 718e00ba6fa34542a71fc6d5758e92a1 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:51.689692 16119 consensus_peers.cc:597] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.691897  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.692836  6937 raft_consensus.cc:2243] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.693497  6937 raft_consensus.cc:2272] T 8b94b09aec6746f69853f14b417f18e3 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:51.694516 16119 consensus_peers.cc:597] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.696954  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.697788  6937 raft_consensus.cc:2243] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.698822  6937 raft_consensus.cc:2272] T 0a574a47fef045e183c154c25260da37 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.702621  6937 tablet_replica.cc:333] stopping tablet replica
W20260709 13:16:51.702688 16045 consensus_peers.cc:597] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.703485  6937 raft_consensus.cc:2243] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.704166  6937 raft_consensus.cc:2272] T c895cd2e2a144ff692fcb615c095e82a P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.707787  6937 tablet_replica.cc:333] stopping tablet replica
W20260709 13:16:51.708312 16045 consensus_peers.cc:597] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.708563  6937 raft_consensus.cc:2243] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.709173  6937 raft_consensus.cc:2272] T e08b5d04b39149c187ddb882b1b5d2de P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.712117  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.712870  6937 raft_consensus.cc:2243] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.713404  6937 raft_consensus.cc:2272] T 4332ee7a936341baad672bfc16703cc7 P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:51.716540  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.717229  6937 raft_consensus.cc:2243] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.717748  6937 raft_consensus.cc:2272] T f72ec4c0d31742baa2d0d9d0aad7caed P bdeb23ab98c34a269f5b2e71b03c5e4e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:51.718040 16119 consensus_peers.cc:597] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.721310  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:16:51.722105  6937 raft_consensus.cc:2243] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:51.722820  6937 raft_consensus.cc:2272] T 9ca16f452c2a4b19a53a12119268b92d P bdeb23ab98c34a269f5b2e71b03c5e4e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:51.728462 16045 consensus_peers.cc:597] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.766475 15971 consensus_peers.cc:597] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.844246  6937 tablet_server.cc:196] TabletServer@127.6.198.65:45841 shutdown complete.
W20260709 13:16:51.876245 16045 consensus_peers.cc:597] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:51.948983  6937 tablet_server.cc:179] TabletServer@127.6.198.66:0 shutting down...
W20260709 13:16:51.979315 16121 consensus_peers.cc:597] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.977429 16047 consensus_peers.cc:597] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:51.987308 16121 consensus_peers.cc:597] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.007871  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:52.010789  6937 tablet_replica.cc:333] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.012246  6937 raft_consensus.cc:2243] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.012895  6937 raft_consensus.cc:2272] T 9719c47150e74128b537e918110116e0 P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.016572  6937 tablet_replica.cc:333] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.017562  6937 raft_consensus.cc:2243] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.018764  6937 raft_consensus.cc:2272] T 4332ee7a936341baad672bfc16703cc7 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.021773  6937 tablet_replica.cc:333] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.022656  6937 raft_consensus.cc:2243] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.023670  6937 raft_consensus.cc:2272] T 0b83d43d98074cb09deac765b8b035e6 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.027395  6937 tablet_replica.cc:333] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.028198  6937 raft_consensus.cc:2243] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.028906  6937 raft_consensus.cc:2272] T 9ca16f452c2a4b19a53a12119268b92d P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.032003  6937 tablet_replica.cc:333] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.032856  6937 raft_consensus.cc:2243] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.033434  6937 raft_consensus.cc:2272] T 4712e5b3e765415b916c9bf89f5b0d74 P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:52.035073 16121 consensus_peers.cc:597] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.036375  6937 tablet_replica.cc:333] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.037259  6937 raft_consensus.cc:2243] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.037947  6937 raft_consensus.cc:2272] T 69d7d4e0bc194613bb410e793fd7bd39 P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:52.038009 16047 consensus_peers.cc:597] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.041046  6937 tablet_replica.cc:333] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.041939  6937 raft_consensus.cc:2243] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.042619  6937 raft_consensus.cc:2272] T 5bd99621a84f4fe994996601dae8988c P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.045513  6937 tablet_replica.cc:333] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.046375  6937 raft_consensus.cc:2243] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.046990  6937 raft_consensus.cc:2272] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.049223  6937 tablet_replica.cc:333] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.049927  6937 raft_consensus.cc:2243] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.050426  6937 raft_consensus.cc:2272] T 28cbf604c51f48da99cb7e3089651236 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:52.052652 16121 consensus_peers.cc:597] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.053081  6937 tablet_replica.cc:333] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.053705  6937 raft_consensus.cc:2243] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.054121  6937 raft_consensus.cc:2272] T de7a15b080c54551b2c51bee7af64600 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.056107  6937 tablet_replica.cc:333] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.056638  6937 raft_consensus.cc:2243] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.057165  6937 raft_consensus.cc:2272] T 3c53d28c984a4016b8461698b47d3300 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.059863  6937 tablet_replica.cc:333] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.060648  6937 raft_consensus.cc:2243] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.061233  6937 raft_consensus.cc:2272] T b5e6891e9cab4a7298664c3c431ce36d P 903b3435239241eab01ea54ffb015ee3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.063891  6937 tablet_replica.cc:333] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.064668  6937 raft_consensus.cc:2243] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.065220  6937 raft_consensus.cc:2272] T dadf33b9db824fdcbb63864c9e975c6b P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.070924  6937 tablet_replica.cc:333] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.071988  6937 raft_consensus.cc:2243] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.072947  6937 raft_consensus.cc:2272] T 1a21a302bc17473392974aeea5a31d60 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.076048  6937 tablet_replica.cc:333] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
W20260709 13:16:52.076740 16047 consensus_peers.cc:597] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.076823  6937 raft_consensus.cc:2243] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.089773  6937 raft_consensus.cc:2272] T 62ac7645b9954621a685e4fc9ddde0dd P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.097618  6937 tablet_replica.cc:333] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.098541  6937 raft_consensus.cc:2243] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.118232  6937 raft_consensus.cc:2272] T 7454f1cf21c0400c8a3a8290d5d148dd P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.121143  6937 tablet_replica.cc:333] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.121987  6937 raft_consensus.cc:2243] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.123040  6937 raft_consensus.cc:2272] T 8f15b348b66b4bffae9c7f9eed8207e2 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.125841  6937 tablet_replica.cc:333] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.126663  6937 raft_consensus.cc:2243] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.127213  6937 raft_consensus.cc:2272] T aaa175423efe491da6747a37df086375 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.130007  6937 tablet_replica.cc:333] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.130735  6937 raft_consensus.cc:2243] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.131723  6937 raft_consensus.cc:2272] T ad97b67157f44c618ddfebbc1c25588b P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.133878  6937 tablet_replica.cc:333] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.134451  6937 raft_consensus.cc:2243] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.134899  6937 raft_consensus.cc:2272] T 5922376de62747aabde22ad134c86f36 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.136713  6937 tablet_replica.cc:333] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.137300  6937 raft_consensus.cc:2243] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.137708  6937 raft_consensus.cc:2272] T d6cabb92769049d088d88c54ec482c85 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.139926  6937 tablet_replica.cc:333] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.140574  6937 raft_consensus.cc:2243] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.141101  6937 raft_consensus.cc:2272] T e38e5c73946b4d14825079217a21a21a P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.143517  6937 tablet_replica.cc:333] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.144038  6937 raft_consensus.cc:2243] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.144470  6937 raft_consensus.cc:2272] T 20762eaf5e494270a0e86a0b95c1437f P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.146577  6937 tablet_replica.cc:333] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.147204  6937 raft_consensus.cc:2243] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.147907  6937 raft_consensus.cc:2272] T 0a574a47fef045e183c154c25260da37 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.150112  6937 tablet_replica.cc:333] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.150880  6937 raft_consensus.cc:2243] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.151379  6937 raft_consensus.cc:2272] T eebbb30a54a04a4aa567c6e16581ad1a P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.153589  6937 tablet_replica.cc:333] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.154098  6937 raft_consensus.cc:2243] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.154546  6937 raft_consensus.cc:2272] T f46c65175b29436c8f0704ffa98d4eb7 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.156471  6937 tablet_replica.cc:333] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.156955  6937 raft_consensus.cc:2243] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.157459  6937 raft_consensus.cc:2272] T f5565bfb243b4eeab5cbb5549d639378 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.159322  6937 tablet_replica.cc:333] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.159816  6937 raft_consensus.cc:2243] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.160548  6937 raft_consensus.cc:2272] T c6319ed0d71e4b4e9608ad673766fb5e P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.162552  6937 tablet_replica.cc:333] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.163123  6937 raft_consensus.cc:2243] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.163528  6937 raft_consensus.cc:2272] T 3424ea4fd86e4453a5d9fde62fc43e7f P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.165512  6937 tablet_replica.cc:333] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.166013  6937 raft_consensus.cc:2243] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.166460  6937 raft_consensus.cc:2272] T bd2e0e0244294ec18481adf638d57ff0 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.168315  6937 tablet_replica.cc:333] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.168833  6937 raft_consensus.cc:2243] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.169252  6937 raft_consensus.cc:2272] T 4633df69636042288f27156e6084dce9 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.171738  6937 tablet_replica.cc:333] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.172333  6937 raft_consensus.cc:2243] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.172829  6937 raft_consensus.cc:2272] T 07ae89d92f6942d196dcc8d146b8f459 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.175334  6937 tablet_replica.cc:333] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.176105  6937 raft_consensus.cc:2243] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.176656  6937 raft_consensus.cc:2272] T cb4ef2e4f6be44efbf83fb70838b4ae7 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:52.176566 16047 consensus_peers.cc:597] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.179219  6937 tablet_replica.cc:333] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.179855  6937 raft_consensus.cc:2243] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.180433  6937 raft_consensus.cc:2272] T e08b5d04b39149c187ddb882b1b5d2de P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.183255  6937 tablet_replica.cc:333] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.183956  6937 raft_consensus.cc:2243] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.184530  6937 raft_consensus.cc:2272] T c895cd2e2a144ff692fcb615c095e82a P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.187276  6937 tablet_replica.cc:333] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.187984  6937 raft_consensus.cc:2243] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.188548  6937 raft_consensus.cc:2272] T 68a3e44fcd5b438486e1ec0241ef1e8d P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.191419  6937 tablet_replica.cc:333] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.192797  6937 raft_consensus.cc:2243] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.193853  6937 raft_consensus.cc:2272] T 31217a26afbc45258ea518a1623dbb01 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.196377  6937 tablet_replica.cc:333] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.197149  6937 raft_consensus.cc:2243] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.197768  6937 raft_consensus.cc:2272] T c9df103185604f988d3abe94ae731141 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.200685  6937 tablet_replica.cc:333] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.201387  6937 raft_consensus.cc:2243] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.202466  6937 raft_consensus.cc:2272] T 718e00ba6fa34542a71fc6d5758e92a1 P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.205156  6937 tablet_replica.cc:333] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.205837  6937 raft_consensus.cc:2243] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:16:52.206210 16047 consensus_peers.cc:597] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.206858  6937 raft_consensus.cc:2272] T 6aa884c0889444ea8aa23dbe7bcfe9de P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.209766  6937 tablet_replica.cc:333] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.210481  6937 raft_consensus.cc:2243] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.211612  6937 raft_consensus.cc:2272] T ef6f890d60f14faead1e5986904c458a P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:52.212508 16121 consensus_peers.cc:597] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.214267  6937 tablet_replica.cc:333] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.214984  6937 raft_consensus.cc:2243] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.215996  6937 raft_consensus.cc:2272] T f7473dd845e447a0b5a453e01dbb4b6b P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.218703  6937 tablet_replica.cc:333] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.219378  6937 raft_consensus.cc:2243] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.220314  6937 raft_consensus.cc:2272] T ce23588dc52d46338893551f44214a28 P 903b3435239241eab01ea54ffb015ee3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.223147  6937 tablet_replica.cc:333] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3: stopping tablet replica
I20260709 13:16:52.223652  6937 raft_consensus.cc:2243] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.224658  6937 raft_consensus.cc:2272] T ab6469b36c7444ba906df4c1f29f6f0e P 903b3435239241eab01ea54ffb015ee3 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:52.251186 16047 consensus_peers.cc:597] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:52.261808 16047 consensus_peers.cc:597] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:52.262892 16121 consensus_peers.cc:597] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:52.266700 16121 consensus_peers.cc:597] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:52.287784 16121 consensus_peers.cc:597] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:52.305518 16121 consensus_peers.cc:597] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.353497  6937 tablet_server.cc:196] TabletServer@127.6.198.66:0 shutdown complete.
W20260709 13:16:52.402722 16121 consensus_peers.cc:597] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.448312  6937 tablet_server.cc:179] TabletServer@127.6.198.67:0 shutting down...
W20260709 13:16:52.474298 16121 consensus_peers.cc:597] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:52.475047 16121 consensus_peers.cc:597] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:52.484577 16120 consensus_peers.cc:597] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 -> Peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Couldn't send request to peer d4d6e13a8f4a473098351ada96fd7dab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.504249  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:52.506119  6937 tablet_replica.cc:333] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.507145  6937 raft_consensus.cc:2243] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.508356  6937 raft_consensus.cc:2272] T 8a7a7d029fb040b881d5165c98065a12 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.510962  6937 tablet_replica.cc:333] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.511580  6937 raft_consensus.cc:2243] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.512033  6937 raft_consensus.cc:2272] T 9719c47150e74128b537e918110116e0 P d4d6e13a8f4a473098351ada96fd7dab [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.514199  6937 tablet_replica.cc:333] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.514858  6937 raft_consensus.cc:2243] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.515317  6937 raft_consensus.cc:2272] T 0b83d43d98074cb09deac765b8b035e6 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.517760  6937 tablet_replica.cc:333] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.518280  6937 raft_consensus.cc:2243] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.519201  6937 raft_consensus.cc:2272] T 9ca16f452c2a4b19a53a12119268b92d P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.521304  6937 tablet_replica.cc:333] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.522015  6937 raft_consensus.cc:2243] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.522609  6937 raft_consensus.cc:2272] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.524561  6937 tablet_replica.cc:333] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.525096  6937 raft_consensus.cc:2243] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.525519  6937 raft_consensus.cc:2272] T e58eccbbe71e465188784170045cdb34 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.527699  6937 tablet_replica.cc:333] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.528282  6937 raft_consensus.cc:2243] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.528709  6937 raft_consensus.cc:2272] T 3c53d28c984a4016b8461698b47d3300 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.530750  6937 tablet_replica.cc:333] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.531312  6937 raft_consensus.cc:2243] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.532037  6937 raft_consensus.cc:2272] T f72ec4c0d31742baa2d0d9d0aad7caed P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.533982  6937 tablet_replica.cc:333] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.534549  6937 raft_consensus.cc:2243] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.534962  6937 raft_consensus.cc:2272] T dadf33b9db824fdcbb63864c9e975c6b P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.536847  6937 tablet_replica.cc:333] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.537390  6937 raft_consensus.cc:2243] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.538189  6937 raft_consensus.cc:2272] T de7a15b080c54551b2c51bee7af64600 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.540191  6937 tablet_replica.cc:333] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.540722  6937 raft_consensus.cc:2243] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.541440  6937 raft_consensus.cc:2272] T 8b94b09aec6746f69853f14b417f18e3 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.543743  6937 tablet_replica.cc:333] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.544354  6937 raft_consensus.cc:2243] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.545183  6937 raft_consensus.cc:2272] T cc17a40e973a46a18d21e55a0a939417 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:52.546715 16120 consensus_peers.cc:597] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 -> Peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Couldn't send request to peer d4d6e13a8f4a473098351ada96fd7dab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.547775  6937 tablet_replica.cc:333] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.548548  6937 raft_consensus.cc:2243] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.549008  6937 raft_consensus.cc:2272] T ad97b67157f44c618ddfebbc1c25588b P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.551587  6937 tablet_replica.cc:333] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.552232  6937 raft_consensus.cc:2243] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.552932  6937 raft_consensus.cc:2272] T 5922376de62747aabde22ad134c86f36 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.555295  6937 tablet_replica.cc:333] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.555843  6937 raft_consensus.cc:2243] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.556279  6937 raft_consensus.cc:2272] T c6319ed0d71e4b4e9608ad673766fb5e P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.558211  6937 tablet_replica.cc:333] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.558784  6937 raft_consensus.cc:2243] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.559233  6937 raft_consensus.cc:2272] T f5565bfb243b4eeab5cbb5549d639378 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.561221  6937 tablet_replica.cc:333] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.561900  6937 raft_consensus.cc:2243] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.562311  6937 raft_consensus.cc:2272] T aaa175423efe491da6747a37df086375 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.564667  6937 tablet_replica.cc:333] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.565397  6937 raft_consensus.cc:2243] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.566481  6937 raft_consensus.cc:2272] T 3424ea4fd86e4453a5d9fde62fc43e7f P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.570719  6937 tablet_replica.cc:333] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.571595  6937 raft_consensus.cc:2243] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.572172  6937 raft_consensus.cc:2272] T bd2e0e0244294ec18481adf638d57ff0 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.574855  6937 tablet_replica.cc:333] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.575616  6937 raft_consensus.cc:2243] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.576704  6937 raft_consensus.cc:2272] T d6cabb92769049d088d88c54ec482c85 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.579731  6937 tablet_replica.cc:333] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.580591  6937 raft_consensus.cc:2243] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.581595  6937 raft_consensus.cc:2272] T 20762eaf5e494270a0e86a0b95c1437f P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.584149  6937 tablet_replica.cc:333] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.585052  6937 raft_consensus.cc:2243] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.585640  6937 raft_consensus.cc:2272] T e38e5c73946b4d14825079217a21a21a P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.588181  6937 tablet_replica.cc:333] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.588913  6937 raft_consensus.cc:2243] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.589422  6937 raft_consensus.cc:2272] T 5bd99621a84f4fe994996601dae8988c P d4d6e13a8f4a473098351ada96fd7dab [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.591953  6937 tablet_replica.cc:333] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.592712  6937 raft_consensus.cc:2243] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.593230  6937 raft_consensus.cc:2272] T bf30209cb5544714bc183c72359c9656 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.595659  6937 tablet_replica.cc:333] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.596493  6937 raft_consensus.cc:2243] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.597494  6937 raft_consensus.cc:2272] T 3a39139d4f2b48d09ba9eb320dfcf265 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.600162  6937 tablet_replica.cc:333] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.600914  6937 raft_consensus.cc:2243] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.601472  6937 raft_consensus.cc:2272] T 034918f94907492ca9a964c053edd50e P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.604178  6937 tablet_replica.cc:333] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.604967  6937 raft_consensus.cc:2243] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.605517  6937 raft_consensus.cc:2272] T 68a3e44fcd5b438486e1ec0241ef1e8d P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.608075  6937 tablet_replica.cc:333] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.608865  6937 raft_consensus.cc:2243] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.609439  6937 raft_consensus.cc:2272] T 7cca9877952a413a9112ea61481bc582 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.612071  6937 tablet_replica.cc:333] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.612846  6937 raft_consensus.cc:2243] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.613353  6937 raft_consensus.cc:2272] T 7454f1cf21c0400c8a3a8290d5d148dd P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:52.613540 16120 consensus_peers.cc:597] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 -> Peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Couldn't send request to peer d4d6e13a8f4a473098351ada96fd7dab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.615985  6937 tablet_replica.cc:333] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.616798  6937 raft_consensus.cc:2243] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.617765  6937 raft_consensus.cc:2272] T c6a3eeeeab11406e90e1fd10d63228e8 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.620533  6937 tablet_replica.cc:333] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.621313  6937 raft_consensus.cc:2243] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.621827  6937 raft_consensus.cc:2272] T d6e83ffadf17413a95b361a2cf2b1b7f P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:52.622813 16120 consensus_peers.cc:597] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 -> Peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Couldn't send request to peer d4d6e13a8f4a473098351ada96fd7dab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.624763  6937 tablet_replica.cc:333] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.625602  6937 raft_consensus.cc:2243] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.626127  6937 raft_consensus.cc:2272] T 07ae89d92f6942d196dcc8d146b8f459 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.628796  6937 tablet_replica.cc:333] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.629518  6937 raft_consensus.cc:2243] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.630089  6937 raft_consensus.cc:2272] T cb4ef2e4f6be44efbf83fb70838b4ae7 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.632764  6937 tablet_replica.cc:333] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.633491  6937 raft_consensus.cc:2243] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.633994  6937 raft_consensus.cc:2272] T ab6469b36c7444ba906df4c1f29f6f0e P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.636724  6937 tablet_replica.cc:333] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.637424  6937 raft_consensus.cc:2243] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.638422  6937 raft_consensus.cc:2272] T 3132182819ce40fdb691736db3dd1061 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.640971  6937 tablet_replica.cc:333] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.641680  6937 raft_consensus.cc:2243] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.642179  6937 raft_consensus.cc:2272] T 1708b4e2385e408abc600d9b2f6d42a1 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.644714  6937 tablet_replica.cc:333] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.645401  6937 raft_consensus.cc:2243] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.645895  6937 raft_consensus.cc:2272] T ce23588dc52d46338893551f44214a28 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.648721  6937 tablet_replica.cc:333] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.649494  6937 raft_consensus.cc:2243] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.650571  6937 raft_consensus.cc:2272] T ccc0c303e2dd466caf0e684c6ea83284 P d4d6e13a8f4a473098351ada96fd7dab [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.653339  6937 tablet_replica.cc:333] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.654073  6937 raft_consensus.cc:2243] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.654626  6937 raft_consensus.cc:2272] T 2c2dba979b4b4cfdb4eb1159d36ae158 P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.657181  6937 tablet_replica.cc:333] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.657904  6937 raft_consensus.cc:2243] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.658489  6937 raft_consensus.cc:2272] T 6aa884c0889444ea8aa23dbe7bcfe9de P d4d6e13a8f4a473098351ada96fd7dab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.661134  6937 tablet_replica.cc:333] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.661852  6937 raft_consensus.cc:2243] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.662364  6937 raft_consensus.cc:2272] T ef6f890d60f14faead1e5986904c458a P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.664858  6937 tablet_replica.cc:333] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.665553  6937 raft_consensus.cc:2243] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:52.666570  6937 raft_consensus.cc:2272] T e08b5d04b39149c187ddb882b1b5d2de P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.669137  6937 tablet_replica.cc:333] T 0a574a47fef045e183c154c25260da37 P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.669763  6937 raft_consensus.cc:2243] T 0a574a47fef045e183c154c25260da37 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.670297  6937 raft_consensus.cc:2272] T 0a574a47fef045e183c154c25260da37 P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:52.672896  6937 tablet_replica.cc:333] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab: stopping tablet replica
I20260709 13:16:52.673558  6937 raft_consensus.cc:2243] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:52.674026  6937 raft_consensus.cc:2272] T eebbb30a54a04a4aa567c6e16581ad1a P d4d6e13a8f4a473098351ada96fd7dab [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:52.676905 16120 consensus_peers.cc:597] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 -> Peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Couldn't send request to peer d4d6e13a8f4a473098351ada96fd7dab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:52.693142 16120 consensus_peers.cc:597] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 -> Peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Couldn't send request to peer d4d6e13a8f4a473098351ada96fd7dab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:52.713584 16120 consensus_peers.cc:597] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 -> Peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Couldn't send request to peer d4d6e13a8f4a473098351ada96fd7dab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:52.746510 16120 consensus_peers.cc:597] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 -> Peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Couldn't send request to peer d4d6e13a8f4a473098351ada96fd7dab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.758538 17157 raft_consensus.cc:493] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:52.759203 17157 raft_consensus.cc:515] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:52.761785 17157 leader_election.cc:290] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
W20260709 13:16:52.765537 16119 leader_election.cc:336] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:52.766136 16121 leader_election.cc:336] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
I20260709 13:16:52.766577 16121 leader_election.cc:304] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:52.767282 17157 raft_consensus.cc:2749] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:52.771472 16120 consensus_peers.cc:597] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 -> Peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Couldn't send request to peer d4d6e13a8f4a473098351ada96fd7dab. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.816179 17170 raft_consensus.cc:493] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:52.816746 17170 raft_consensus.cc:515] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:52.818666 17170 leader_election.cc:290] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:52.823146 16119 leader_election.cc:336] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:52.824290 16120 leader_election.cc:336] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
I20260709 13:16:52.824630 16120 leader_election.cc:304] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:52.825186 17157 raft_consensus.cc:2749] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:52.825923 17157 raft_consensus.cc:493] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:52.826313 17157 raft_consensus.cc:515] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:52.828048 17157 leader_election.cc:290] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:52.832114 16119 leader_election.cc:336] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:52.832628 16120 leader_election.cc:336] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
I20260709 13:16:52.832958 16120 leader_election.cc:304] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:52.833482 17157 raft_consensus.cc:2749] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:52.872655 16120 consensus_peers.cc:597] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 -> Peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Couldn't send request to peer d4d6e13a8f4a473098351ada96fd7dab. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:52.924576 17187 raft_consensus.cc:493] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:52.925091 17187 raft_consensus.cc:515] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:52.927207 17187 leader_election.cc:290] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
W20260709 13:16:52.932742 16121 leader_election.cc:336] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
W20260709 13:16:52.933151 16119 leader_election.cc:336] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:52.933632 16119 leader_election.cc:304] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:52.934408 17187 raft_consensus.cc:2749] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:52.947618 17187 raft_consensus.cc:493] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:52.948091 17187 raft_consensus.cc:515] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:52.950037 17187 leader_election.cc:290] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:52.952801 16119 leader_election.cc:336] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:52.954070 16120 leader_election.cc:336] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
I20260709 13:16:52.954416 16120 leader_election.cc:304] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:52.954939 17187 raft_consensus.cc:2749] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:52.962651 17187 raft_consensus.cc:493] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:52.963049 17187 raft_consensus.cc:515] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:52.964736 17187 leader_election.cc:290] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
W20260709 13:16:52.969069 16121 leader_election.cc:336] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
W20260709 13:16:52.970253 16119 leader_election.cc:336] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:52.970706 16119 leader_election.cc:304] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:52.971390 17170 raft_consensus.cc:2749] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:52.981968 17187 raft_consensus.cc:493] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:52.982488 17187 raft_consensus.cc:515] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:52.984556 17187 leader_election.cc:290] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:52.988159 16120 leader_election.cc:336] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
W20260709 13:16:52.988644 16119 leader_election.cc:336] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:52.989064 16119 leader_election.cc:304] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:52.989719 17187 raft_consensus.cc:2749] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:53.023244 17187 raft_consensus.cc:493] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 903b3435239241eab01ea54ffb015ee3)
I20260709 13:16:53.023655 17187 raft_consensus.cc:515] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:53.025468 17187 leader_election.cc:290] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
W20260709 13:16:53.029263 16121 leader_election.cc:336] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
W20260709 13:16:53.030673 16119 leader_election.cc:336] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:53.031083 16119 leader_election.cc:304] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:53.031715 17187 raft_consensus.cc:2749] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.039491 17187 raft_consensus.cc:493] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:53.039858 17187 raft_consensus.cc:515] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:53.041389 17187 leader_election.cc:290] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
W20260709 13:16:53.045169 16119 leader_election.cc:336] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:53.045662 16121 leader_election.cc:336] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
I20260709 13:16:53.046088 16121 leader_election.cc:304] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:53.046780 17187 raft_consensus.cc:2749] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:53.097962 17187 raft_consensus.cc:493] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 903b3435239241eab01ea54ffb015ee3)
I20260709 13:16:53.098518 17187 raft_consensus.cc:515] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:53.100586 17187 leader_election.cc:290] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
W20260709 13:16:53.103881 16121 leader_election.cc:336] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
W20260709 13:16:53.104310 16120 leader_election.cc:336] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
I20260709 13:16:53.104642 16120 leader_election.cc:304] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:53.105175 17187 raft_consensus.cc:2749] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.261759 17157 raft_consensus.cc:493] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 903b3435239241eab01ea54ffb015ee3)
I20260709 13:16:53.262296 17157 raft_consensus.cc:515] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:53.267272 17157 leader_election.cc:290] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
W20260709 13:16:53.271355 16121 leader_election.cc:336] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
W20260709 13:16:53.272257 16119 leader_election.cc:336] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:53.272771 16119 leader_election.cc:304] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:53.273617 17188 raft_consensus.cc:2749] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:53.322695 17188 raft_consensus.cc:493] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 903b3435239241eab01ea54ffb015ee3)
I20260709 13:16:53.323274 17188 raft_consensus.cc:515] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:53.325580 17188 leader_election.cc:290] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
W20260709 13:16:53.329229 16121 leader_election.cc:336] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
W20260709 13:16:53.329758 16120 leader_election.cc:336] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
I20260709 13:16:53.330089 16120 leader_election.cc:304] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:53.330647 17188 raft_consensus.cc:2749] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.351737 17188 raft_consensus.cc:493] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 903b3435239241eab01ea54ffb015ee3)
I20260709 13:16:53.352283 17188 raft_consensus.cc:515] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:53.354489 17188 leader_election.cc:290] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
W20260709 13:16:53.357880 16119 leader_election.cc:336] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:53.359002 16121 leader_election.cc:336] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
I20260709 13:16:53.359416 16121 leader_election.cc:304] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:53.360339 17170 raft_consensus.cc:2749] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.378798 17170 raft_consensus.cc:493] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 903b3435239241eab01ea54ffb015ee3)
I20260709 13:16:53.379276 17170 raft_consensus.cc:515] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:53.381346 17170 leader_election.cc:290] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
W20260709 13:16:53.387063 16120 leader_election.cc:336] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
W20260709 13:16:53.387689 16121 leader_election.cc:336] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
I20260709 13:16:53.388068 16121 leader_election.cc:304] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:53.388715 17170 raft_consensus.cc:2749] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.442044 17188 raft_consensus.cc:493] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 903b3435239241eab01ea54ffb015ee3)
I20260709 13:16:53.442493 17188 raft_consensus.cc:515] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:53.444023 17188 leader_election.cc:290] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
W20260709 13:16:53.447749 16121 leader_election.cc:336] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
W20260709 13:16:53.448380 16120 leader_election.cc:336] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
I20260709 13:16:53.448719 16120 leader_election.cc:304] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:53.449240 17188 raft_consensus.cc:2749] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.456019 17188 raft_consensus.cc:493] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4d6e13a8f4a473098351ada96fd7dab)
I20260709 13:16:53.456576 17188 raft_consensus.cc:515] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:53.458730 17188 leader_election.cc:290] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:53.462069 16120 leader_election.cc:336] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
W20260709 13:16:53.462584 16119 leader_election.cc:336] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:53.462901 16119 leader_election.cc:304] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:53.463418 17188 raft_consensus.cc:2749] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.469907 17188 raft_consensus.cc:493] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 903b3435239241eab01ea54ffb015ee3)
I20260709 13:16:53.470407 17188 raft_consensus.cc:515] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:53.472549 17188 leader_election.cc:290] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
W20260709 13:16:53.475945 16119 leader_election.cc:336] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:53.476567 16121 leader_election.cc:336] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
I20260709 13:16:53.476877 16121 leader_election.cc:304] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:53.477387 17188 raft_consensus.cc:2749] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.478984 17188 raft_consensus.cc:493] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 903b3435239241eab01ea54ffb015ee3)
I20260709 13:16:53.479387 17188 raft_consensus.cc:515] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:53.481026 17188 leader_election.cc:290] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
W20260709 13:16:53.484515 16121 leader_election.cc:336] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
W20260709 13:16:53.486536 16120 leader_election.cc:336] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
I20260709 13:16:53.486949 16120 leader_election.cc:304] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:53.487485 17188 raft_consensus.cc:2749] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.517514 17188 raft_consensus.cc:493] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 903b3435239241eab01ea54ffb015ee3)
I20260709 13:16:53.517931 17188 raft_consensus.cc:515] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:53.519492 17188 leader_election.cc:290] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
W20260709 13:16:53.523579 16119 leader_election.cc:336] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:53.524080 16121 leader_election.cc:336] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
I20260709 13:16:53.524509 16121 leader_election.cc:304] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:53.525163 17188 raft_consensus.cc:2749] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.533943 17188 raft_consensus.cc:493] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 903b3435239241eab01ea54ffb015ee3)
I20260709 13:16:53.534369 17188 raft_consensus.cc:515] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:53.536026 17188 leader_election.cc:290] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
W20260709 13:16:53.539525 16119 leader_election.cc:336] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:53.539987 16121 leader_election.cc:336] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
I20260709 13:16:53.540416 16121 leader_election.cc:304] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:53.541054 17188 raft_consensus.cc:2749] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.567791 17188 raft_consensus.cc:493] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4d6e13a8f4a473098351ada96fd7dab)
I20260709 13:16:53.568200 17188 raft_consensus.cc:515] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:53.569896 17188 leader_election.cc:290] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:53.573714 16120 leader_election.cc:336] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
W20260709 13:16:53.574236 16119 leader_election.cc:336] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:53.574626 16119 leader_election.cc:304] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:53.575152 17188 raft_consensus.cc:2749] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:16:53.689023 17188 raft_consensus.cc:493] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4d6e13a8f4a473098351ada96fd7dab)
I20260709 13:16:53.689479 17188 raft_consensus.cc:515] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:53.691226 17188 leader_election.cc:290] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
W20260709 13:16:53.694836 16120 leader_election.cc:336] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
W20260709 13:16:53.695350 16121 leader_election.cc:336] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
I20260709 13:16:53.695683 16121 leader_election.cc:304] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:53.696318 17188 raft_consensus.cc:2749] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:53.763954  6937 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 16103)
W20260709 13:16:53.779320 16119 consensus_peers.cc:597] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:53.828127 17188 raft_consensus.cc:493] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d4d6e13a8f4a473098351ada96fd7dab)
I20260709 13:16:53.828799 17188 raft_consensus.cc:515] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:53.831074 17188 leader_election.cc:290] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:53.835592 16119 leader_election.cc:336] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:53.843257 16120 leader_election.cc:336] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
I20260709 13:16:53.843670 16120 leader_election.cc:304] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:53.844336 17188 raft_consensus.cc:2749] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:53.846565 17188 raft_consensus.cc:493] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d4d6e13a8f4a473098351ada96fd7dab)
I20260709 13:16:53.846964 17188 raft_consensus.cc:515] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:53.848439 17188 leader_election.cc:290] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:53.852084 16120 leader_election.cc:336] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
W20260709 13:16:53.852541 16119 leader_election.cc:336] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:53.852867 16119 leader_election.cc:304] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:53.853397 17188 raft_consensus.cc:2749] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:53.863006 16119 consensus_peers.cc:597] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:53.930773 17170 raft_consensus.cc:493] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d4d6e13a8f4a473098351ada96fd7dab)
I20260709 13:16:53.931329 17170 raft_consensus.cc:515] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:53.933472 17170 leader_election.cc:290] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:53.938118 16119 leader_election.cc:336] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:53.938727 16120 leader_election.cc:336] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
I20260709 13:16:53.939152 16120 leader_election.cc:304] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:53.939839 17170 raft_consensus.cc:2749] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:16:53.962446 16119 consensus_peers.cc:597] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:53.984679 16119 consensus_peers.cc:597] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:53.988860 17170 raft_consensus.cc:493] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4d6e13a8f4a473098351ada96fd7dab)
I20260709 13:16:53.989274 17170 raft_consensus.cc:515] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } } peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } }
I20260709 13:16:53.990866 17170 leader_election.cc:290] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879), d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795)
W20260709 13:16:53.994613 16120 leader_election.cc:336] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
W20260709 13:16:53.995177 16121 leader_election.cc:336] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
I20260709 13:16:53.995599 16121 leader_election.cc:304] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:53.996290 17170 raft_consensus.cc:2749] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:54.022691 16119 consensus_peers.cc:597] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:54.066571 17170 raft_consensus.cc:493] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d4d6e13a8f4a473098351ada96fd7dab)
I20260709 13:16:54.067158 17170 raft_consensus.cc:515] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:54.069309 17170 leader_election.cc:290] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:54.072966 16119 leader_election.cc:336] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:54.073604 16120 leader_election.cc:336] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
I20260709 13:16:54.073921 16120 leader_election.cc:304] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:54.074502 17170 raft_consensus.cc:2749] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:54.079747 17170 raft_consensus.cc:493] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d4d6e13a8f4a473098351ada96fd7dab)
I20260709 13:16:54.080240 17170 raft_consensus.cc:515] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:54.082370 17170 leader_election.cc:290] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:54.085601 16120 leader_election.cc:336] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
W20260709 13:16:54.086179 16119 leader_election.cc:336] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:54.086550 16119 leader_election.cc:304] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:54.087073 17170 raft_consensus.cc:2749] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:16:54.120623 17170 raft_consensus.cc:493] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d4d6e13a8f4a473098351ada96fd7dab)
I20260709 13:16:54.121167 17170 raft_consensus.cc:515] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4d6e13a8f4a473098351ada96fd7dab" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 32795 } } peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } }
I20260709 13:16:54.123354 17170 leader_election.cc:290] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795), bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841)
W20260709 13:16:54.126657 16120 leader_election.cc:336] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d4d6e13a8f4a473098351ada96fd7dab (127.6.198.67:32795): Network error: Client connection negotiation failed: client connection to 127.6.198.67:32795: connect: Connection refused (error 111)
W20260709 13:16:54.127281 16119 leader_election.cc:336] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
I20260709 13:16:54.127677 16119 leader_election.cc:304] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: bdeb23ab98c34a269f5b2e71b03c5e4e, d4d6e13a8f4a473098351ada96fd7dab
I20260709 13:16:54.128345 17170 raft_consensus.cc:2749] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:16:54.191030 16119 consensus_peers.cc:597] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:54.219846 16121 consensus_peers.cc:597] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:54.309710  6937 tablet_server.cc:196] TabletServer@127.6.198.67:0 shutdown complete.
I20260709 13:16:54.379880  6937 tablet_server.cc:179] TabletServer@127.6.198.68:0 shutting down...
W20260709 13:16:54.382434 16119 consensus_peers.cc:597] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:54.393551 17170 raft_consensus.cc:493] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader bdeb23ab98c34a269f5b2e71b03c5e4e)
I20260709 13:16:54.394078 17170 raft_consensus.cc:515] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95f0816c41142699d80a2e326e17ab8" member_type: VOTER last_known_addr { host: "127.6.198.68" port: 36069 } } peers { permanent_uuid: "bdeb23ab98c34a269f5b2e71b03c5e4e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45841 } } peers { permanent_uuid: "903b3435239241eab01ea54ffb015ee3" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 32879 } }
I20260709 13:16:54.399271 17170 leader_election.cc:290] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841), 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879)
W20260709 13:16:54.402592 16119 leader_election.cc:336] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111)
W20260709 13:16:54.404253 16121 consensus_peers.cc:597] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:54.404700 16121 leader_election.cc:336] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111)
I20260709 13:16:54.405018 16121 leader_election.cc:304] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c95f0816c41142699d80a2e326e17ab8; no voters: 903b3435239241eab01ea54ffb015ee3, bdeb23ab98c34a269f5b2e71b03c5e4e
I20260709 13:16:54.405804 17170 raft_consensus.cc:2749] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:16:54.423017  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:54.423961  6937 tablet_replica.cc:333] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.424919  6937 raft_consensus.cc:2243] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.425479  6937 raft_consensus.cc:2272] T e58eccbbe71e465188784170045cdb34 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.428264  6937 tablet_replica.cc:333] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
W20260709 13:16:54.428944 16121 consensus_peers.cc:597] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 -> Peer 903b3435239241eab01ea54ffb015ee3 (127.6.198.66:32879): Couldn't send request to peer 903b3435239241eab01ea54ffb015ee3. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:32879: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:54.429035  6937 raft_consensus.cc:2243] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.429587  6937 raft_consensus.cc:2272] T 0b83d43d98074cb09deac765b8b035e6 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.431613  6937 tablet_replica.cc:333] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.432170  6937 raft_consensus.cc:2243] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.432967  6937 raft_consensus.cc:2272] T d7ddd2107d2e4e73a47f9e4f6bdc73fb P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.434989  6937 tablet_replica.cc:333] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.435547  6937 raft_consensus.cc:2243] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.435928  6937 raft_consensus.cc:2272] T 4332ee7a936341baad672bfc16703cc7 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.437845  6937 tablet_replica.cc:333] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.438432  6937 raft_consensus.cc:2243] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.438804  6937 raft_consensus.cc:2272] T f72ec4c0d31742baa2d0d9d0aad7caed P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.440696  6937 tablet_replica.cc:333] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.441354  6937 raft_consensus.cc:2243] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.441859  6937 raft_consensus.cc:2272] T ad97b67157f44c618ddfebbc1c25588b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.444478  6937 tablet_replica.cc:333] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.445206  6937 raft_consensus.cc:2243] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.446023  6937 raft_consensus.cc:2272] T 28cbf604c51f48da99cb7e3089651236 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.448066  6937 tablet_replica.cc:333] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.448580  6937 raft_consensus.cc:2243] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.449007  6937 raft_consensus.cc:2272] T b5e6891e9cab4a7298664c3c431ce36d P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.450865  6937 tablet_replica.cc:333] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.451372  6937 raft_consensus.cc:2243] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.452078  6937 raft_consensus.cc:2272] T dadf33b9db824fdcbb63864c9e975c6b P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.453881  6937 tablet_replica.cc:333] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.454484  6937 raft_consensus.cc:2243] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.454908  6937 raft_consensus.cc:2272] T 62ac7645b9954621a685e4fc9ddde0dd P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.456670  6937 tablet_replica.cc:333] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.457219  6937 raft_consensus.cc:2243] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.457880  6937 raft_consensus.cc:2272] T 1a21a302bc17473392974aeea5a31d60 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.459792  6937 tablet_replica.cc:333] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.460399  6937 raft_consensus.cc:2243] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.460808  6937 raft_consensus.cc:2272] T 3424ea4fd86e4453a5d9fde62fc43e7f P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.462705  6937 tablet_replica.cc:333] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.463254  6937 raft_consensus.cc:2243] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.463996  6937 raft_consensus.cc:2272] T bd2e0e0244294ec18481adf638d57ff0 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.465672  6937 tablet_replica.cc:333] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.466323  6937 raft_consensus.cc:2243] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.466835  6937 raft_consensus.cc:2272] T de7a15b080c54551b2c51bee7af64600 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.469441  6937 tablet_replica.cc:333] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.470036  6937 raft_consensus.cc:2243] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.470546  6937 raft_consensus.cc:2272] T 8b94b09aec6746f69853f14b417f18e3 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:16:54.470613 16119 consensus_peers.cc:597] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 -> Peer bdeb23ab98c34a269f5b2e71b03c5e4e (127.6.198.65:45841): Couldn't send request to peer bdeb23ab98c34a269f5b2e71b03c5e4e. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45841: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:16:54.472505  6937 tablet_replica.cc:333] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.473064  6937 raft_consensus.cc:2243] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.473791  6937 raft_consensus.cc:2272] T aaa175423efe491da6747a37df086375 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.475835  6937 tablet_replica.cc:333] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.476366  6937 raft_consensus.cc:2243] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.476758  6937 raft_consensus.cc:2272] T c9df103185604f988d3abe94ae731141 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.478685  6937 tablet_replica.cc:333] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.479223  6937 raft_consensus.cc:2243] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.479614  6937 raft_consensus.cc:2272] T 31217a26afbc45258ea518a1623dbb01 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.481300  6937 tablet_replica.cc:333] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.481846  6937 raft_consensus.cc:2243] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.482256  6937 raft_consensus.cc:2272] T 7cca9877952a413a9112ea61481bc582 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.484189  6937 tablet_replica.cc:333] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.484711  6937 raft_consensus.cc:2243] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.485110  6937 raft_consensus.cc:2272] T 7454f1cf21c0400c8a3a8290d5d148dd P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.487041  6937 tablet_replica.cc:333] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.487545  6937 raft_consensus.cc:2243] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.487927  6937 raft_consensus.cc:2272] T 8f15b348b66b4bffae9c7f9eed8207e2 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.489717  6937 tablet_replica.cc:333] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.490253  6937 raft_consensus.cc:2243] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.490999  6937 raft_consensus.cc:2272] T 07ae89d92f6942d196dcc8d146b8f459 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.492789  6937 tablet_replica.cc:333] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.493320  6937 raft_consensus.cc:2243] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.493741  6937 raft_consensus.cc:2272] T 3a39139d4f2b48d09ba9eb320dfcf265 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.495644  6937 tablet_replica.cc:333] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.496171  6937 raft_consensus.cc:2243] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.496601  6937 raft_consensus.cc:2272] T 034918f94907492ca9a964c053edd50e P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.498409  6937 tablet_replica.cc:333] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.498927  6937 raft_consensus.cc:2243] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.499639  6937 raft_consensus.cc:2272] T 4633df69636042288f27156e6084dce9 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.501504  6937 tablet_replica.cc:333] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.502027  6937 raft_consensus.cc:2243] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.502460  6937 raft_consensus.cc:2272] T c6a3eeeeab11406e90e1fd10d63228e8 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.504220  6937 tablet_replica.cc:333] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.504753  6937 raft_consensus.cc:2243] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.505448  6937 raft_consensus.cc:2272] T d6e83ffadf17413a95b361a2cf2b1b7f P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.507328  6937 tablet_replica.cc:333] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.507891  6937 raft_consensus.cc:2243] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.508288  6937 raft_consensus.cc:2272] T 718e00ba6fa34542a71fc6d5758e92a1 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.510188  6937 tablet_replica.cc:333] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.510872  6937 raft_consensus.cc:2243] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.511595  6937 raft_consensus.cc:2272] T c895cd2e2a144ff692fcb615c095e82a P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.513525  6937 tablet_replica.cc:333] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.514063  6937 raft_consensus.cc:2243] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.514531  6937 raft_consensus.cc:2272] T cc17a40e973a46a18d21e55a0a939417 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.516418  6937 tablet_replica.cc:333] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.516996  6937 raft_consensus.cc:2243] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.517742  6937 raft_consensus.cc:2272] T 68a3e44fcd5b438486e1ec0241ef1e8d P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.519833  6937 tablet_replica.cc:333] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.520418  6937 raft_consensus.cc:2243] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.520843  6937 raft_consensus.cc:2272] T 4712e5b3e765415b916c9bf89f5b0d74 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.522845  6937 tablet_replica.cc:333] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.523396  6937 raft_consensus.cc:2243] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.523887  6937 raft_consensus.cc:2272] T 69d7d4e0bc194613bb410e793fd7bd39 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.525730  6937 tablet_replica.cc:333] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.526264  6937 raft_consensus.cc:2243] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.527051  6937 raft_consensus.cc:2272] T bf30209cb5544714bc183c72359c9656 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.528889  6937 tablet_replica.cc:333] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.529434  6937 raft_consensus.cc:2243] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.530166  6937 raft_consensus.cc:2272] T eebbb30a54a04a4aa567c6e16581ad1a P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.532042  6937 tablet_replica.cc:333] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.532650  6937 raft_consensus.cc:2243] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.533072  6937 raft_consensus.cc:2272] T 8a7a7d029fb040b881d5165c98065a12 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.535044  6937 tablet_replica.cc:333] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.535559  6937 raft_consensus.cc:2243] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.536273  6937 raft_consensus.cc:2272] T f46c65175b29436c8f0704ffa98d4eb7 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.538214  6937 tablet_replica.cc:333] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.539040  6937 raft_consensus.cc:2243] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.539615  6937 raft_consensus.cc:2272] T ab6469b36c7444ba906df4c1f29f6f0e P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.541934  6937 tablet_replica.cc:333] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.542541  6937 raft_consensus.cc:2243] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.542982  6937 raft_consensus.cc:2272] T ef6f890d60f14faead1e5986904c458a P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.545159  6937 tablet_replica.cc:333] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.545883  6937 raft_consensus.cc:2243] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.546325  6937 raft_consensus.cc:2272] T f7473dd845e447a0b5a453e01dbb4b6b P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.548401  6937 tablet_replica.cc:333] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.548916  6937 raft_consensus.cc:2243] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.549345  6937 raft_consensus.cc:2272] T ccc0c303e2dd466caf0e684c6ea83284 P c95f0816c41142699d80a2e326e17ab8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.551294  6937 tablet_replica.cc:333] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.551774  6937 raft_consensus.cc:2243] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.552222  6937 raft_consensus.cc:2272] T 2c2dba979b4b4cfdb4eb1159d36ae158 P c95f0816c41142699d80a2e326e17ab8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.554004  6937 tablet_replica.cc:333] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.554481  6937 raft_consensus.cc:2243] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:54.554893  6937 raft_consensus.cc:2272] T 3132182819ce40fdb691736db3dd1061 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.556670  6937 tablet_replica.cc:333] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8: stopping tablet replica
I20260709 13:16:54.557231  6937 raft_consensus.cc:2243] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.558002  6937 raft_consensus.cc:2272] T 1708b4e2385e408abc600d9b2f6d42a1 P c95f0816c41142699d80a2e326e17ab8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.623910  6937 tablet_server.cc:196] TabletServer@127.6.198.68:0 shutdown complete.
I20260709 13:16:54.685640  6937 master.cc:562] Master@127.6.198.126:41247 shutting down...
I20260709 13:16:54.715407  6937 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:54.716035  6937 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:54.716325  6937 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6fd52bd8ed974e4ca4e0ee6a84db3ebc: stopping tablet replica
I20260709 13:16:54.739960  6937 master.cc:584] Master@127.6.198.126:41247 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (36437 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:16:54.835274  6937 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.198.126:35929
I20260709 13:16:54.836483  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:54.842062 17191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:54.842105 17192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:54.843556 17194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:54.844404  6937 server_base.cc:1061] running on GCE node
I20260709 13:16:54.845468  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:54.845685  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:54.845846  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603014845830 us; error 0 us; skew 500 ppm
I20260709 13:16:54.846450  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:54.848934  6937 webserver.cc:533] Webserver started at http://127.6.198.126:38087/ using document root <none> and password file <none>
I20260709 13:16:54.849424  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:54.849622  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:54.849880  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:54.851122  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/master-0-root/instance:
uuid: "3553f97babe548af91484ab5daf50c73"
format_stamp: "Formatted at 2026-07-09 13:16:54 on dist-test-slave-tqzw"
I20260709 13:16:54.855559  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.002s
I20260709 13:16:54.859127 17199 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:54.860071  6937 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:16:54.860333  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/master-0-root
uuid: "3553f97babe548af91484ab5daf50c73"
format_stamp: "Formatted at 2026-07-09 13:16:54 on dist-test-slave-tqzw"
I20260709 13:16:54.860580  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:54.877939  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:54.879158  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:54.920099  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.126:35929
I20260709 13:16:54.920214 17250 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.126:35929 every 8 connection(s)
I20260709 13:16:54.924118 17251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:54.936496 17251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73: Bootstrap starting.
I20260709 13:16:54.941375 17251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:54.945669 17251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73: No bootstrap required, opened a new log
I20260709 13:16:54.947976 17251 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3553f97babe548af91484ab5daf50c73" member_type: VOTER }
I20260709 13:16:54.948352 17251 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:54.948523 17251 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3553f97babe548af91484ab5daf50c73, State: Initialized, Role: FOLLOWER
I20260709 13:16:54.949116 17251 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3553f97babe548af91484ab5daf50c73" member_type: VOTER }
I20260709 13:16:54.949524 17251 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:16:54.949709 17251 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:16:54.949923 17251 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:54.954509 17251 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3553f97babe548af91484ab5daf50c73" member_type: VOTER }
I20260709 13:16:54.954981 17251 leader_election.cc:304] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3553f97babe548af91484ab5daf50c73; no voters: 
I20260709 13:16:54.956131 17251 leader_election.cc:290] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:16:54.956583 17254 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:54.957950 17254 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [term 1 LEADER]: Becoming Leader. State: Replica: 3553f97babe548af91484ab5daf50c73, State: Running, Role: LEADER
I20260709 13:16:54.958684 17254 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3553f97babe548af91484ab5daf50c73" member_type: VOTER }
I20260709 13:16:54.959101 17251 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:16:54.963399 17255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3553f97babe548af91484ab5daf50c73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3553f97babe548af91484ab5daf50c73" member_type: VOTER } }
I20260709 13:16:54.964190 17255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:54.964216 17256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3553f97babe548af91484ab5daf50c73. Latest consensus state: current_term: 1 leader_uuid: "3553f97babe548af91484ab5daf50c73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3553f97babe548af91484ab5daf50c73" member_type: VOTER } }
I20260709 13:16:54.965265 17256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:54.970866 17261 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:16:54.978646 17261 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:16:54.979684  6937 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:16:54.988235 17261 catalog_manager.cc:1383] Generated new cluster ID: fd4653d05ceb48c7bde8a890251f7669
I20260709 13:16:54.988549 17261 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:16:55.016100 17261 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:16:55.017602 17261 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:16:55.031106 17261 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73: Generated new TSK 0
I20260709 13:16:55.031728 17261 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:16:55.047070  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:55.053336 17272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:55.054478 17273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:55.056591 17275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:55.056514  6937 server_base.cc:1061] running on GCE node
I20260709 13:16:55.057977  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:55.058157  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:55.058518  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603015058478 us; error 0 us; skew 500 ppm
I20260709 13:16:55.059092  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:55.062005  6937 webserver.cc:533] Webserver started at http://127.6.198.65:46053/ using document root <none> and password file <none>
I20260709 13:16:55.062582  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:55.062783  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:55.063059  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:55.064282  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-0-root/instance:
uuid: "0136a742d5fe41c0bfae2764293ee082"
format_stamp: "Formatted at 2026-07-09 13:16:55 on dist-test-slave-tqzw"
I20260709 13:16:55.068889  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:16:55.072523 17280 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:55.073403  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:16:55.073657  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-0-root
uuid: "0136a742d5fe41c0bfae2764293ee082"
format_stamp: "Formatted at 2026-07-09 13:16:55 on dist-test-slave-tqzw"
I20260709 13:16:55.073894  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:55.091117  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:55.092273  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:55.093660  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:55.095959  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:55.096138  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:55.096436  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:55.096588  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:55.144497  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.65:45147
I20260709 13:16:55.144639 17342 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.65:45147 every 8 connection(s)
I20260709 13:16:55.151228  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:55.159549 17347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:55.159857 17348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:55.164281 17350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:55.164923  6937 server_base.cc:1061] running on GCE node
I20260709 13:16:55.166780  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:55.167011  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:55.167215  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603015167199 us; error 0 us; skew 500 ppm
I20260709 13:16:55.167778  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:55.170727  6937 webserver.cc:533] Webserver started at http://127.6.198.66:41763/ using document root <none> and password file <none>
I20260709 13:16:55.171378  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:55.171631  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:55.171916  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:55.172807 17343 heartbeater.cc:344] Connected to a master server at 127.6.198.126:35929
I20260709 13:16:55.172957  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-1-root/instance:
uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
format_stamp: "Formatted at 2026-07-09 13:16:55 on dist-test-slave-tqzw"
I20260709 13:16:55.173270 17343 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:55.174081 17343 heartbeater.cc:507] Master 127.6.198.126:35929 requested a full tablet report, sending...
I20260709 13:16:55.176148 17216 ts_manager.cc:194] Registered new tserver with Master: 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147)
I20260709 13:16:55.178318  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:16:55.178393 17216 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54640
I20260709 13:16:55.182675 17355 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:55.183472  6937 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:16:55.183800  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-1-root
uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
format_stamp: "Formatted at 2026-07-09 13:16:55 on dist-test-slave-tqzw"
I20260709 13:16:55.184087  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:55.194718  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:55.195880  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:55.197377  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:55.199662  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:55.199855  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:55.200162  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:55.200313  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:55.242249  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.66:40141
I20260709 13:16:55.242440 17417 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.66:40141 every 8 connection(s)
I20260709 13:16:55.247192  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:55.254968 17421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:55.255863 17422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:55.259371 17424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:55.260239  6937 server_base.cc:1061] running on GCE node
I20260709 13:16:55.260681 17418 heartbeater.cc:344] Connected to a master server at 127.6.198.126:35929
I20260709 13:16:55.261186 17418 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:55.261274  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:55.261626  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:55.261840  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603015261821 us; error 0 us; skew 500 ppm
I20260709 13:16:55.262029 17418 heartbeater.cc:507] Master 127.6.198.126:35929 requested a full tablet report, sending...
I20260709 13:16:55.262799  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:55.265414 17216 ts_manager.cc:194] Registered new tserver with Master: 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
I20260709 13:16:55.267023 17216 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54644
I20260709 13:16:55.267483  6937 webserver.cc:533] Webserver started at http://127.6.198.67:45993/ using document root <none> and password file <none>
I20260709 13:16:55.268049  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:55.268257  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:55.268527  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:55.269661  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-2-root/instance:
uuid: "baa0fccfc08d477ba279ad0968edb388"
format_stamp: "Formatted at 2026-07-09 13:16:55 on dist-test-slave-tqzw"
I20260709 13:16:55.274662  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 13:16:55.278179 17429 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:55.279145  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:16:55.279528  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-2-root
uuid: "baa0fccfc08d477ba279ad0968edb388"
format_stamp: "Formatted at 2026-07-09 13:16:55 on dist-test-slave-tqzw"
I20260709 13:16:55.279906  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:55.292485  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:55.293771  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:55.295284  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:55.297780  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:55.298034  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:55.298285  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:55.298534  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:16:55.351838  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.67:42031
I20260709 13:16:55.351984 17491 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.67:42031 every 8 connection(s)
I20260709 13:16:55.367363 17492 heartbeater.cc:344] Connected to a master server at 127.6.198.126:35929
I20260709 13:16:55.367920 17492 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:55.368772 17492 heartbeater.cc:507] Master 127.6.198.126:35929 requested a full tablet report, sending...
I20260709 13:16:55.370777 17216 ts_manager.cc:194] Registered new tserver with Master: baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:55.371389  6937 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015943341s
I20260709 13:16:55.372824 17216 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54652
I20260709 13:16:55.400660 17216 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54654:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:16:55.403959 17216 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:16:55.488842 17457 tablet_service.cc:1511] Processing CreateTablet for tablet 04780cde858b4000bfd73d06bbb04158 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:16:55.490319 17457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04780cde858b4000bfd73d06bbb04158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.492412 17455 tablet_service.cc:1511] Processing CreateTablet for tablet 75f1eba07e9a41b9a1a5e4b47fb76d5e (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:16:55.492835 17456 tablet_service.cc:1511] Processing CreateTablet for tablet f7153ce9502848edac1c5ef91ef65715 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:16:55.496332 17455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f1eba07e9a41b9a1a5e4b47fb76d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.497255 17308 tablet_service.cc:1511] Processing CreateTablet for tablet 04780cde858b4000bfd73d06bbb04158 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:16:55.498687 17308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04780cde858b4000bfd73d06bbb04158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.501888 17456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7153ce9502848edac1c5ef91ef65715. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.503845 17307 tablet_service.cc:1511] Processing CreateTablet for tablet f7153ce9502848edac1c5ef91ef65715 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:16:55.505395 17307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7153ce9502848edac1c5ef91ef65715. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.507642 17306 tablet_service.cc:1511] Processing CreateTablet for tablet 75f1eba07e9a41b9a1a5e4b47fb76d5e (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:16:55.508987 17306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f1eba07e9a41b9a1a5e4b47fb76d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.511541 17305 tablet_service.cc:1511] Processing CreateTablet for tablet 7e989e4c8d2f4c3690126a1eca30ddfa (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:16:55.512876 17305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e989e4c8d2f4c3690126a1eca30ddfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.519845 17304 tablet_service.cc:1511] Processing CreateTablet for tablet d293e2c8bea3443189ee867e1d5fa846 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:16:55.521260 17304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d293e2c8bea3443189ee867e1d5fa846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.504823 17454 tablet_service.cc:1511] Processing CreateTablet for tablet 7e989e4c8d2f4c3690126a1eca30ddfa (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:16:55.523180 17454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e989e4c8d2f4c3690126a1eca30ddfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.525295 17303 tablet_service.cc:1511] Processing CreateTablet for tablet 2423d9c513494d918102933924b43074 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:16:55.529315 17302 tablet_service.cc:1511] Processing CreateTablet for tablet 177781087d244f0a8d0edd7e6df0160b (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:16:55.531488 17301 tablet_service.cc:1511] Processing CreateTablet for tablet 6c6dc3aeee5c486b83aa69bac75c7868 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:16:55.533840 17300 tablet_service.cc:1511] Processing CreateTablet for tablet 8a5d2fb3f9ed4769a772ac524a4713e6 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:16:55.538904 17303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2423d9c513494d918102933924b43074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.539424 17383 tablet_service.cc:1511] Processing CreateTablet for tablet 04780cde858b4000bfd73d06bbb04158 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:16:55.540714 17383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04780cde858b4000bfd73d06bbb04158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.543529 17382 tablet_service.cc:1511] Processing CreateTablet for tablet f7153ce9502848edac1c5ef91ef65715 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:16:55.545079 17382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7153ce9502848edac1c5ef91ef65715. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.556651 17380 tablet_service.cc:1511] Processing CreateTablet for tablet 7e989e4c8d2f4c3690126a1eca30ddfa (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:16:55.558081 17380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e989e4c8d2f4c3690126a1eca30ddfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.561029 17452 tablet_service.cc:1511] Processing CreateTablet for tablet 2423d9c513494d918102933924b43074 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:16:55.564574 17451 tablet_service.cc:1511] Processing CreateTablet for tablet 177781087d244f0a8d0edd7e6df0160b (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:16:55.566061 17451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 177781087d244f0a8d0edd7e6df0160b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.571525 17453 tablet_service.cc:1511] Processing CreateTablet for tablet d293e2c8bea3443189ee867e1d5fa846 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:16:55.573897 17449 tablet_service.cc:1511] Processing CreateTablet for tablet 8a5d2fb3f9ed4769a772ac524a4713e6 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:16:55.575227 17450 tablet_service.cc:1511] Processing CreateTablet for tablet 6c6dc3aeee5c486b83aa69bac75c7868 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:16:55.554082 17381 tablet_service.cc:1511] Processing CreateTablet for tablet 75f1eba07e9a41b9a1a5e4b47fb76d5e (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:16:55.578763 17381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75f1eba07e9a41b9a1a5e4b47fb76d5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.582693 17450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6dc3aeee5c486b83aa69bac75c7868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.583245 17448 tablet_service.cc:1511] Processing CreateTablet for tablet 4522d72f7948494a83af8aa9e4fbaddc (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:16:55.584633 17448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4522d72f7948494a83af8aa9e4fbaddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.586585 17452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2423d9c513494d918102933924b43074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.589438 17299 tablet_service.cc:1511] Processing CreateTablet for tablet 4522d72f7948494a83af8aa9e4fbaddc (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:16:55.591504 17379 tablet_service.cc:1511] Processing CreateTablet for tablet d293e2c8bea3443189ee867e1d5fa846 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:16:55.592033 17378 tablet_service.cc:1511] Processing CreateTablet for tablet 2423d9c513494d918102933924b43074 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:16:55.592929 17379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d293e2c8bea3443189ee867e1d5fa846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.593533 17302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 177781087d244f0a8d0edd7e6df0160b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.594381 17377 tablet_service.cc:1511] Processing CreateTablet for tablet 177781087d244f0a8d0edd7e6df0160b (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:16:55.595743 17377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 177781087d244f0a8d0edd7e6df0160b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.598374 17453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d293e2c8bea3443189ee867e1d5fa846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.599300 17301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6dc3aeee5c486b83aa69bac75c7868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.599596 17376 tablet_service.cc:1511] Processing CreateTablet for tablet 6c6dc3aeee5c486b83aa69bac75c7868 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:16:55.601394 17376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c6dc3aeee5c486b83aa69bac75c7868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.602713 17300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a5d2fb3f9ed4769a772ac524a4713e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.603835 17375 tablet_service.cc:1511] Processing CreateTablet for tablet 8a5d2fb3f9ed4769a772ac524a4713e6 (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:16:55.605268 17375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a5d2fb3f9ed4769a772ac524a4713e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.605938 17449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a5d2fb3f9ed4769a772ac524a4713e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.606513 17299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4522d72f7948494a83af8aa9e4fbaddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.607405 17374 tablet_service.cc:1511] Processing CreateTablet for tablet 4522d72f7948494a83af8aa9e4fbaddc (DEFAULT_TABLE table=test-workload [id=41502e3e769c4ef284ca6576ef6d156c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:16:55.609028 17374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4522d72f7948494a83af8aa9e4fbaddc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.614679 17378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2423d9c513494d918102933924b43074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:55.772662 17509 tablet_bootstrap.cc:492] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:16:55.780273 17509 tablet_bootstrap.cc:654] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:55.792631 17507 tablet_bootstrap.cc:492] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388: Bootstrap starting.
I20260709 13:16:55.797833 17508 tablet_bootstrap.cc:492] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8: Bootstrap starting.
I20260709 13:16:55.799731 17507 tablet_bootstrap.cc:654] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:55.806017 17508 tablet_bootstrap.cc:654] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:55.816937 17509 tablet_bootstrap.cc:492] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082: No bootstrap required, opened a new log
I20260709 13:16:55.817767 17509 ts_tablet_manager.cc:1403] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.052s	user 0.010s	sys 0.003s
I20260709 13:16:55.819860 17507 tablet_bootstrap.cc:492] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388: No bootstrap required, opened a new log
I20260709 13:16:55.820432 17507 ts_tablet_manager.cc:1403] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.005s
I20260709 13:16:55.820906 17509 raft_consensus.cc:359] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.820480 17508 tablet_bootstrap.cc:492] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8: No bootstrap required, opened a new log
I20260709 13:16:55.822468 17509 raft_consensus.cc:385] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:55.822715 17508 ts_tablet_manager.cc:1403] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent bootstrapping tablet: real 0.035s	user 0.007s	sys 0.009s
I20260709 13:16:55.822814 17509 raft_consensus.cc:740] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:16:55.823066 17507 raft_consensus.cc:359] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.823904 17507 raft_consensus.cc:385] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:55.824267 17507 raft_consensus.cc:740] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Initialized, Role: FOLLOWER
I20260709 13:16:55.823794 17509 consensus_queue.cc:260] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.824951 17507 consensus_queue.cc:260] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.826164 17343 heartbeater.cc:499] Master 127.6.198.126:35929 was elected leader, sending a full tablet report...
I20260709 13:16:55.826027 17508 raft_consensus.cc:359] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.826586 17509 ts_tablet_manager.cc:1434] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.008s
I20260709 13:16:55.826853 17508 raft_consensus.cc:385] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:55.827148 17508 raft_consensus.cc:740] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:55.827629 17509 tablet_bootstrap.cc:492] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:16:55.828047 17508 consensus_queue.cc:260] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.833978 17507 ts_tablet_manager.cc:1434] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.003s
I20260709 13:16:55.834702 17492 heartbeater.cc:499] Master 127.6.198.126:35929 was elected leader, sending a full tablet report...
I20260709 13:16:55.835778 17509 tablet_bootstrap.cc:654] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:55.843824 17507 tablet_bootstrap.cc:492] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388: Bootstrap starting.
I20260709 13:16:55.853729 17507 tablet_bootstrap.cc:654] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388: Neither blocks nor log segments found. Creating new log.
W20260709 13:16:55.856530 17493 tablet.cc:2406] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:16:55.859788 17509 tablet_bootstrap.cc:492] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082: No bootstrap required, opened a new log
I20260709 13:16:55.860517 17509 ts_tablet_manager.cc:1403] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.033s	user 0.018s	sys 0.012s
I20260709 13:16:55.864183 17507 tablet_bootstrap.cc:492] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388: No bootstrap required, opened a new log
I20260709 13:16:55.864703 17507 ts_tablet_manager.cc:1403] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.000s
I20260709 13:16:55.864542 17509 raft_consensus.cc:359] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.865674 17509 raft_consensus.cc:385] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:55.866068 17509 raft_consensus.cc:740] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:16:55.867108 17509 consensus_queue.cc:260] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.867827 17507 raft_consensus.cc:359] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.868595 17507 raft_consensus.cc:385] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:55.868896 17507 raft_consensus.cc:740] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Initialized, Role: FOLLOWER
I20260709 13:16:55.869889 17509 ts_tablet_manager.cc:1434] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:16:55.869653 17507 consensus_queue.cc:260] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.870173 17418 heartbeater.cc:499] Master 127.6.198.126:35929 was elected leader, sending a full tablet report...
I20260709 13:16:55.870572 17508 ts_tablet_manager.cc:1434] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent starting tablet: real 0.047s	user 0.019s	sys 0.028s
I20260709 13:16:55.873034 17508 tablet_bootstrap.cc:492] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8: Bootstrap starting.
I20260709 13:16:55.873570 17507 ts_tablet_manager.cc:1434] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:16:55.874501 17507 tablet_bootstrap.cc:492] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388: Bootstrap starting.
I20260709 13:16:55.879295 17508 tablet_bootstrap.cc:654] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:55.870903 17509 tablet_bootstrap.cc:492] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:16:55.880729 17507 tablet_bootstrap.cc:654] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:55.886611 17509 tablet_bootstrap.cc:654] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:55.891994 17508 tablet_bootstrap.cc:492] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8: No bootstrap required, opened a new log
I20260709 13:16:55.892627 17507 tablet_bootstrap.cc:492] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388: No bootstrap required, opened a new log
I20260709 13:16:55.892628 17508 ts_tablet_manager.cc:1403] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 13:16:55.897050 17507 ts_tablet_manager.cc:1403] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.009s
I20260709 13:16:55.893083 17514 raft_consensus.cc:493] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.900629 17514 raft_consensus.cc:515] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.900761 17508 raft_consensus.cc:359] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } }
I20260709 13:16:55.901790 17508 raft_consensus.cc:385] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:55.902194 17508 raft_consensus.cc:740] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:55.903196 17508 consensus_queue.cc:260] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } }
I20260709 13:16:55.909323 17508 ts_tablet_manager.cc:1434] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 13:16:55.910005 17514 leader_election.cc:290] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
I20260709 13:16:55.910374 17508 tablet_bootstrap.cc:492] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8: Bootstrap starting.
I20260709 13:16:55.907094 17507 raft_consensus.cc:359] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } }
I20260709 13:16:55.914495 17507 raft_consensus.cc:385] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:55.914790 17507 raft_consensus.cc:740] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Initialized, Role: FOLLOWER
I20260709 13:16:55.915563 17507 consensus_queue.cc:260] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } }
I20260709 13:16:55.916819 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7153ce9502848edac1c5ef91ef65715" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082" is_pre_election: true
I20260709 13:16:55.917726 17507 ts_tablet_manager.cc:1434] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.000s
W20260709 13:16:55.918489 17431 leader_election.cc:343] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:55.923354 17513 raft_consensus.cc:493] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.923808 17508 tablet_bootstrap.cc:654] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:55.923866 17513 raft_consensus.cc:515] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.929183 17513 leader_election.cc:290] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:55.931622 17509 tablet_bootstrap.cc:492] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: No bootstrap required, opened a new log
I20260709 13:16:55.932318 17509 ts_tablet_manager.cc:1403] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.011s
I20260709 13:16:55.936280 17509 raft_consensus.cc:359] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.937202 17509 raft_consensus.cc:385] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:55.937568 17509 raft_consensus.cc:740] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:16:55.938477 17509 consensus_queue.cc:260] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.941201 17507 tablet_bootstrap.cc:492] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388: Bootstrap starting.
I20260709 13:16:55.947620 17509 ts_tablet_manager.cc:1434] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:16:55.948664 17509 tablet_bootstrap.cc:492] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:16:55.953799 17393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7153ce9502848edac1c5ef91ef65715" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" is_pre_election: true
W20260709 13:16:55.955746 17433 leader_election.cc:343] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:55.956297 17433 leader_election.cc:304] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: baa0fccfc08d477ba279ad0968edb388; no voters: 0136a742d5fe41c0bfae2764293ee082, 4ca45d6eb7e24e6cb99de9028e1142b8
I20260709 13:16:55.957119 17514 raft_consensus.cc:2749] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:55.959005 17509 tablet_bootstrap.cc:654] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:55.964046 17508 tablet_bootstrap.cc:492] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8: No bootstrap required, opened a new log
I20260709 13:16:55.964602 17508 ts_tablet_manager.cc:1403] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent bootstrapping tablet: real 0.054s	user 0.018s	sys 0.002s
I20260709 13:16:55.966113 17515 raft_consensus.cc:493] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.966673 17507 tablet_bootstrap.cc:654] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:55.966631 17515 raft_consensus.cc:515] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } }
I20260709 13:16:55.967826 17508 raft_consensus.cc:359] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.968621 17508 raft_consensus.cc:385] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:55.968978 17508 raft_consensus.cc:740] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:55.969426 17515 leader_election.cc:290] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031), 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147)
I20260709 13:16:55.969950 17508 consensus_queue.cc:260] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:55.972558 17508 ts_tablet_manager.cc:1434] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
W20260709 13:16:55.972652 17264 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:55.973361 17467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a5d2fb3f9ed4769a772ac524a4713e6" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388" is_pre_election: true
W20260709 13:16:55.975425 17282 leader_election.cc:343] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:55.979336 17508 tablet_bootstrap.cc:492] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8: Bootstrap starting.
I20260709 13:16:55.989159 17508 tablet_bootstrap.cc:654] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:55.991605 17393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a5d2fb3f9ed4769a772ac524a4713e6" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" is_pre_election: true
W20260709 13:16:55.995126 17284 leader_election.cc:343] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:55.995781 17509 tablet_bootstrap.cc:492] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082: No bootstrap required, opened a new log
I20260709 13:16:55.996584 17509 ts_tablet_manager.cc:1403] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.048s	user 0.015s	sys 0.006s
I20260709 13:16:55.995568 17284 leader_election.cc:304] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082; no voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388
I20260709 13:16:55.998796 17513 raft_consensus.cc:2749] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:56.000262 17509 raft_consensus.cc:359] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.001271 17509 raft_consensus.cc:385] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.001638 17509 raft_consensus.cc:740] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.005707 17509 consensus_queue.cc:260] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.008437 17509 ts_tablet_manager.cc:1434] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 13:16:56.009248 17507 tablet_bootstrap.cc:492] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388: No bootstrap required, opened a new log
I20260709 13:16:56.009442 17509 tablet_bootstrap.cc:492] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:16:56.009764 17507 ts_tablet_manager.cc:1403] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388: Time spent bootstrapping tablet: real 0.070s	user 0.010s	sys 0.015s
I20260709 13:16:56.011790 17508 tablet_bootstrap.cc:492] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8: No bootstrap required, opened a new log
I20260709 13:16:56.012725 17507 raft_consensus.cc:359] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.013645 17507 raft_consensus.cc:385] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.014132 17507 raft_consensus.cc:740] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.015067 17507 consensus_queue.cc:260] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.015764 17467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04780cde858b4000bfd73d06bbb04158" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388" is_pre_election: true
I20260709 13:16:56.016405 17467 raft_consensus.cc:2468] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 0.
I20260709 13:16:56.016741 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04780cde858b4000bfd73d06bbb04158" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082" is_pre_election: true
I20260709 13:16:56.017366 17509 tablet_bootstrap.cc:654] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.017822 17357 leader_election.cc:304] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:16:56.018726 17515 raft_consensus.cc:2804] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:56.019114 17515 raft_consensus.cc:493] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.019435 17515 raft_consensus.cc:3060] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.021917 17507 ts_tablet_manager.cc:1434] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:16:56.023100 17507 tablet_bootstrap.cc:492] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388: Bootstrap starting.
W20260709 13:16:56.026813 17357 leader_election.cc:343] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:56.029531 17507 tablet_bootstrap.cc:654] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.013693 17508 ts_tablet_manager.cc:1403] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.009s
I20260709 13:16:56.031339 17515 raft_consensus.cc:515] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } }
I20260709 13:16:56.033145 17532 raft_consensus.cc:493] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.033334 17515 leader_election.cc:290] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 election: Requested vote from peers baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031), 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147)
I20260709 13:16:56.033671 17532 raft_consensus.cc:515] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.034775 17509 tablet_bootstrap.cc:492] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082: No bootstrap required, opened a new log
I20260709 13:16:56.035001 17467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04780cde858b4000bfd73d06bbb04158" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
I20260709 13:16:56.035835 17467 raft_consensus.cc:3060] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.035883 17532 leader_election.cc:290] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.036652 17317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082" is_pre_election: true
I20260709 13:16:56.033685 17508 raft_consensus.cc:359] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.037811 17508 raft_consensus.cc:385] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.038192 17508 raft_consensus.cc:740] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.035202 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04780cde858b4000bfd73d06bbb04158" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:16:56.039474 17466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388" is_pre_election: true
I20260709 13:16:56.039326 17508 consensus_queue.cc:260] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.040177 17466 raft_consensus.cc:2468] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 0.
W20260709 13:16:56.041742 17357 leader_election.cc:343] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:16:56.042403 17357 leader_election.cc:343] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:56.043063 17357 leader_election.cc:304] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 0136a742d5fe41c0bfae2764293ee082
I20260709 13:16:56.043574 17467 raft_consensus.cc:2468] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 1.
I20260709 13:16:56.044206 17515 raft_consensus.cc:2804] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:56.044606 17515 raft_consensus.cc:493] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.044826 17509 ts_tablet_manager.cc:1403] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.036s	user 0.007s	sys 0.016s
I20260709 13:16:56.044972 17515 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.047767 17509 raft_consensus.cc:359] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.048478 17509 raft_consensus.cc:385] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.048764 17509 raft_consensus.cc:740] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.049608 17357 leader_election.cc:304] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 0136a742d5fe41c0bfae2764293ee082
I20260709 13:16:56.050414 17532 raft_consensus.cc:2804] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:56.050774 17508 ts_tablet_manager.cc:1434] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 13:16:56.051610 17515 raft_consensus.cc:515] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.054191 17515 leader_election.cc:290] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.056502 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:16:56.058130 17467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
W20260709 13:16:56.058324 17357 leader_election.cc:343] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:56.058399 17507 tablet_bootstrap.cc:492] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388: No bootstrap required, opened a new log
I20260709 13:16:56.058754 17467 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.058977 17532 raft_consensus.cc:697] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Becoming Leader. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:16:56.059188 17507 ts_tablet_manager.cc:1403] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.014s
I20260709 13:16:56.059280 17508 tablet_bootstrap.cc:492] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8: Bootstrap starting.
I20260709 13:16:56.059882 17532 consensus_queue.cc:237] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } }
I20260709 13:16:56.062750 17507 raft_consensus.cc:359] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.063618 17507 raft_consensus.cc:385] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.063352 17509 consensus_queue.cc:260] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.065801 17509 ts_tablet_manager.cc:1434] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:16:56.066874 17508 tablet_bootstrap.cc:654] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.067282 17509 tablet_bootstrap.cc:492] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:16:56.065894 17467 raft_consensus.cc:2468] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 1.
I20260709 13:16:56.064049 17507 raft_consensus.cc:740] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.075727 17357 leader_election.cc:304] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 0136a742d5fe41c0bfae2764293ee082
I20260709 13:16:56.075809 17507 consensus_queue.cc:260] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.078663 17507 ts_tablet_manager.cc:1434] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:16:56.074574 17509 tablet_bootstrap.cc:654] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.077019 17532 raft_consensus.cc:2804] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:56.076993 17215 catalog_manager.cc:5697] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:56.082924 17532 raft_consensus.cc:697] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Becoming Leader. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:16:56.083232 17507 tablet_bootstrap.cc:492] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388: Bootstrap starting.
I20260709 13:16:56.083927 17532 consensus_queue.cc:237] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.072420 17508 tablet_bootstrap.cc:492] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8: No bootstrap required, opened a new log
I20260709 13:16:56.085538 17508 ts_tablet_manager.cc:1403] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.003s
I20260709 13:16:56.085618 17534 raft_consensus.cc:493] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.086140 17534 raft_consensus.cc:515] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.088251 17534 leader_election.cc:290] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.088586 17508 raft_consensus.cc:359] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.089308 17508 raft_consensus.cc:385] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.089592 17508 raft_consensus.cc:740] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.090209 17507 tablet_bootstrap.cc:654] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.091885 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f1eba07e9a41b9a1a5e4b47fb76d5e" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082" is_pre_election: true
I20260709 13:16:56.090319 17508 consensus_queue.cc:260] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.093721 17318 raft_consensus.cc:2468] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 0.
I20260709 13:16:56.094816 17467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f1eba07e9a41b9a1a5e4b47fb76d5e" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388" is_pre_election: true
I20260709 13:16:56.095512 17467 raft_consensus.cc:2468] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 0.
I20260709 13:16:56.095419 17357 leader_election.cc:304] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, 4ca45d6eb7e24e6cb99de9028e1142b8; no voters: 
I20260709 13:16:56.096642 17508 ts_tablet_manager.cc:1434] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:16:56.097345 17534 raft_consensus.cc:2804] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:56.097893 17534 raft_consensus.cc:493] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.098371 17534 raft_consensus.cc:3060] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.097677 17508 tablet_bootstrap.cc:492] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8: Bootstrap starting.
I20260709 13:16:56.108986 17534 raft_consensus.cc:515] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.110356 17508 tablet_bootstrap.cc:654] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.113883 17509 tablet_bootstrap.cc:492] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: No bootstrap required, opened a new log
I20260709 13:16:56.114516 17509 ts_tablet_manager.cc:1403] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.047s	user 0.006s	sys 0.008s
I20260709 13:16:56.120796 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f1eba07e9a41b9a1a5e4b47fb76d5e" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:16:56.121595 17318 raft_consensus.cc:3060] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.126602 17508 tablet_bootstrap.cc:492] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8: No bootstrap required, opened a new log
I20260709 13:16:56.127149 17508 ts_tablet_manager.cc:1403] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.008s
I20260709 13:16:56.127969 17215 catalog_manager.cc:5697] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:56.130101 17508 raft_consensus.cc:359] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.130872 17508 raft_consensus.cc:385] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.131174 17508 raft_consensus.cc:740] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.131996 17508 consensus_queue.cc:260] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.132521 17467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f1eba07e9a41b9a1a5e4b47fb76d5e" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
I20260709 13:16:56.133344 17467 raft_consensus.cc:3060] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.134479 17508 ts_tablet_manager.cc:1434] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.005s
I20260709 13:16:56.135480 17508 tablet_bootstrap.cc:492] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8: Bootstrap starting.
I20260709 13:16:56.139657 17513 raft_consensus.cc:493] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.140327 17513 raft_consensus.cc:515] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.143563 17508 tablet_bootstrap.cc:654] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.144294 17466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2423d9c513494d918102933924b43074" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388" is_pre_election: true
I20260709 13:16:56.144693 17393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2423d9c513494d918102933924b43074" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" is_pre_election: true
W20260709 13:16:56.145817 17282 leader_election.cc:343] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:56.146142 17513 leader_election.cc:290] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
W20260709 13:16:56.146361 17284 leader_election.cc:343] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:56.147028 17284 leader_election.cc:304] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082; no voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388
I20260709 13:16:56.148006 17513 raft_consensus.cc:2749] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:56.151012 17534 leader_election.cc:290] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.151556 17509 raft_consensus.cc:359] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.152316 17509 raft_consensus.cc:385] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.152606 17509 raft_consensus.cc:740] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.152901 17467 raft_consensus.cc:2468] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 1.
I20260709 13:16:56.153271 17508 tablet_bootstrap.cc:492] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8: No bootstrap required, opened a new log
I20260709 13:16:56.153348 17509 consensus_queue.cc:260] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.154006 17508 ts_tablet_manager.cc:1403] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.005s
I20260709 13:16:56.154443 17357 leader_election.cc:304] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:16:56.155224 17509 ts_tablet_manager.cc:1434] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.000s
I20260709 13:16:56.156272 17509 tablet_bootstrap.cc:492] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:16:56.157413 17508 raft_consensus.cc:359] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.158177 17508 raft_consensus.cc:385] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.158546 17508 raft_consensus.cc:740] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.159296 17508 consensus_queue.cc:260] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.159727 17318 raft_consensus.cc:2468] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 1.
I20260709 13:16:56.160431 17532 raft_consensus.cc:2804] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:56.160920 17532 raft_consensus.cc:697] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Becoming Leader. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:16:56.161942 17508 ts_tablet_manager.cc:1434] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:16:56.161742 17532 consensus_queue.cc:237] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.163077 17508 tablet_bootstrap.cc:492] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8: Bootstrap starting.
I20260709 13:16:56.170246 17508 tablet_bootstrap.cc:654] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.172619 17507 tablet_bootstrap.cc:492] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388: No bootstrap required, opened a new log
I20260709 13:16:56.173205 17507 ts_tablet_manager.cc:1403] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388: Time spent bootstrapping tablet: real 0.090s	user 0.022s	sys 0.053s
I20260709 13:16:56.174319 17509 tablet_bootstrap.cc:654] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.173890 17215 catalog_manager.cc:5697] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:56.177024 17507 raft_consensus.cc:359] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.178004 17507 raft_consensus.cc:385] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.178380 17507 raft_consensus.cc:740] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.179484 17507 consensus_queue.cc:260] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.188170 17507 ts_tablet_manager.cc:1434] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:16:56.189522 17507 tablet_bootstrap.cc:492] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388: Bootstrap starting.
I20260709 13:16:56.195676 17508 tablet_bootstrap.cc:492] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8: No bootstrap required, opened a new log
I20260709 13:16:56.196185 17508 ts_tablet_manager.cc:1403] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent bootstrapping tablet: real 0.033s	user 0.022s	sys 0.007s
I20260709 13:16:56.198729 17508 raft_consensus.cc:359] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.199388 17508 raft_consensus.cc:385] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.199656 17508 raft_consensus.cc:740] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.200281 17508 consensus_queue.cc:260] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.201152 17507 tablet_bootstrap.cc:654] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.201156 17509 tablet_bootstrap.cc:492] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082: No bootstrap required, opened a new log
I20260709 13:16:56.201786 17509 ts_tablet_manager.cc:1403] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.011s
I20260709 13:16:56.202163 17508 ts_tablet_manager.cc:1434] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:16:56.203119 17508 tablet_bootstrap.cc:492] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8: Bootstrap starting.
I20260709 13:16:56.205055 17509 raft_consensus.cc:359] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.205986 17509 raft_consensus.cc:385] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.206393 17509 raft_consensus.cc:740] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.207413 17509 consensus_queue.cc:260] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.208453 17508 tablet_bootstrap.cc:654] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.211030 17509 ts_tablet_manager.cc:1434] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:16:56.212059 17509 tablet_bootstrap.cc:492] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:16:56.213614 17507 tablet_bootstrap.cc:492] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388: No bootstrap required, opened a new log
I20260709 13:16:56.214260 17507 ts_tablet_manager.cc:1403] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.002s
I20260709 13:16:56.218008 17507 raft_consensus.cc:359] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.219097 17507 raft_consensus.cc:385] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.219444 17509 tablet_bootstrap.cc:654] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.219480 17507 raft_consensus.cc:740] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.220898 17507 consensus_queue.cc:260] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.230646 17507 ts_tablet_manager.cc:1434] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.001s
I20260709 13:16:56.231176 17509 tablet_bootstrap.cc:492] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082: No bootstrap required, opened a new log
I20260709 13:16:56.231653 17507 tablet_bootstrap.cc:492] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388: Bootstrap starting.
I20260709 13:16:56.231674 17509 ts_tablet_manager.cc:1403] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.008s
I20260709 13:16:56.234571 17509 raft_consensus.cc:359] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.235311 17509 raft_consensus.cc:385] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.235620 17509 raft_consensus.cc:740] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.236400 17509 consensus_queue.cc:260] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.237725 17507 tablet_bootstrap.cc:654] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.238442 17509 ts_tablet_manager.cc:1434] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:16:56.239485 17509 tablet_bootstrap.cc:492] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:16:56.240918 17508 tablet_bootstrap.cc:492] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8: No bootstrap required, opened a new log
I20260709 13:16:56.241421 17508 ts_tablet_manager.cc:1403] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.024s
I20260709 13:16:56.244133 17508 raft_consensus.cc:359] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.244769 17508 raft_consensus.cc:385] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.245041 17508 raft_consensus.cc:740] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.245774 17508 consensus_queue.cc:260] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.246609 17509 tablet_bootstrap.cc:654] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.250273 17508 ts_tablet_manager.cc:1434] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:16:56.251214 17508 tablet_bootstrap.cc:492] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8: Bootstrap starting.
I20260709 13:16:56.256536 17508 tablet_bootstrap.cc:654] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.264907 17508 tablet_bootstrap.cc:492] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8: No bootstrap required, opened a new log
I20260709 13:16:56.264994 17507 tablet_bootstrap.cc:492] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388: No bootstrap required, opened a new log
I20260709 13:16:56.265337 17508 ts_tablet_manager.cc:1403] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.005s
I20260709 13:16:56.265506 17507 ts_tablet_manager.cc:1403] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.016s
I20260709 13:16:56.265583 17509 tablet_bootstrap.cc:492] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082: No bootstrap required, opened a new log
I20260709 13:16:56.266263 17509 ts_tablet_manager.cc:1403] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.015s
I20260709 13:16:56.268108 17507 raft_consensus.cc:359] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.268057 17508 raft_consensus.cc:359] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.268901 17507 raft_consensus.cc:385] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.268977 17508 raft_consensus.cc:385] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.268774 17509 raft_consensus.cc:359] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } }
I20260709 13:16:56.269259 17507 raft_consensus.cc:740] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.269366 17508 raft_consensus.cc:740] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.269519 17509 raft_consensus.cc:385] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.269896 17509 raft_consensus.cc:740] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.270112 17532 raft_consensus.cc:493] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.270299 17508 consensus_queue.cc:260] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.270308 17507 consensus_queue.cc:260] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.270572 17509 consensus_queue.cc:260] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } }
I20260709 13:16:56.272444 17508 ts_tablet_manager.cc:1434] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:16:56.273970 17509 ts_tablet_manager.cc:1434] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:16:56.274674 17514 raft_consensus.cc:493] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.274892 17509 tablet_bootstrap.cc:492] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:16:56.275192 17514 raft_consensus.cc:515] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.270648 17532 raft_consensus.cc:515] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.277227 17532 leader_election.cc:290] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.279358 17467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6dc3aeee5c486b83aa69bac75c7868" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388" is_pre_election: true
I20260709 13:16:56.279579 17513 raft_consensus.cc:493] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.280023 17467 raft_consensus.cc:2468] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 0.
I20260709 13:16:56.280028 17513 raft_consensus.cc:515] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.280998 17509 tablet_bootstrap.cc:654] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.281183 17393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a5d2fb3f9ed4769a772ac524a4713e6" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" is_pre_election: true
I20260709 13:16:56.281426 17357 leader_election.cc:304] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:16:56.281816 17393 raft_consensus.cc:2468] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 0.
I20260709 13:16:56.283181 17532 raft_consensus.cc:2804] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:56.283594 17393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7153ce9502848edac1c5ef91ef65715" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" is_pre_election: true
I20260709 13:16:56.283658 17433 leader_election.cc:304] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:16:56.284373 17393 raft_consensus.cc:2468] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0136a742d5fe41c0bfae2764293ee082 in term 0.
I20260709 13:16:56.285625 17284 leader_election.cc:304] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, 4ca45d6eb7e24e6cb99de9028e1142b8; no voters: 
I20260709 13:16:56.285990 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a5d2fb3f9ed4769a772ac524a4713e6" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082" is_pre_election: true
I20260709 13:16:56.286695 17318 raft_consensus.cc:2468] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 0.
I20260709 13:16:56.283627 17532 raft_consensus.cc:493] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.287207 17532 raft_consensus.cc:3060] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.287693 17317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6dc3aeee5c486b83aa69bac75c7868" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082" is_pre_election: true
I20260709 13:16:56.288301 17317 raft_consensus.cc:2468] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 0.
I20260709 13:16:56.289048 17534 raft_consensus.cc:493] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.289662 17534 raft_consensus.cc:515] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.291673 17534 leader_election.cc:290] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.282235 17513 leader_election.cc:290] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.293031 17513 raft_consensus.cc:2804] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:56.293365 17513 raft_consensus.cc:493] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.293371 17467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e989e4c8d2f4c3690126a1eca30ddfa" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388" is_pre_election: true
I20260709 13:16:56.293789 17317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e989e4c8d2f4c3690126a1eca30ddfa" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082" is_pre_election: true
I20260709 13:16:56.277280 17514 leader_election.cc:290] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
W20260709 13:16:56.294912 17357 leader_election.cc:343] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:56.295198 17514 raft_consensus.cc:2804] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:56.295537 17514 raft_consensus.cc:493] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.293840 17513 raft_consensus.cc:3060] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.295913 17514 raft_consensus.cc:3060] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:56.295653 17357 leader_election.cc:343] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:56.298873 17357 leader_election.cc:304] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8; no voters: 0136a742d5fe41c0bfae2764293ee082, baa0fccfc08d477ba279ad0968edb388
I20260709 13:16:56.300073 17507 ts_tablet_manager.cc:1434] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.001s
I20260709 13:16:56.300117 17534 raft_consensus.cc:2749] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:56.304066 17509 tablet_bootstrap.cc:492] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: No bootstrap required, opened a new log
I20260709 13:16:56.304539 17509 ts_tablet_manager.cc:1403] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.000s
I20260709 13:16:56.307080 17513 raft_consensus.cc:515] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.307111 17509 raft_consensus.cc:359] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.308023 17509 raft_consensus.cc:385] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.308310 17509 raft_consensus.cc:740] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.309213 17507 tablet_bootstrap.cc:492] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388: Bootstrap starting.
I20260709 13:16:56.308970 17509 consensus_queue.cc:260] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.309633 17513 leader_election.cc:290] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 election: Requested vote from peers 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.311663 17509 ts_tablet_manager.cc:1434] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:16:56.313884 17514 raft_consensus.cc:515] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.309551 17532 raft_consensus.cc:515] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.319015 17514 leader_election.cc:290] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 1 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
I20260709 13:16:56.322368 17467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7153ce9502848edac1c5ef91ef65715" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388" is_pre_election: true
I20260709 13:16:56.323052 17467 raft_consensus.cc:2468] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0136a742d5fe41c0bfae2764293ee082 in term 0.
I20260709 13:16:56.323421 17393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7153ce9502848edac1c5ef91ef65715" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
I20260709 13:16:56.322273 17317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a5d2fb3f9ed4769a772ac524a4713e6" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:16:56.324218 17393 raft_consensus.cc:3060] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.324316 17317 raft_consensus.cc:3060] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.326097 17507 tablet_bootstrap.cc:654] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.330084 17532 leader_election.cc:290] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.331153 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6dc3aeee5c486b83aa69bac75c7868" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:16:56.331598 17318 raft_consensus.cc:3060] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.331467 17467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6dc3aeee5c486b83aa69bac75c7868" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
I20260709 13:16:56.332383 17467 raft_consensus.cc:3060] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.336524 17393 raft_consensus.cc:2468] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0136a742d5fe41c0bfae2764293ee082 in term 1.
I20260709 13:16:56.337766 17556 raft_consensus.cc:493] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.338168 17284 leader_election.cc:304] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, 4ca45d6eb7e24e6cb99de9028e1142b8; no voters: 
I20260709 13:16:56.338284 17556 raft_consensus.cc:515] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.340205 17317 raft_consensus.cc:2468] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 1.
I20260709 13:16:56.340257 17513 raft_consensus.cc:2804] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:56.337710 17318 raft_consensus.cc:2468] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 1.
I20260709 13:16:56.341528 17507 tablet_bootstrap.cc:492] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388: No bootstrap required, opened a new log
I20260709 13:16:56.341910 17556 leader_election.cc:290] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.342010 17507 ts_tablet_manager.cc:1403] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.011s
I20260709 13:16:56.342177 17393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177781087d244f0a8d0edd7e6df0160b" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" is_pre_election: true
I20260709 13:16:56.342857 17393 raft_consensus.cc:2468] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0136a742d5fe41c0bfae2764293ee082 in term 0.
I20260709 13:16:56.343812 17357 leader_election.cc:304] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, 4ca45d6eb7e24e6cb99de9028e1142b8; no voters: 
I20260709 13:16:56.344354 17467 raft_consensus.cc:2468] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 1.
I20260709 13:16:56.344480 17284 leader_election.cc:304] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, 4ca45d6eb7e24e6cb99de9028e1142b8; no voters: 
I20260709 13:16:56.344674 17532 raft_consensus.cc:2804] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:56.345129 17513 raft_consensus.cc:697] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 1 LEADER]: Becoming Leader. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Running, Role: LEADER
I20260709 13:16:56.344955 17507 raft_consensus.cc:359] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.345570 17532 raft_consensus.cc:697] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Becoming Leader. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:16:56.345651 17507 raft_consensus.cc:385] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.345762 17558 raft_consensus.cc:2804] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:56.346267 17507 raft_consensus.cc:740] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.346041 17513 consensus_queue.cc:237] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.346554 17558 raft_consensus.cc:493] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.347116 17558 raft_consensus.cc:3060] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.347046 17507 consensus_queue.cc:260] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.347913 17532 consensus_queue.cc:237] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.349395 17507 ts_tablet_manager.cc:1434] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:16:56.350368 17507 tablet_bootstrap.cc:492] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388: Bootstrap starting.
I20260709 13:16:56.355623 17507 tablet_bootstrap.cc:654] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.356119 17558 raft_consensus.cc:515] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.359174 17558 leader_election.cc:290] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 election: Requested vote from peers 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.360416 17393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177781087d244f0a8d0edd7e6df0160b" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
I20260709 13:16:56.362414 17534 raft_consensus.cc:493] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.366758 17534 raft_consensus.cc:515] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.367225 17507 tablet_bootstrap.cc:492] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388: No bootstrap required, opened a new log
I20260709 13:16:56.367733 17507 ts_tablet_manager.cc:1403] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.004s
I20260709 13:16:56.368645 17555 raft_consensus.cc:493] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.368772 17534 leader_election.cc:290] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.369192 17555 raft_consensus.cc:515] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.369683 17465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177781087d244f0a8d0edd7e6df0160b" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
I20260709 13:16:56.366189 17466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177781087d244f0a8d0edd7e6df0160b" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388" is_pre_election: true
I20260709 13:16:56.370397 17507 raft_consensus.cc:359] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.371416 17507 raft_consensus.cc:385] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.371013 17393 raft_consensus.cc:3060] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.371913 17507 raft_consensus.cc:740] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.371711 17467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7153ce9502848edac1c5ef91ef65715" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
I20260709 13:16:56.372272 17392 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a5d2fb3f9ed4769a772ac524a4713e6" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
I20260709 13:16:56.372664 17467 raft_consensus.cc:3060] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.372838 17507 consensus_queue.cc:260] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.374768 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2423d9c513494d918102933924b43074" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082" is_pre_election: true
I20260709 13:16:56.375313 17507 ts_tablet_manager.cc:1434] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:16:56.375464 17318 raft_consensus.cc:2468] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 0.
I20260709 13:16:56.377211 17464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2423d9c513494d918102933924b43074" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388" is_pre_election: true
I20260709 13:16:56.377878 17464 raft_consensus.cc:2468] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 0.
I20260709 13:16:56.379119 17466 raft_consensus.cc:2468] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0136a742d5fe41c0bfae2764293ee082 in term 0.
I20260709 13:16:56.379215 17357 leader_election.cc:304] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:16:56.372895 17392 raft_consensus.cc:3060] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.380295 17393 raft_consensus.cc:2468] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0136a742d5fe41c0bfae2764293ee082 in term 1.
I20260709 13:16:56.380631 17534 raft_consensus.cc:2804] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:56.381076 17534 raft_consensus.cc:493] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.381417 17534 raft_consensus.cc:3060] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.381704 17467 raft_consensus.cc:2468] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0136a742d5fe41c0bfae2764293ee082 in term 1.
W20260709 13:16:56.384214 17282 leader_election.cc:343] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:56.385406 17555 leader_election.cc:290] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
I20260709 13:16:56.386109 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4522d72f7948494a83af8aa9e4fbaddc" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082" is_pre_election: true
I20260709 13:16:56.386759 17318 raft_consensus.cc:2468] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 0.
I20260709 13:16:56.387445 17392 raft_consensus.cc:2468] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 1.
I20260709 13:16:56.388824 17391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4522d72f7948494a83af8aa9e4fbaddc" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" is_pre_election: true
I20260709 13:16:56.389736 17391 raft_consensus.cc:2468] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 0.
I20260709 13:16:56.390983 17433 leader_election.cc:304] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:16:56.390995 17284 leader_election.cc:304] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, 4ca45d6eb7e24e6cb99de9028e1142b8; no voters: baa0fccfc08d477ba279ad0968edb388
I20260709 13:16:56.394866 17558 raft_consensus.cc:2804] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:56.396148 17558 raft_consensus.cc:697] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 1 LEADER]: Becoming Leader. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Running, Role: LEADER
I20260709 13:16:56.396497 17555 raft_consensus.cc:2804] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:56.397313 17555 raft_consensus.cc:697] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 1 LEADER]: Becoming Leader. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Running, Role: LEADER
I20260709 13:16:56.397437 17558 consensus_queue.cc:237] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.393464 17431 leader_election.cc:304] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:16:56.398131 17555 consensus_queue.cc:237] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.398770 17514 raft_consensus.cc:2804] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:56.399134 17514 raft_consensus.cc:493] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.399458 17514 raft_consensus.cc:3060] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.403607 17534 raft_consensus.cc:515] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.406001 17534 leader_election.cc:290] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.410367 17216 catalog_manager.cc:5697] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "0136a742d5fe41c0bfae2764293ee082" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:56.411089 17317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2423d9c513494d918102933924b43074" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:16:56.410804 17214 catalog_manager.cc:5697] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:56.411767 17317 raft_consensus.cc:3060] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.412389 17465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2423d9c513494d918102933924b43074" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
I20260709 13:16:56.412976 17465 raft_consensus.cc:3060] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.415088 17514 raft_consensus.cc:515] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.417193 17514 leader_election.cc:290] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 1 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
I20260709 13:16:56.419587 17318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4522d72f7948494a83af8aa9e4fbaddc" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:16:56.420159 17318 raft_consensus.cc:3060] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.421612 17391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4522d72f7948494a83af8aa9e4fbaddc" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
I20260709 13:16:56.422194 17391 raft_consensus.cc:3060] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.424364 17317 raft_consensus.cc:2468] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 1.
I20260709 13:16:56.425545 17357 leader_election.cc:304] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, 4ca45d6eb7e24e6cb99de9028e1142b8; no voters: 
I20260709 13:16:56.426312 17534 raft_consensus.cc:2804] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:56.426821 17534 raft_consensus.cc:697] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Becoming Leader. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:16:56.427547 17534 consensus_queue.cc:237] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.430444 17465 raft_consensus.cc:2468] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 1.
I20260709 13:16:56.436757 17318 raft_consensus.cc:2468] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 1.
I20260709 13:16:56.437848 17431 leader_election.cc:304] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:16:56.438848 17555 raft_consensus.cc:2804] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:56.441303 17555 raft_consensus.cc:697] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [term 1 LEADER]: Becoming Leader. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Running, Role: LEADER
I20260709 13:16:56.438649 17391 raft_consensus.cc:2468] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 1.
I20260709 13:16:56.442117 17555 consensus_queue.cc:237] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.452497 17216 catalog_manager.cc:5697] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:56.453687 17215 catalog_manager.cc:5697] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 reported cstate change: term changed from 0 to 1, leader changed from <none> to baa0fccfc08d477ba279ad0968edb388 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "baa0fccfc08d477ba279ad0968edb388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:56.453687 17214 catalog_manager.cc:5697] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "0136a742d5fe41c0bfae2764293ee082" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:56.470230 17215 catalog_manager.cc:5697] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 reported cstate change: term changed from 0 to 1, leader changed from <none> to baa0fccfc08d477ba279ad0968edb388 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "baa0fccfc08d477ba279ad0968edb388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:56.471707 17534 consensus_queue.cc:1048] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:56.481150 17318 raft_consensus.cc:3060] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.486155 17558 raft_consensus.cc:493] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.486215 17513 raft_consensus.cc:493] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.486860 17558 raft_consensus.cc:515] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.486960 17513 raft_consensus.cc:515] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.489034 17558 leader_election.cc:290] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.490103 17534 consensus_queue.cc:1048] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:56.491688 17467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388" is_pre_election: true
I20260709 13:16:56.491748 17393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e989e4c8d2f4c3690126a1eca30ddfa" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" is_pre_election: true
I20260709 13:16:56.492581 17467 raft_consensus.cc:2393] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0136a742d5fe41c0bfae2764293ee082 in current term 1: Already voted for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in this term.
I20260709 13:16:56.492502 17393 raft_consensus.cc:2468] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0136a742d5fe41c0bfae2764293ee082 in term 0.
I20260709 13:16:56.492846 17466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e989e4c8d2f4c3690126a1eca30ddfa" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388" is_pre_election: true
I20260709 13:16:56.493412 17391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" is_pre_election: true
I20260709 13:16:56.493814 17284 leader_election.cc:304] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, 4ca45d6eb7e24e6cb99de9028e1142b8; no voters: 
I20260709 13:16:56.494015 17513 leader_election.cc:290] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.494706 17513 raft_consensus.cc:2804] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:56.495530 17513 raft_consensus.cc:493] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.495867 17513 raft_consensus.cc:3060] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.493498 17466 raft_consensus.cc:2468] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0136a742d5fe41c0bfae2764293ee082 in term 0.
I20260709 13:16:56.497979 17284 leader_election.cc:304] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082; no voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388
I20260709 13:16:56.498805 17558 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.503381 17513 raft_consensus.cc:515] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.505791 17513 leader_election.cc:290] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 election: Requested vote from peers 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:56.507979 17558 raft_consensus.cc:2749] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:56.508865 17391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e989e4c8d2f4c3690126a1eca30ddfa" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
I20260709 13:16:56.509385 17391 raft_consensus.cc:3060] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.512431 17466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e989e4c8d2f4c3690126a1eca30ddfa" candidate_uuid: "0136a742d5fe41c0bfae2764293ee082" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
I20260709 13:16:56.513012 17466 raft_consensus.cc:3060] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.515029 17391 raft_consensus.cc:2468] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0136a742d5fe41c0bfae2764293ee082 in term 1.
I20260709 13:16:56.516114 17284 leader_election.cc:304] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, 4ca45d6eb7e24e6cb99de9028e1142b8; no voters: 
I20260709 13:16:56.516863 17558 raft_consensus.cc:2804] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:56.517333 17558 raft_consensus.cc:697] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 1 LEADER]: Becoming Leader. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Running, Role: LEADER
I20260709 13:16:56.518126 17558 consensus_queue.cc:237] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:56.520448 17466 raft_consensus.cc:2468] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0136a742d5fe41c0bfae2764293ee082 in term 1.
I20260709 13:16:56.524336 17534 consensus_queue.cc:1048] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:56.527356 17215 catalog_manager.cc:5697] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "0136a742d5fe41c0bfae2764293ee082" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:56.534093 17532 consensus_queue.cc:1048] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:56.581033 17532 consensus_queue.cc:1048] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:56.591084 17515 consensus_queue.cc:1048] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:56.596101  6937 ts_manager.cc:295] Set tserver state for 0136a742d5fe41c0bfae2764293ee082 to MAINTENANCE_MODE
I20260709 13:16:56.597029  6937 tablet_server.cc:179] TabletServer@127.6.198.65:0 shutting down...
I20260709 13:16:56.632683  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:16:56.633430  6937 tablet_replica.cc:333] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082: stopping tablet replica
I20260709 13:16:56.634276  6937 raft_consensus.cc:2243] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:56.634807  6937 raft_consensus.cc:2272] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:56.637056  6937 tablet_replica.cc:333] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: stopping tablet replica
I20260709 13:16:56.637590  6937 raft_consensus.cc:2243] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:56.638159  6937 pending_rounds.cc:70] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: Trying to abort 1 pending ops.
I20260709 13:16:56.638406  6937 pending_rounds.cc:77] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305637955054333952 op_type: NO_OP noop_request { }
I20260709 13:16:56.638708  6937 raft_consensus.cc:2889] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:16:56.638983  6937 raft_consensus.cc:2272] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:56.640987  6937 tablet_replica.cc:333] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082: stopping tablet replica
I20260709 13:16:56.641507  6937 raft_consensus.cc:2243] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:56.641903  6937 raft_consensus.cc:2272] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:56.644023  6937 tablet_replica.cc:333] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082: stopping tablet replica
I20260709 13:16:56.644546  6937 raft_consensus.cc:2243] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:56.644901  6937 raft_consensus.cc:2272] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:56.646561  6937 tablet_replica.cc:333] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: stopping tablet replica
I20260709 13:16:56.647048  6937 raft_consensus.cc:2243] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:56.647600  6937 pending_rounds.cc:70] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: Trying to abort 1 pending ops.
I20260709 13:16:56.647809  6937 pending_rounds.cc:77] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305637955667640320 op_type: NO_OP noop_request { }
I20260709 13:16:56.648072  6937 raft_consensus.cc:2889] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:16:56.648331  6937 raft_consensus.cc:2272] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:56.650451  6937 tablet_replica.cc:333] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082: stopping tablet replica
I20260709 13:16:56.650947  6937 raft_consensus.cc:2243] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:56.651314  6937 raft_consensus.cc:2272] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:56.653081  6937 tablet_replica.cc:333] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082: stopping tablet replica
I20260709 13:16:56.653577  6937 raft_consensus.cc:2243] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:56.653957  6937 raft_consensus.cc:2272] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:56.656123  6937 tablet_replica.cc:333] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: stopping tablet replica
I20260709 13:16:56.656611  6937 raft_consensus.cc:2243] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:16:56.657160  6937 pending_rounds.cc:70] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: Trying to abort 1 pending ops.
I20260709 13:16:56.657364  6937 pending_rounds.cc:77] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305637955176062976 op_type: NO_OP noop_request { }
I20260709 13:16:56.657631  6937 raft_consensus.cc:2889] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:16:56.657895  6937 raft_consensus.cc:2272] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:56.659833  6937 tablet_replica.cc:333] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082: stopping tablet replica
I20260709 13:16:56.660312  6937 raft_consensus.cc:2243] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:56.660693  6937 raft_consensus.cc:2272] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:56.662324  6937 tablet_replica.cc:333] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082: stopping tablet replica
I20260709 13:16:56.662834  6937 raft_consensus.cc:2243] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:16:56.663146  6937 raft_consensus.cc:2272] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:16:56.691692  6937 tablet_server.cc:196] TabletServer@127.6.198.65:0 shutdown complete.
I20260709 13:16:56.869146 17555 consensus_queue.cc:1048] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:56.893527 17357 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111) [suppressed 275 similar messages]
I20260709 13:16:56.900492 17515 consensus_queue.cc:1048] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:56.901093 17431 consensus_peers.cc:597] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:56.901355 17357 consensus_peers.cc:597] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:56.904172 17515 consensus_queue.cc:1048] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:56.921144 17357 consensus_peers.cc:597] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:56.937057 17357 consensus_peers.cc:597] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:56.957052 17578 consensus_queue.cc:1048] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:56.969491 17431 consensus_peers.cc:597] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:56.973636 17264 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:16:57.100883 17357 consensus_peers.cc:597] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:57.170195 17357 consensus_peers.cc:597] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:57.974653 17264 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:16:58.066897 17534 raft_consensus.cc:493] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.067353 17534 raft_consensus.cc:515] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:58.068815 17534 leader_election.cc:290] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:58.070180 17466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177781087d244f0a8d0edd7e6df0160b" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388" is_pre_election: true
I20260709 13:16:58.070917 17466 raft_consensus.cc:2468] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 0.
I20260709 13:16:58.071942 17357 leader_election.cc:304] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:16:58.072654 17534 raft_consensus.cc:2804] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:16:58.072715 17357 leader_election.cc:336] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111)
I20260709 13:16:58.073066 17534 raft_consensus.cc:493] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.073432 17534 raft_consensus.cc:3060] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.078763 17534 raft_consensus.cc:515] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:58.080509 17534 leader_election.cc:290] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 2 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:58.081866 17466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177781087d244f0a8d0edd7e6df0160b" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
I20260709 13:16:58.082469 17466 raft_consensus.cc:3060] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 0 FOLLOWER]: Advancing to term 2
W20260709 13:16:58.084071 17357 leader_election.cc:336] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111)
I20260709 13:16:58.087678 17466 raft_consensus.cc:2468] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 2.
I20260709 13:16:58.088464 17357 leader_election.cc:304] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 0136a742d5fe41c0bfae2764293ee082
I20260709 13:16:58.088991 17534 raft_consensus.cc:2804] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:58.089370 17534 raft_consensus.cc:697] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 LEADER]: Becoming Leader. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:16:58.090116 17534 consensus_queue.cc:237] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:58.095981 17216 catalog_manager.cc:5697] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 reported cstate change: term changed from 1 to 2, leader changed from 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65) to 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.317970 17514 raft_consensus.cc:493] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.318459 17514 raft_consensus.cc:515] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:58.320217 17514 leader_election.cc:290] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
I20260709 13:16:58.321345 17391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e989e4c8d2f4c3690126a1eca30ddfa" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" is_pre_election: true
I20260709 13:16:58.322017 17391 raft_consensus.cc:2468] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 1.
I20260709 13:16:58.323099 17433 leader_election.cc:304] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 
W20260709 13:16:58.323637 17431 leader_election.cc:336] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111)
I20260709 13:16:58.323741 17514 raft_consensus.cc:2804] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:16:58.324154 17514 raft_consensus.cc:493] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.324409 17514 raft_consensus.cc:3060] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.329114 17514 raft_consensus.cc:515] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:58.330601 17514 leader_election.cc:290] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 2 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
I20260709 13:16:58.331620 17391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e989e4c8d2f4c3690126a1eca30ddfa" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
I20260709 13:16:58.332199 17391 raft_consensus.cc:3060] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:58.333873 17431 leader_election.cc:336] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111)
I20260709 13:16:58.337113 17391 raft_consensus.cc:2468] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 2.
I20260709 13:16:58.338070 17433 leader_election.cc:304] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 0136a742d5fe41c0bfae2764293ee082
I20260709 13:16:58.338697 17514 raft_consensus.cc:2804] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:58.339505 17514 raft_consensus.cc:697] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Becoming Leader. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Running, Role: LEADER
I20260709 13:16:58.340258 17514 consensus_queue.cc:237] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:58.347057 17216 catalog_manager.cc:5697] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 reported cstate change: term changed from 1 to 2, leader changed from 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65) to baa0fccfc08d477ba279ad0968edb388 (127.6.198.67). New cstate: current_term: 2 leader_uuid: "baa0fccfc08d477ba279ad0968edb388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: HEALTHY } } }
W20260709 13:16:58.496037 17357 consensus_peers.cc:597] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:58.576800 17534 raft_consensus.cc:493] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.577212 17534 raft_consensus.cc:515] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:58.578927 17534 leader_election.cc:290] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:58.580343 17466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7153ce9502848edac1c5ef91ef65715" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388" is_pre_election: true
I20260709 13:16:58.581025 17466 raft_consensus.cc:2468] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 1.
I20260709 13:16:58.582260 17357 leader_election.cc:304] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:16:58.583065 17534 raft_consensus.cc:2804] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:16:58.583370 17357 leader_election.cc:336] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111)
I20260709 13:16:58.583479 17534 raft_consensus.cc:493] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.583925 17534 raft_consensus.cc:3060] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.589514 17534 raft_consensus.cc:515] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:58.591200 17534 leader_election.cc:290] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 2 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:16:58.592587 17466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7153ce9502848edac1c5ef91ef65715" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
I20260709 13:16:58.593242 17466 raft_consensus.cc:3060] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:16:58.594429 17357 leader_election.cc:336] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111)
I20260709 13:16:58.598699 17466 raft_consensus.cc:2468] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 2.
I20260709 13:16:58.599571 17357 leader_election.cc:304] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 0136a742d5fe41c0bfae2764293ee082
I20260709 13:16:58.600168 17534 raft_consensus.cc:2804] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:58.601035 17534 raft_consensus.cc:697] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 LEADER]: Becoming Leader. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:16:58.601804 17534 consensus_queue.cc:237] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:16:58.608115 17216 catalog_manager.cc:5697] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 reported cstate change: term changed from 1 to 2, leader changed from 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65) to 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.716234 17466 raft_consensus.cc:1275] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Refusing update from remote peer 4ca45d6eb7e24e6cb99de9028e1142b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:58.717288 17534 consensus_queue.cc:1048] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:58.821111 17391 raft_consensus.cc:1275] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:58.822180 17514 consensus_queue.cc:1048] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:58.836920 17431 consensus_peers.cc:597] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:58.975492 17264 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:16:59.048722 17466 raft_consensus.cc:1275] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Refusing update from remote peer 4ca45d6eb7e24e6cb99de9028e1142b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:59.049981 17534 consensus_queue.cc:1048] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:16:59.062964 17357 consensus_peers.cc:597] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:59.136054 17357 consensus_peers.cc:597] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:59.347851 17431 consensus_peers.cc:597] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:59.415560 17431 consensus_peers.cc:597] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:59.432703 17357 consensus_peers.cc:597] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:59.515353 17357 consensus_peers.cc:597] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:59.626787 17357 consensus_peers.cc:597] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:59.815495 17357 consensus_peers.cc:597] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:16:59.976449 17264 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:17:00.822839 17357 consensus_peers.cc:597] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:00.977417 17264 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:17:01.362607 17357 consensus_peers.cc:597] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:01.490306 17431 consensus_peers.cc:597] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:01.701035 17357 consensus_peers.cc:597] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:01.709689  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:01.720156 17606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:01.720206 17607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:01.723012  6937 server_base.cc:1061] running on GCE node
W20260709 13:17:01.723867 17610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:01.724949  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:01.725134  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:01.725268  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603021725257 us; error 0 us; skew 500 ppm
I20260709 13:17:01.725751  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:01.728384  6937 webserver.cc:533] Webserver started at http://127.6.198.65:46053/ using document root <none> and password file <none>
I20260709 13:17:01.728852  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:01.729007  6937 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:17:01.732766  6937 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:17:01.736603 17614 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:01.737396  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:17:01.737680  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-0-root
uuid: "0136a742d5fe41c0bfae2764293ee082"
format_stamp: "Formatted at 2026-07-09 13:16:55 on dist-test-slave-tqzw"
I20260709 13:17:01.738091  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602718985547-6937-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:01.753046  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:01.754242  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:01.755721  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:01.759864 17621 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:17:01.790998 17431 consensus_peers.cc:597] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:01.808647  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:17:01.808971  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.050s	user 0.003s	sys 0.000s
I20260709 13:17:01.809232  6937 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:17:01.815162 17621 tablet_bootstrap.cc:492] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:17:01.828073 17621 tablet_bootstrap.cc:492] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:01.828789 17621 tablet_bootstrap.cc:492] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap complete.
I20260709 13:17:01.829281 17621 ts_tablet_manager.cc:1403] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:17:01.831497 17621 raft_consensus.cc:359] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:01.832346 17621 raft_consensus.cc:740] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:17:01.833009 17621 consensus_queue.cc:260] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:01.835487 17621 ts_tablet_manager.cc:1434] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:17:01.836108 17621 tablet_bootstrap.cc:492] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:17:01.847627  6937 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:17:01.847873  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.039s	user 0.030s	sys 0.007s
I20260709 13:17:01.848068 17621 tablet_bootstrap.cc:492] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:17:01.848886 17621 tablet_bootstrap.cc:492] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: Bootstrap complete.
I20260709 13:17:01.849426 17621 ts_tablet_manager.cc:1403] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:17:01.852013 17621 raft_consensus.cc:359] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:01.852720 17621 raft_consensus.cc:740] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:17:01.853670 17621 consensus_queue.cc:260] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:01.855295 17621 ts_tablet_manager.cc:1434] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:17:01.855976 17621 tablet_bootstrap.cc:492] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:17:01.868564 17621 tablet_bootstrap.cc:492] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:01.869240 17621 tablet_bootstrap.cc:492] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap complete.
I20260709 13:17:01.869789 17621 ts_tablet_manager.cc:1403] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20260709 13:17:01.872520 17621 raft_consensus.cc:359] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:01.873281 17621 raft_consensus.cc:740] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:17:01.873951 17621 consensus_queue.cc:260] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:01.876092 17621 ts_tablet_manager.cc:1434] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:17:01.876946 17621 tablet_bootstrap.cc:492] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:17:01.889195 17621 tablet_bootstrap.cc:492] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:01.890075 17621 tablet_bootstrap.cc:492] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap complete.
I20260709 13:17:01.890697 17621 ts_tablet_manager.cc:1403] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 13:17:01.893859 17621 raft_consensus.cc:359] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:01.894570 17621 raft_consensus.cc:740] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:17:01.895274 17621 consensus_queue.cc:260] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:01.896713 17621 ts_tablet_manager.cc:1434] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:17:01.897339 17621 tablet_bootstrap.cc:492] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:17:01.910317  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.65:45147
I20260709 13:17:01.910467 17683 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.65:45147 every 8 connection(s)
I20260709 13:17:01.911468 17621 tablet_bootstrap.cc:492] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:01.912401 17621 tablet_bootstrap.cc:492] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap complete.
I20260709 13:17:01.913012 17621 ts_tablet_manager.cc:1403] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.004s
I20260709 13:17:01.916191 17621 raft_consensus.cc:359] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } }
I20260709 13:17:01.916736  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:01.916935 17621 raft_consensus.cc:740] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:17:01.917729 17621 consensus_queue.cc:260] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } }
I20260709 13:17:01.919955 17621 ts_tablet_manager.cc:1434] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:17:01.920835 17621 tablet_bootstrap.cc:492] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:17:01.942905 17621 tablet_bootstrap.cc:492] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:17:01.943185 17684 heartbeater.cc:344] Connected to a master server at 127.6.198.126:35929
I20260709 13:17:01.943755 17684 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:01.943943 17621 tablet_bootstrap.cc:492] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: Bootstrap complete.
I20260709 13:17:01.944661 17621 ts_tablet_manager.cc:1403] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.012s
I20260709 13:17:01.944737 17684 heartbeater.cc:507] Master 127.6.198.126:35929 requested a full tablet report, sending...
I20260709 13:17:01.947659 17621 raft_consensus.cc:359] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:01.948405 17621 raft_consensus.cc:740] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:17:01.949265 17621 consensus_queue.cc:260] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:01.950328 17393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c6dc3aeee5c486b83aa69bac75c7868"
dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
mode: GRACEFUL
new_leader_uuid: "baa0fccfc08d477ba279ad0968edb388"
 from {username='slave'} at 127.0.0.1:38226
I20260709 13:17:01.951089 17621 ts_tablet_manager.cc:1434] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:17:01.951139 17393 raft_consensus.cc:606] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Received request to transfer leadership to TS baa0fccfc08d477ba279ad0968edb388
I20260709 13:17:01.952033 17621 tablet_bootstrap.cc:492] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:17:01.951879 17216 ts_manager.cc:194] Re-registered known tserver with Master: 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147)
I20260709 13:17:01.954660 17393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75f1eba07e9a41b9a1a5e4b47fb76d5e"
dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
mode: GRACEFUL
new_leader_uuid: "baa0fccfc08d477ba279ad0968edb388"
 from {username='slave'} at 127.0.0.1:38226
I20260709 13:17:01.955282 17393 raft_consensus.cc:606] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Received request to transfer leadership to TS baa0fccfc08d477ba279ad0968edb388
I20260709 13:17:01.957996 17393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846"
dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
mode: GRACEFUL
new_leader_uuid: "baa0fccfc08d477ba279ad0968edb388"
 from {username='slave'} at 127.0.0.1:38226
I20260709 13:17:01.958621 17393 raft_consensus.cc:606] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Received request to transfer leadership to TS baa0fccfc08d477ba279ad0968edb388
I20260709 13:17:01.960065  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:17:01.961068  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:01.961876  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:01.963948 17216 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49492
I20260709 13:17:01.966822 17621 tablet_bootstrap.cc:492] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:01.967545 17684 heartbeater.cc:499] Master 127.6.198.126:35929 was elected leader, sending a full tablet report...
I20260709 13:17:01.967731 17621 tablet_bootstrap.cc:492] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082: Bootstrap complete.
I20260709 13:17:01.968372 17621 ts_tablet_manager.cc:1403] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.007s
I20260709 13:17:01.970927 17621 raft_consensus.cc:359] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:01.971603 17621 raft_consensus.cc:740] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:17:01.972260 17621 consensus_queue.cc:260] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:01.973565 17621 ts_tablet_manager.cc:1434] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:17:01.974171 17621 tablet_bootstrap.cc:492] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
W20260709 13:17:01.978672 17264 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:01.985664 17621 tablet_bootstrap.cc:492] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:17:01.986428 17621 tablet_bootstrap.cc:492] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap complete.
I20260709 13:17:01.986944 17621 ts_tablet_manager.cc:1403] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.013s	user 0.012s	sys 0.000s
I20260709 13:17:01.989013 17621 raft_consensus.cc:359] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:01.989702 17621 raft_consensus.cc:740] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:17:01.990438 17621 consensus_queue.cc:260] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:01.992143 17621 ts_tablet_manager.cc:1434] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.005s	user 0.001s	sys 0.003s
I20260709 13:17:01.992929 17621 tablet_bootstrap.cc:492] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:17:02.003190 17621 tablet_bootstrap.cc:492] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:02.004022 17621 tablet_bootstrap.cc:492] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082: Bootstrap complete.
I20260709 13:17:02.004683 17621 ts_tablet_manager.cc:1403] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 13:17:02.006470 17604 raft_consensus.cc:993] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8: : Instructing follower baa0fccfc08d477ba279ad0968edb388 to start an election
I20260709 13:17:02.007093 17605 raft_consensus.cc:1081] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Signalling peer baa0fccfc08d477ba279ad0968edb388 to start an election
I20260709 13:17:02.007200 17621 raft_consensus.cc:359] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:02.007961 17621 raft_consensus.cc:740] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.008599 17621 consensus_queue.cc:260] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:02.008793 17466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c6dc3aeee5c486b83aa69bac75c7868"
dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
 from {username='slave'} at 127.0.0.1:34472
I20260709 13:17:02.009426 17466 raft_consensus.cc:493] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:02.009811 17466 raft_consensus.cc:3060] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.010473 17621 ts_tablet_manager.cc:1434] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:17:02.011298 17621 tablet_bootstrap.cc:492] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap starting.
I20260709 13:17:02.018184 17466 raft_consensus.cc:515] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:02.021257 17655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6dc3aeee5c486b83aa69bac75c7868" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:17:02.021370 17393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c6dc3aeee5c486b83aa69bac75c7868" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
I20260709 13:17:02.021984 17655 raft_consensus.cc:3060] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.022004 17393 raft_consensus.cc:3055] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:02.022624 17393 raft_consensus.cc:740] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:17:02.023671 17393 consensus_queue.cc:260] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:02.025038 17393 raft_consensus.cc:3060] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.022485 17466 leader_election.cc:290] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 2 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
I20260709 13:17:02.031543 17621 tablet_bootstrap.cc:492] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:02.032227 17655 raft_consensus.cc:2468] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 2.
I20260709 13:17:02.032505 17621 tablet_bootstrap.cc:492] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082: Bootstrap complete.
I20260709 13:17:02.033304 17621 ts_tablet_manager.cc:1403] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.008s
I20260709 13:17:02.033571 17431 leader_election.cc:304] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:17:02.034554 17599 raft_consensus.cc:2804] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:02.035787 17599 raft_consensus.cc:697] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Becoming Leader. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Running, Role: LEADER
I20260709 13:17:02.036664 17621 raft_consensus.cc:359] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:02.036959 17599 consensus_queue.cc:237] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:02.037633 17621 raft_consensus.cc:740] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0136a742d5fe41c0bfae2764293ee082, State: Initialized, Role: FOLLOWER
I20260709 13:17:02.038666 17621 consensus_queue.cc:260] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:02.041294 17621 ts_tablet_manager.cc:1434] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:02.041435 17393 raft_consensus.cc:2468] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 2.
I20260709 13:17:02.047821 17215 catalog_manager.cc:5697] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 reported cstate change: term changed from 1 to 2, leader changed from 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66) to baa0fccfc08d477ba279ad0968edb388 (127.6.198.67). New cstate: current_term: 2 leader_uuid: "baa0fccfc08d477ba279ad0968edb388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:02.137138 17604 raft_consensus.cc:993] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8: : Instructing follower baa0fccfc08d477ba279ad0968edb388 to start an election
I20260709 13:17:02.137564 17605 raft_consensus.cc:1081] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Signalling peer baa0fccfc08d477ba279ad0968edb388 to start an election
I20260709 13:17:02.138963 17466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846"
dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
 from {username='slave'} at 127.0.0.1:34472
I20260709 13:17:02.139436 17466 raft_consensus.cc:493] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:02.139743 17466 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.144953 17466 raft_consensus.cc:515] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:02.146412 17466 leader_election.cc:290] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 2 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
I20260709 13:17:02.147264 17657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:17:02.147395 17393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
I20260709 13:17:02.147903 17657 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.147979 17393 raft_consensus.cc:3055] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:02.148372 17393 raft_consensus.cc:740] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:17:02.149219 17393 consensus_queue.cc:260] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:02.150154 17393 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.153801 17657 raft_consensus.cc:2468] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 2.
I20260709 13:17:02.154789 17431 leader_election.cc:304] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:17:02.155426 17393 raft_consensus.cc:2468] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 2.
I20260709 13:17:02.155596 17698 raft_consensus.cc:2804] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:02.156086 17698 raft_consensus.cc:697] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Becoming Leader. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Running, Role: LEADER
I20260709 13:17:02.157012 17698 consensus_queue.cc:237] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:02.160790 17657 raft_consensus.cc:3060] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.164882 17215 catalog_manager.cc:5697] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 reported cstate change: term changed from 1 to 2, leader changed from 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66) to baa0fccfc08d477ba279ad0968edb388 (127.6.198.67). New cstate: current_term: 2 leader_uuid: "baa0fccfc08d477ba279ad0968edb388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:02.166158 17657 pending_rounds.cc:85] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: Aborting all ops after (but not including) 0
I20260709 13:17:02.166486 17657 pending_rounds.cc:107] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:17:02.166787 17657 raft_consensus.cc:2889] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:02.314170 17604 raft_consensus.cc:993] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8: : Instructing follower baa0fccfc08d477ba279ad0968edb388 to start an election
I20260709 13:17:02.314693 17605 raft_consensus.cc:1081] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Signalling peer baa0fccfc08d477ba279ad0968edb388 to start an election
I20260709 13:17:02.316095 17466 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75f1eba07e9a41b9a1a5e4b47fb76d5e"
dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
 from {username='slave'} at 127.0.0.1:34472
I20260709 13:17:02.316576 17466 raft_consensus.cc:493] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:02.316905 17466 raft_consensus.cc:3060] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.321899 17466 raft_consensus.cc:515] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:02.323972 17466 leader_election.cc:290] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 2 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
I20260709 13:17:02.325166 17393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f1eba07e9a41b9a1a5e4b47fb76d5e" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
I20260709 13:17:02.325875 17393 raft_consensus.cc:3055] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:02.326181 17393 raft_consensus.cc:740] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:17:02.327065 17393 consensus_queue.cc:260] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:02.328460 17393 raft_consensus.cc:3060] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.328922 17657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75f1eba07e9a41b9a1a5e4b47fb76d5e" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:17:02.329609 17657 raft_consensus.cc:3060] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.336047 17393 raft_consensus.cc:2468] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 2.
I20260709 13:17:02.336627 17657 raft_consensus.cc:2468] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 2.
I20260709 13:17:02.337319 17433 leader_election.cc:304] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ca45d6eb7e24e6cb99de9028e1142b8, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:17:02.338253 17698 raft_consensus.cc:2804] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:02.338737 17698 raft_consensus.cc:697] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Becoming Leader. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Running, Role: LEADER
I20260709 13:17:02.339722 17698 consensus_queue.cc:237] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:02.347254 17216 catalog_manager.cc:5697] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 reported cstate change: term changed from 1 to 2, leader changed from 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66) to baa0fccfc08d477ba279ad0968edb388 (127.6.198.67). New cstate: current_term: 2 leader_uuid: "baa0fccfc08d477ba279ad0968edb388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:02.353888 17657 raft_consensus.cc:3060] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.358754 17657 pending_rounds.cc:85] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: Aborting all ops after (but not including) 0
I20260709 13:17:02.358943 17657 pending_rounds.cc:107] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:17:02.359170 17657 raft_consensus.cc:2889] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:02.491214 17657 raft_consensus.cc:3060] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:02.496160 17657 pending_rounds.cc:85] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: Aborting all ops after (but not including) 0
I20260709 13:17:02.496403 17657 pending_rounds.cc:107] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:17:02.496629 17657 raft_consensus.cc:2889] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:02.502830 17657 raft_consensus.cc:1275] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:02.504055 17710 consensus_queue.cc:1048] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:02.514892 17393 raft_consensus.cc:1275] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:02.516371 17599 consensus_queue.cc:1048] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:02.598155 17393 raft_consensus.cc:1275] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:02.599514 17599 consensus_queue.cc:1048] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:02.608438 17657 raft_consensus.cc:1275] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:02.610492 17698 consensus_queue.cc:1048] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:02.732501 17657 raft_consensus.cc:1275] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:02.733680 17710 consensus_queue.cc:1048] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:02.744266 17393 raft_consensus.cc:1275] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:02.745754 17710 consensus_queue.cc:1048] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:02.970435  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:02.979748 17264 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:02.986665 17466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a5d2fb3f9ed4769a772ac524a4713e6"
dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
mode: GRACEFUL
new_leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
 from {username='slave'} at 127.0.0.1:42806
I20260709 13:17:02.987232 17466 raft_consensus.cc:606] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 1 LEADER]: Received request to transfer leadership to TS 4ca45d6eb7e24e6cb99de9028e1142b8
I20260709 13:17:02.989444 17466 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846"
dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
mode: GRACEFUL
new_leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
 from {username='slave'} at 127.0.0.1:42806
I20260709 13:17:02.989848 17466 raft_consensus.cc:606] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Received request to transfer leadership to TS 4ca45d6eb7e24e6cb99de9028e1142b8
I20260709 13:17:02.990828  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:02.991266  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:02.991807  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:03.091557 17720 raft_consensus.cc:993] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388: : Instructing follower 4ca45d6eb7e24e6cb99de9028e1142b8 to start an election
I20260709 13:17:03.092054 17710 raft_consensus.cc:1081] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Signalling peer 4ca45d6eb7e24e6cb99de9028e1142b8 to start an election
I20260709 13:17:03.093344 17393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846"
dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
 from {username='slave'} at 127.0.0.1:60804
I20260709 13:17:03.093832 17393 raft_consensus.cc:493] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:03.094134 17393 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.101203 17393 raft_consensus.cc:515] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:03.103181 17393 leader_election.cc:290] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 3 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:17:03.104352 17466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
I20260709 13:17:03.104977 17466 raft_consensus.cc:3055] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:03.105301 17466 raft_consensus.cc:740] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Running, Role: LEADER
I20260709 13:17:03.105574 17657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:17:03.106695 17657 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.106679 17466 consensus_queue.cc:260] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:03.107946 17466 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.111810 17657 raft_consensus.cc:2468] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 3.
I20260709 13:17:03.111995 17466 raft_consensus.cc:2468] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 3.
I20260709 13:17:03.112771 17357 leader_election.cc:304] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, 4ca45d6eb7e24e6cb99de9028e1142b8; no voters: 
I20260709 13:17:03.113353 17604 raft_consensus.cc:2804] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:03.113796 17604 raft_consensus.cc:697] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 LEADER]: Becoming Leader. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:17:03.114528 17604 consensus_queue.cc:237] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:03.119621 17215 catalog_manager.cc:5697] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 reported cstate change: term changed from 2 to 3, leader changed from baa0fccfc08d477ba279ad0968edb388 (127.6.198.67) to 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66). New cstate: current_term: 3 leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.133795 17710 raft_consensus.cc:993] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388: : Instructing follower 4ca45d6eb7e24e6cb99de9028e1142b8 to start an election
I20260709 13:17:03.134152 17599 raft_consensus.cc:1081] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 1 LEADER]: Signalling peer 4ca45d6eb7e24e6cb99de9028e1142b8 to start an election
I20260709 13:17:03.135445 17393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a5d2fb3f9ed4769a772ac524a4713e6"
dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
 from {username='slave'} at 127.0.0.1:60804
I20260709 13:17:03.135874 17393 raft_consensus.cc:493] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:03.136155 17393 raft_consensus.cc:3060] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.142884 17393 raft_consensus.cc:515] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:03.144801 17393 leader_election.cc:290] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 2 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:17:03.145387 17657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a5d2fb3f9ed4769a772ac524a4713e6" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:17:03.145617 17466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a5d2fb3f9ed4769a772ac524a4713e6" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
I20260709 13:17:03.146008 17657 raft_consensus.cc:3060] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.146183 17466 raft_consensus.cc:3055] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:03.146585 17466 raft_consensus.cc:740] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Running, Role: LEADER
I20260709 13:17:03.147228 17466 consensus_queue.cc:260] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:03.148157 17466 raft_consensus.cc:3060] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:03.150945 17657 raft_consensus.cc:2468] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 2.
I20260709 13:17:03.151868 17357 leader_election.cc:304] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, 4ca45d6eb7e24e6cb99de9028e1142b8; no voters: 
I20260709 13:17:03.152505 17604 raft_consensus.cc:2804] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:03.152748 17466 raft_consensus.cc:2468] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 2.
I20260709 13:17:03.153007 17604 raft_consensus.cc:697] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 LEADER]: Becoming Leader. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:17:03.154038 17604 consensus_queue.cc:237] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:03.161316 17215 catalog_manager.cc:5697] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 reported cstate change: term changed from 1 to 2, leader changed from baa0fccfc08d477ba279ad0968edb388 (127.6.198.67) to 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.644913 17657 raft_consensus.cc:1275] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Refusing update from remote peer 4ca45d6eb7e24e6cb99de9028e1142b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:03.646076 17604 consensus_queue.cc:1048] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:03.655441 17466 raft_consensus.cc:1275] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Refusing update from remote peer 4ca45d6eb7e24e6cb99de9028e1142b8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:03.659646 17604 consensus_queue.cc:1048] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:03.711925 17657 raft_consensus.cc:1275] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 3 FOLLOWER]: Refusing update from remote peer 4ca45d6eb7e24e6cb99de9028e1142b8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:03.713133 17732 consensus_queue.cc:1048] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:03.720988 17465 raft_consensus.cc:1275] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 3 FOLLOWER]: Refusing update from remote peer 4ca45d6eb7e24e6cb99de9028e1142b8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:03.723067 17716 consensus_queue.cc:1048] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:17:03.980742 17264 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:03.998870  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:04.006104 17393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "177781087d244f0a8d0edd7e6df0160b"
dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
mode: GRACEFUL
new_leader_uuid: "baa0fccfc08d477ba279ad0968edb388"
 from {username='slave'} at 127.0.0.1:38226
I20260709 13:17:04.006711 17393 raft_consensus.cc:606] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 LEADER]: Received request to transfer leadership to TS baa0fccfc08d477ba279ad0968edb388
I20260709 13:17:04.009521 17393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846"
dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
mode: GRACEFUL
new_leader_uuid: "baa0fccfc08d477ba279ad0968edb388"
 from {username='slave'} at 127.0.0.1:38226
I20260709 13:17:04.010020 17393 raft_consensus.cc:606] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 LEADER]: Received request to transfer leadership to TS baa0fccfc08d477ba279ad0968edb388
I20260709 13:17:04.011186  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:04.011806  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:04.012519  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:04.118050 17716 raft_consensus.cc:993] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8: : Instructing follower baa0fccfc08d477ba279ad0968edb388 to start an election
I20260709 13:17:04.118623 17732 raft_consensus.cc:1081] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 LEADER]: Signalling peer baa0fccfc08d477ba279ad0968edb388 to start an election
I20260709 13:17:04.120133 17465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "177781087d244f0a8d0edd7e6df0160b"
dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
 from {username='slave'} at 127.0.0.1:34472
I20260709 13:17:04.120735 17465 raft_consensus.cc:493] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:04.121037 17465 raft_consensus.cc:3060] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:04.127496 17465 raft_consensus.cc:515] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:04.129698 17465 leader_election.cc:290] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 3 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
I20260709 13:17:04.130573 17655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177781087d244f0a8d0edd7e6df0160b" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:17:04.131052 17393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177781087d244f0a8d0edd7e6df0160b" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
I20260709 13:17:04.131318 17655 raft_consensus.cc:3060] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:04.131698 17393 raft_consensus.cc:3055] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:04.132055 17393 raft_consensus.cc:740] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:17:04.132201 17732 raft_consensus.cc:993] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8: : Instructing follower baa0fccfc08d477ba279ad0968edb388 to start an election
I20260709 13:17:04.132699 17736 raft_consensus.cc:1081] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 LEADER]: Signalling peer baa0fccfc08d477ba279ad0968edb388 to start an election
I20260709 13:17:04.133131 17393 consensus_queue.cc:260] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:04.134747 17465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846"
dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
 from {username='slave'} at 127.0.0.1:34472
I20260709 13:17:04.134940 17393 raft_consensus.cc:3060] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:04.135419 17465 raft_consensus.cc:493] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:04.135767 17465 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:04.138903 17655 raft_consensus.cc:2468] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 3.
I20260709 13:17:04.141692 17431 leader_election.cc:304] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:17:04.142549 17599 raft_consensus.cc:2804] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:04.143110 17393 raft_consensus.cc:2468] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 3.
I20260709 13:17:04.143023 17465 raft_consensus.cc:515] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:04.145617 17599 raft_consensus.cc:697] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 3 LEADER]: Becoming Leader. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Running, Role: LEADER
I20260709 13:17:04.146765 17599 consensus_queue.cc:237] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:04.147959 17393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
I20260709 13:17:04.148761 17393 raft_consensus.cc:3055] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:04.149117 17393 raft_consensus.cc:740] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:17:04.149217 17655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:17:04.150156 17393 consensus_queue.cc:260] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:04.151100 17655 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:04.151687 17393 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:04.152125 17465 leader_election.cc:290] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 4 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
I20260709 13:17:04.157608 17215 catalog_manager.cc:5697] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 reported cstate change: term changed from 2 to 3, leader changed from 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66) to baa0fccfc08d477ba279ad0968edb388 (127.6.198.67). New cstate: current_term: 3 leader_uuid: "baa0fccfc08d477ba279ad0968edb388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:04.159373 17655 raft_consensus.cc:2468] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 4.
I20260709 13:17:04.161108 17431 leader_election.cc:304] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:17:04.161839 17599 raft_consensus.cc:2804] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:04.162392 17599 raft_consensus.cc:697] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 4 LEADER]: Becoming Leader. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Running, Role: LEADER
I20260709 13:17:04.163241 17599 consensus_queue.cc:237] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:04.165557 17393 raft_consensus.cc:2468] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 4.
I20260709 13:17:04.182017 17215 catalog_manager.cc:5697] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 reported cstate change: term changed from 3 to 4, leader changed from 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66) to baa0fccfc08d477ba279ad0968edb388 (127.6.198.67). New cstate: current_term: 4 leader_uuid: "baa0fccfc08d477ba279ad0968edb388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:04.568243 17393 raft_consensus.cc:1275] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:04.569563 17599 consensus_queue.cc:1048] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:04.580157 17657 raft_consensus.cc:1275] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 3 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:04.582427 17599 consensus_queue.cc:1048] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:04.647392 17657 raft_consensus.cc:1275] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 4 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:04.648742 17599 consensus_queue.cc:1048] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:04.657805 17393 raft_consensus.cc:1275] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 4 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:04.660399 17599 consensus_queue.cc:1048] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:17:04.981738 17264 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:05.019512  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:05.026690 17465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e989e4c8d2f4c3690126a1eca30ddfa"
dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
mode: GRACEFUL
new_leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
 from {username='slave'} at 127.0.0.1:42806
I20260709 13:17:05.027175 17465 raft_consensus.cc:606] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Received request to transfer leadership to TS 4ca45d6eb7e24e6cb99de9028e1142b8
I20260709 13:17:05.030011 17465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846"
dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
mode: GRACEFUL
new_leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
 from {username='slave'} at 127.0.0.1:42806
I20260709 13:17:05.030464 17465 raft_consensus.cc:606] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 4 LEADER]: Received request to transfer leadership to TS 4ca45d6eb7e24e6cb99de9028e1142b8
I20260709 13:17:05.031538  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:05.031968  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:05.032455  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:05.098178 17599 raft_consensus.cc:993] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388: : Instructing follower 4ca45d6eb7e24e6cb99de9028e1142b8 to start an election
I20260709 13:17:05.098914 17738 raft_consensus.cc:1081] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 4 LEADER]: Signalling peer 4ca45d6eb7e24e6cb99de9028e1142b8 to start an election
I20260709 13:17:05.100481 17393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846"
dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
 from {username='slave'} at 127.0.0.1:60804
I20260709 13:17:05.100998 17393 raft_consensus.cc:493] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:05.101382 17393 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:05.107857 17393 raft_consensus.cc:515] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:05.109516 17393 leader_election.cc:290] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 5 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:17:05.110232 17655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:17:05.110968 17465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
I20260709 13:17:05.111105 17655 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:05.111835 17465 raft_consensus.cc:3055] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:05.112265 17465 raft_consensus.cc:740] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Running, Role: LEADER
I20260709 13:17:05.113534 17465 consensus_queue.cc:260] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:05.114773 17465 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:05.119177 17655 raft_consensus.cc:2468] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 5.
I20260709 13:17:05.119715 17465 raft_consensus.cc:2468] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 5.
I20260709 13:17:05.120260 17357 leader_election.cc:304] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, 4ca45d6eb7e24e6cb99de9028e1142b8; no voters: 
I20260709 13:17:05.120971 17716 raft_consensus.cc:2804] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:05.121461 17716 raft_consensus.cc:697] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 5 LEADER]: Becoming Leader. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:17:05.122200 17716 consensus_queue.cc:237] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:05.128545 17216 catalog_manager.cc:5697] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 reported cstate change: term changed from 4 to 5, leader changed from baa0fccfc08d477ba279ad0968edb388 (127.6.198.67) to 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66). New cstate: current_term: 5 leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:05.320369 17599 raft_consensus.cc:993] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388: : Instructing follower 4ca45d6eb7e24e6cb99de9028e1142b8 to start an election
I20260709 13:17:05.320868 17738 raft_consensus.cc:1081] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Signalling peer 4ca45d6eb7e24e6cb99de9028e1142b8 to start an election
I20260709 13:17:05.322427 17393 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e989e4c8d2f4c3690126a1eca30ddfa"
dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
 from {username='slave'} at 127.0.0.1:60804
I20260709 13:17:05.323019 17393 raft_consensus.cc:493] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:05.323309 17393 raft_consensus.cc:3060] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:05.328258 17393 raft_consensus.cc:515] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:05.330271 17393 leader_election.cc:290] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 3 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), baa0fccfc08d477ba279ad0968edb388 (127.6.198.67:42031)
I20260709 13:17:05.331344 17657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e989e4c8d2f4c3690126a1eca30ddfa" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:17:05.331687 17465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e989e4c8d2f4c3690126a1eca30ddfa" candidate_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
I20260709 13:17:05.332139 17657 raft_consensus.cc:3060] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:05.332410 17465 raft_consensus.cc:3055] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:05.332849 17465 raft_consensus.cc:740] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Running, Role: LEADER
I20260709 13:17:05.333738 17465 consensus_queue.cc:260] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:05.335136 17465 raft_consensus.cc:3060] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:05.339495 17657 raft_consensus.cc:2468] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 3.
I20260709 13:17:05.340801 17357 leader_election.cc:304] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, 4ca45d6eb7e24e6cb99de9028e1142b8; no voters: 
I20260709 13:17:05.341277 17465 raft_consensus.cc:2468] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ca45d6eb7e24e6cb99de9028e1142b8 in term 3.
I20260709 13:17:05.341619 17716 raft_consensus.cc:2804] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:05.342167 17716 raft_consensus.cc:697] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 LEADER]: Becoming Leader. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:17:05.343106 17716 consensus_queue.cc:237] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:05.349721 17216 catalog_manager.cc:5697] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 reported cstate change: term changed from 2 to 3, leader changed from baa0fccfc08d477ba279ad0968edb388 (127.6.198.67) to 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66). New cstate: current_term: 3 leader_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:05.640066 17657 raft_consensus.cc:1275] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 5 FOLLOWER]: Refusing update from remote peer 4ca45d6eb7e24e6cb99de9028e1142b8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:05.641403 17751 consensus_queue.cc:1048] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:05.650405 17465 raft_consensus.cc:1275] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 5 FOLLOWER]: Refusing update from remote peer 4ca45d6eb7e24e6cb99de9028e1142b8: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:05.651835 17751 consensus_queue.cc:1048] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:05.785617 17465 raft_consensus.cc:1275] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 3 FOLLOWER]: Refusing update from remote peer 4ca45d6eb7e24e6cb99de9028e1142b8: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:05.787099 17751 consensus_queue.cc:1048] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:05.795269 17657 raft_consensus.cc:1275] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 3 FOLLOWER]: Refusing update from remote peer 4ca45d6eb7e24e6cb99de9028e1142b8: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:05.796511 17761 consensus_queue.cc:1048] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:05.982679 17264 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:06.038928  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:06.046991 17393 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "04780cde858b4000bfd73d06bbb04158"
dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
mode: GRACEFUL
new_leader_uuid: "baa0fccfc08d477ba279ad0968edb388"
 from {username='slave'} at 127.0.0.1:38226
I20260709 13:17:06.047624 17393 raft_consensus.cc:606] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Received request to transfer leadership to TS baa0fccfc08d477ba279ad0968edb388
I20260709 13:17:06.050068 17391 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846"
dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
mode: GRACEFUL
new_leader_uuid: "baa0fccfc08d477ba279ad0968edb388"
 from {username='slave'} at 127.0.0.1:38226
I20260709 13:17:06.050571 17391 raft_consensus.cc:606] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 5 LEADER]: Received request to transfer leadership to TS baa0fccfc08d477ba279ad0968edb388
I20260709 13:17:06.051671  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:06.052145  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:06.052681  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:06.202193 17761 raft_consensus.cc:993] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8: : Instructing follower baa0fccfc08d477ba279ad0968edb388 to start an election
I20260709 13:17:06.202740 17716 raft_consensus.cc:1081] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 5 LEADER]: Signalling peer baa0fccfc08d477ba279ad0968edb388 to start an election
I20260709 13:17:06.204198 17465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846"
dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
 from {username='slave'} at 127.0.0.1:34472
I20260709 13:17:06.204672 17465 raft_consensus.cc:493] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:06.204983 17465 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:06.209967 17465 raft_consensus.cc:515] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:06.211546 17465 leader_election.cc:290] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 6 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
I20260709 13:17:06.212383 17657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:17:06.212584 17391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d293e2c8bea3443189ee867e1d5fa846" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: true dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
I20260709 13:17:06.212993 17657 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:06.213102 17391 raft_consensus.cc:3055] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:06.213512 17391 raft_consensus.cc:740] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:17:06.214303 17391 consensus_queue.cc:260] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:06.215610 17391 raft_consensus.cc:3060] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:06.219848 17657 raft_consensus.cc:2468] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 6.
I20260709 13:17:06.220973 17431 leader_election.cc:304] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:17:06.221728 17599 raft_consensus.cc:2804] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:06.222842 17391 raft_consensus.cc:2468] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 6.
I20260709 13:17:06.222939 17599 raft_consensus.cc:697] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 6 LEADER]: Becoming Leader. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Running, Role: LEADER
I20260709 13:17:06.224006 17599 consensus_queue.cc:237] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } }
I20260709 13:17:06.231649 17216 catalog_manager.cc:5697] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 reported cstate change: term changed from 5 to 6, leader changed from 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66) to baa0fccfc08d477ba279ad0968edb388 (127.6.198.67). New cstate: current_term: 6 leader_uuid: "baa0fccfc08d477ba279ad0968edb388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.344035 17761 raft_consensus.cc:993] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8: : Instructing follower baa0fccfc08d477ba279ad0968edb388 to start an election
I20260709 13:17:06.344527 17716 raft_consensus.cc:1081] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Signalling peer baa0fccfc08d477ba279ad0968edb388 to start an election
I20260709 13:17:06.346076 17465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "04780cde858b4000bfd73d06bbb04158"
dest_uuid: "baa0fccfc08d477ba279ad0968edb388"
 from {username='slave'} at 127.0.0.1:34472
I20260709 13:17:06.346557 17465 raft_consensus.cc:493] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:06.346879 17465 raft_consensus.cc:3060] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:06.351609 17465 raft_consensus.cc:515] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } }
I20260709 13:17:06.353237 17465 leader_election.cc:290] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 2 election: Requested vote from peers 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147), 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141)
I20260709 13:17:06.354003 17657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04780cde858b4000bfd73d06bbb04158" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0136a742d5fe41c0bfae2764293ee082"
I20260709 13:17:06.354369 17391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04780cde858b4000bfd73d06bbb04158" candidate_uuid: "baa0fccfc08d477ba279ad0968edb388" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8"
I20260709 13:17:06.354641 17657 raft_consensus.cc:3060] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:06.354951 17391 raft_consensus.cc:3055] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:06.355274 17391 raft_consensus.cc:740] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4ca45d6eb7e24e6cb99de9028e1142b8, State: Running, Role: LEADER
I20260709 13:17:06.356093 17391 consensus_queue.cc:260] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } }
I20260709 13:17:06.357419 17391 raft_consensus.cc:3060] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:06.361243 17657 raft_consensus.cc:2468] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 2.
I20260709 13:17:06.362504 17431 leader_election.cc:304] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0136a742d5fe41c0bfae2764293ee082, baa0fccfc08d477ba279ad0968edb388; no voters: 
I20260709 13:17:06.363250 17599 raft_consensus.cc:2804] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:06.363747 17599 raft_consensus.cc:697] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Becoming Leader. State: Replica: baa0fccfc08d477ba279ad0968edb388, State: Running, Role: LEADER
I20260709 13:17:06.364298 17391 raft_consensus.cc:2468] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baa0fccfc08d477ba279ad0968edb388 in term 2.
I20260709 13:17:06.364727 17599 consensus_queue.cc:237] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } }
I20260709 13:17:06.372014 17216 catalog_manager.cc:5697] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 reported cstate change: term changed from 1 to 2, leader changed from 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66) to baa0fccfc08d477ba279ad0968edb388 (127.6.198.67). New cstate: current_term: 2 leader_uuid: "baa0fccfc08d477ba279ad0968edb388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0fccfc08d477ba279ad0968edb388" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 42031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:06.659230 17657 raft_consensus.cc:1275] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 6 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:17:06.660400 17599 consensus_queue.cc:1048] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:17:06.671319 17391 raft_consensus.cc:1275] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 6 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:17:06.678083 17599 consensus_queue.cc:1048] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:17:06.832401 17657 raft_consensus.cc:1275] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:06.833526 17599 consensus_queue.cc:1048] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0136a742d5fe41c0bfae2764293ee082" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 45147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:06.841579 17391 raft_consensus.cc:1275] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Refusing update from remote peer baa0fccfc08d477ba279ad0968edb388: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:06.843011 17763 consensus_queue.cc:1048] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ca45d6eb7e24e6cb99de9028e1142b8" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:06.983520 17264 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:07.058467  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:07.086869  6937 tablet_server.cc:179] TabletServer@127.6.198.65:45147 shutting down...
I20260709 13:17:07.103544  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:07.104163  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:07.104753  6937 raft_consensus.cc:2243] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.105208  6937 raft_consensus.cc:2272] T 04780cde858b4000bfd73d06bbb04158 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.107630  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:07.108199  6937 raft_consensus.cc:2243] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.108594  6937 raft_consensus.cc:2272] T 2423d9c513494d918102933924b43074 P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.110291  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:07.110838  6937 raft_consensus.cc:2243] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.111217  6937 raft_consensus.cc:2272] T f7153ce9502848edac1c5ef91ef65715 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.112865  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:07.113349  6937 raft_consensus.cc:2243] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.113725  6937 raft_consensus.cc:2272] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.115396  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:07.115895  6937 raft_consensus.cc:2243] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.116288  6937 raft_consensus.cc:2272] T 6c6dc3aeee5c486b83aa69bac75c7868 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.117918  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:07.118453  6937 raft_consensus.cc:2243] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.118849  6937 raft_consensus.cc:2272] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 0136a742d5fe41c0bfae2764293ee082 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.120572  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:07.121234  6937 raft_consensus.cc:2243] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.121694  6937 raft_consensus.cc:2272] T 4522d72f7948494a83af8aa9e4fbaddc P 0136a742d5fe41c0bfae2764293ee082 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.124142  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:07.124778  6937 raft_consensus.cc:2243] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.125254  6937 raft_consensus.cc:2272] T 177781087d244f0a8d0edd7e6df0160b P 0136a742d5fe41c0bfae2764293ee082 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.127396  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:07.127807  6937 raft_consensus.cc:2243] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.128149  6937 raft_consensus.cc:2272] T 7e989e4c8d2f4c3690126a1eca30ddfa P 0136a742d5fe41c0bfae2764293ee082 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.129721  6937 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:07.130111  6937 raft_consensus.cc:2243] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.130568  6937 raft_consensus.cc:2272] T d293e2c8bea3443189ee867e1d5fa846 P 0136a742d5fe41c0bfae2764293ee082 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.159950  6937 tablet_server.cc:196] TabletServer@127.6.198.65:45147 shutdown complete.
I20260709 13:17:07.178079  6937 tablet_server.cc:179] TabletServer@127.6.198.66:0 shutting down...
I20260709 13:17:07.199368  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:07.199977  6937 tablet_replica.cc:333] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8: stopping tablet replica
I20260709 13:17:07.200650  6937 raft_consensus.cc:2243] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.201467  6937 raft_consensus.cc:2272] T 2423d9c513494d918102933924b43074 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.203678  6937 tablet_replica.cc:333] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8: stopping tablet replica
I20260709 13:17:07.204305  6937 raft_consensus.cc:2243] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.204699  6937 raft_consensus.cc:2272] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.206844  6937 tablet_replica.cc:333] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8: stopping tablet replica
I20260709 13:17:07.207370  6937 raft_consensus.cc:2243] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.208156  6937 raft_consensus.cc:2272] T f7153ce9502848edac1c5ef91ef65715 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.210094  6937 tablet_replica.cc:333] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8: stopping tablet replica
I20260709 13:17:07.210654  6937 raft_consensus.cc:2243] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.211063  6937 raft_consensus.cc:2272] T 6c6dc3aeee5c486b83aa69bac75c7868 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.212898  6937 tablet_replica.cc:333] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8: stopping tablet replica
I20260709 13:17:07.213398  6937 raft_consensus.cc:2243] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.213776  6937 raft_consensus.cc:2272] T 04780cde858b4000bfd73d06bbb04158 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.216092  6937 tablet_replica.cc:333] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8: stopping tablet replica
I20260709 13:17:07.216573  6937 raft_consensus.cc:2243] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.216971  6937 raft_consensus.cc:2272] T 177781087d244f0a8d0edd7e6df0160b P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.218865  6937 tablet_replica.cc:333] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8: stopping tablet replica
I20260709 13:17:07.219386  6937 raft_consensus.cc:2243] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.220093  6937 raft_consensus.cc:2272] T 8a5d2fb3f9ed4769a772ac524a4713e6 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.222108  6937 tablet_replica.cc:333] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8: stopping tablet replica
I20260709 13:17:07.222653  6937 raft_consensus.cc:2243] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.223076  6937 raft_consensus.cc:2272] T 4522d72f7948494a83af8aa9e4fbaddc P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.224934  6937 tablet_replica.cc:333] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8: stopping tablet replica
I20260709 13:17:07.225412  6937 raft_consensus.cc:2243] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.226096  6937 raft_consensus.cc:2272] T 7e989e4c8d2f4c3690126a1eca30ddfa P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.227933  6937 tablet_replica.cc:333] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8: stopping tablet replica
I20260709 13:17:07.228395  6937 raft_consensus.cc:2243] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.228777  6937 raft_consensus.cc:2272] T d293e2c8bea3443189ee867e1d5fa846 P 4ca45d6eb7e24e6cb99de9028e1142b8 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.249816  6937 tablet_server.cc:196] TabletServer@127.6.198.66:0 shutdown complete.
I20260709 13:17:07.274108  6937 tablet_server.cc:179] TabletServer@127.6.198.67:0 shutting down...
W20260709 13:17:07.283310 17431 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111) [suppressed 99 similar messages]
W20260709 13:17:07.286669 17431 consensus_peers.cc:597] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.298727 17433 consensus_peers.cc:597] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 -> Peer 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141): Couldn't send request to peer 4ca45d6eb7e24e6cb99de9028e1142b8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:40141: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:07.301920  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:07.302654  6937 tablet_replica.cc:333] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388: stopping tablet replica
I20260709 13:17:07.303431  6937 raft_consensus.cc:2243] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.304069  6937 raft_consensus.cc:2272] T 2423d9c513494d918102933924b43074 P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.306815  6937 tablet_replica.cc:333] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388: stopping tablet replica
I20260709 13:17:07.307382  6937 raft_consensus.cc:2243] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.308084  6937 raft_consensus.cc:2272] T 177781087d244f0a8d0edd7e6df0160b P baa0fccfc08d477ba279ad0968edb388 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.313158  6937 tablet_replica.cc:333] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388: stopping tablet replica
I20260709 13:17:07.314126  6937 raft_consensus.cc:2243] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.314668  6937 raft_consensus.cc:2272] T 7e989e4c8d2f4c3690126a1eca30ddfa P baa0fccfc08d477ba279ad0968edb388 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:07.315119 17433 consensus_peers.cc:597] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 -> Peer 4ca45d6eb7e24e6cb99de9028e1142b8 (127.6.198.66:40141): Couldn't send request to peer 4ca45d6eb7e24e6cb99de9028e1142b8. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:40141: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.315428 17431 consensus_peers.cc:597] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 -> Peer 0136a742d5fe41c0bfae2764293ee082 (127.6.198.65:45147): Couldn't send request to peer 0136a742d5fe41c0bfae2764293ee082. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:45147: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:07.316975  6937 tablet_replica.cc:333] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388: stopping tablet replica
I20260709 13:17:07.317536  6937 raft_consensus.cc:2243] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.318255  6937 raft_consensus.cc:2272] T 4522d72f7948494a83af8aa9e4fbaddc P baa0fccfc08d477ba279ad0968edb388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.320845  6937 tablet_replica.cc:333] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388: stopping tablet replica
I20260709 13:17:07.321424  6937 raft_consensus.cc:2243] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 6 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.322135  6937 raft_consensus.cc:2272] T d293e2c8bea3443189ee867e1d5fa846 P baa0fccfc08d477ba279ad0968edb388 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.324558  6937 tablet_replica.cc:333] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388: stopping tablet replica
I20260709 13:17:07.325049  6937 raft_consensus.cc:2243] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.325440  6937 raft_consensus.cc:2272] T 8a5d2fb3f9ed4769a772ac524a4713e6 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.327395  6937 tablet_replica.cc:333] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388: stopping tablet replica
I20260709 13:17:07.327944  6937 raft_consensus.cc:2243] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.328660  6937 raft_consensus.cc:2272] T 04780cde858b4000bfd73d06bbb04158 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.331123  6937 tablet_replica.cc:333] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388: stopping tablet replica
I20260709 13:17:07.331621  6937 raft_consensus.cc:2243] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.332288  6937 raft_consensus.cc:2272] T 6c6dc3aeee5c486b83aa69bac75c7868 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.334014  6937 tablet_replica.cc:333] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388: stopping tablet replica
I20260709 13:17:07.334535  6937 raft_consensus.cc:2243] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.335198  6937 raft_consensus.cc:2272] T 75f1eba07e9a41b9a1a5e4b47fb76d5e P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.337075  6937 tablet_replica.cc:333] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388: stopping tablet replica
I20260709 13:17:07.337528  6937 raft_consensus.cc:2243] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.337927  6937 raft_consensus.cc:2272] T f7153ce9502848edac1c5ef91ef65715 P baa0fccfc08d477ba279ad0968edb388 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.367667  6937 tablet_server.cc:196] TabletServer@127.6.198.67:0 shutdown complete.
I20260709 13:17:07.390400  6937 master.cc:562] Master@127.6.198.126:35929 shutting down...
I20260709 13:17:07.409641  6937 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.410095  6937 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.410416  6937 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3553f97babe548af91484ab5daf50c73: stopping tablet replica
I20260709 13:17:07.428337  6937 master.cc:584] Master@127.6.198.126:35929 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (12636 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:17:07.471381  6937 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.198.126:41679
I20260709 13:17:07.472362  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:07.477522 17783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:07.477716 17784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:07.478549 17786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:07.479214  6937 server_base.cc:1061] running on GCE node
I20260709 13:17:07.480003  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:07.480166  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:07.480360  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603027480328 us; error 0 us; skew 500 ppm
I20260709 13:17:07.480863  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:07.483121  6937 webserver.cc:533] Webserver started at http://127.6.198.126:40663/ using document root <none> and password file <none>
I20260709 13:17:07.483587  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:07.483796  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:07.484066  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:07.485098  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/master-0-root/instance:
uuid: "f5831f2d82844288b78d489a742ddc77"
format_stamp: "Formatted at 2026-07-09 13:17:07 on dist-test-slave-tqzw"
I20260709 13:17:07.489420  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:17:07.492543 17791 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:07.493260  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:17:07.493541  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/master-0-root
uuid: "f5831f2d82844288b78d489a742ddc77"
format_stamp: "Formatted at 2026-07-09 13:17:07 on dist-test-slave-tqzw"
I20260709 13:17:07.493816  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:07.503579  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:07.504640  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:07.540513  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.126:41679
I20260709 13:17:07.540609 17842 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.126:41679 every 8 connection(s)
I20260709 13:17:07.544216 17843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:07.554275 17843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77: Bootstrap starting.
I20260709 13:17:07.558848 17843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:07.562784 17843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77: No bootstrap required, opened a new log
I20260709 13:17:07.564787 17843 raft_consensus.cc:359] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5831f2d82844288b78d489a742ddc77" member_type: VOTER }
I20260709 13:17:07.565152 17843 raft_consensus.cc:385] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:07.565378 17843 raft_consensus.cc:740] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5831f2d82844288b78d489a742ddc77, State: Initialized, Role: FOLLOWER
I20260709 13:17:07.565923 17843 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5831f2d82844288b78d489a742ddc77" member_type: VOTER }
I20260709 13:17:07.566383 17843 raft_consensus.cc:399] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:07.566701 17843 raft_consensus.cc:493] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:07.566998 17843 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:07.571111 17843 raft_consensus.cc:515] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5831f2d82844288b78d489a742ddc77" member_type: VOTER }
I20260709 13:17:07.571578 17843 leader_election.cc:304] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5831f2d82844288b78d489a742ddc77; no voters: 
I20260709 13:17:07.572626 17843 leader_election.cc:290] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:07.573009 17846 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:07.574278 17846 raft_consensus.cc:697] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [term 1 LEADER]: Becoming Leader. State: Replica: f5831f2d82844288b78d489a742ddc77, State: Running, Role: LEADER
I20260709 13:17:07.574923 17846 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5831f2d82844288b78d489a742ddc77" member_type: VOTER }
I20260709 13:17:07.575349 17843 sys_catalog.cc:565] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:07.579087 17847 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5831f2d82844288b78d489a742ddc77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5831f2d82844288b78d489a742ddc77" member_type: VOTER } }
I20260709 13:17:07.580016 17848 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5831f2d82844288b78d489a742ddc77. Latest consensus state: current_term: 1 leader_uuid: "f5831f2d82844288b78d489a742ddc77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5831f2d82844288b78d489a742ddc77" member_type: VOTER } }
I20260709 13:17:07.580480 17847 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:07.580578 17848 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:07.582276 17853 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:07.587122 17853 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:07.590378  6937 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:07.595079 17853 catalog_manager.cc:1383] Generated new cluster ID: b15812a9036a4209ae2a6cdc4b7aa0a2
I20260709 13:17:07.595326 17853 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:07.617911 17853 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:07.619119 17853 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:07.630357 17853 catalog_manager.cc:6070] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77: Generated new TSK 0
I20260709 13:17:07.630921 17853 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:07.656284  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:07.661885 17864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:07.663069 17865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:07.664527 17867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:07.665395  6937 server_base.cc:1061] running on GCE node
I20260709 13:17:07.666194  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:07.666445  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:07.666575  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603027666562 us; error 0 us; skew 500 ppm
I20260709 13:17:07.667130  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:07.669237  6937 webserver.cc:533] Webserver started at http://127.6.198.65:35461/ using document root <none> and password file <none>
I20260709 13:17:07.669700  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:07.669854  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:07.670063  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:07.671108  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-0-root/instance:
uuid: "6533af55484e4160a399e4886ad5242e"
format_stamp: "Formatted at 2026-07-09 13:17:07 on dist-test-slave-tqzw"
I20260709 13:17:07.675195  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:17:07.678655 17872 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:07.679373  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:17:07.679661  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-0-root
uuid: "6533af55484e4160a399e4886ad5242e"
format_stamp: "Formatted at 2026-07-09 13:17:07 on dist-test-slave-tqzw"
I20260709 13:17:07.679951  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:07.691313  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:07.692405  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:07.693805  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:07.696043  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:07.696251  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:07.696486  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:07.696653  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:07.735551  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.65:37765
I20260709 13:17:07.735663 17934 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.65:37765 every 8 connection(s)
I20260709 13:17:07.740372  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:07.747725 17939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:07.748410 17940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:07.752302 17942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:07.752790  6937 server_base.cc:1061] running on GCE node
I20260709 13:17:07.753787 17935 heartbeater.cc:344] Connected to a master server at 127.6.198.126:41679
I20260709 13:17:07.753995  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:17:07.754172 17935 heartbeater.cc:461] Registering TS with master...
W20260709 13:17:07.754314  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:07.754676  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603027754650 us; error 0 us; skew 500 ppm
I20260709 13:17:07.755129 17935 heartbeater.cc:507] Master 127.6.198.126:41679 requested a full tablet report, sending...
I20260709 13:17:07.755276  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:07.757330 17808 ts_manager.cc:194] Registered new tserver with Master: 6533af55484e4160a399e4886ad5242e (127.6.198.65:37765)
I20260709 13:17:07.758490  6937 webserver.cc:533] Webserver started at http://127.6.198.66:33467/ using document root <none> and password file <none>
I20260709 13:17:07.759131  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:07.759369  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:07.759642  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:07.759809 17808 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45364
I20260709 13:17:07.761003  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-1-root/instance:
uuid: "61e0f20eba074de0a6eb1b1f14c0bafa"
format_stamp: "Formatted at 2026-07-09 13:17:07 on dist-test-slave-tqzw"
I20260709 13:17:07.765309  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:17:07.768421 17947 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:07.769091  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:17:07.769380  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-1-root
uuid: "61e0f20eba074de0a6eb1b1f14c0bafa"
format_stamp: "Formatted at 2026-07-09 13:17:07 on dist-test-slave-tqzw"
I20260709 13:17:07.769634  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:07.807713  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:07.808897  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:07.810285  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:07.812403  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:07.812626  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:07.812872  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:07.813045  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:07.853039  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.66:35901
I20260709 13:17:07.853156 18009 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.66:35901 every 8 connection(s)
I20260709 13:17:07.857945  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:07.864626 18013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:07.864938 18014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:07.867923  6937 server_base.cc:1061] running on GCE node
I20260709 13:17:07.868346 18010 heartbeater.cc:344] Connected to a master server at 127.6.198.126:41679
I20260709 13:17:07.868772 18010 heartbeater.cc:461] Registering TS with master...
W20260709 13:17:07.869549 18017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:07.869771 18010 heartbeater.cc:507] Master 127.6.198.126:41679 requested a full tablet report, sending...
I20260709 13:17:07.870549  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:07.870860  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:07.871039  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603027871024 us; error 0 us; skew 500 ppm
I20260709 13:17:07.871631  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:07.871996 17808 ts_manager.cc:194] Registered new tserver with Master: 61e0f20eba074de0a6eb1b1f14c0bafa (127.6.198.66:35901)
I20260709 13:17:07.873651 17808 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45372
I20260709 13:17:07.874140  6937 webserver.cc:533] Webserver started at http://127.6.198.67:39299/ using document root <none> and password file <none>
I20260709 13:17:07.874826  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:07.875052  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:07.875312  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:07.876395  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-2-root/instance:
uuid: "9d1296949a5742af94fa7e8b3388cee6"
format_stamp: "Formatted at 2026-07-09 13:17:07 on dist-test-slave-tqzw"
I20260709 13:17:07.880649  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:17:07.883966 18021 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:07.884735  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:17:07.885056  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-2-root
uuid: "9d1296949a5742af94fa7e8b3388cee6"
format_stamp: "Formatted at 2026-07-09 13:17:07 on dist-test-slave-tqzw"
I20260709 13:17:07.885411  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602718985547-6937-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:07.899147  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:07.900282  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:07.901676  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:07.903854  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:07.904083  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:07.904320  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:07.904489  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:07.944164  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.67:36675
I20260709 13:17:07.944286 18083 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.67:36675 every 8 connection(s)
I20260709 13:17:07.958190 18084 heartbeater.cc:344] Connected to a master server at 127.6.198.126:41679
I20260709 13:17:07.958709 18084 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:07.959373 18084 heartbeater.cc:507] Master 127.6.198.126:41679 requested a full tablet report, sending...
I20260709 13:17:07.961248 17808 ts_manager.cc:194] Registered new tserver with Master: 9d1296949a5742af94fa7e8b3388cee6 (127.6.198.67:36675)
I20260709 13:17:07.961827  6937 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014531465s
I20260709 13:17:07.962674 17808 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45386
I20260709 13:17:07.986409 17808 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45390:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:07.988670 17808 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:08.038592 18049 tablet_service.cc:1511] Processing CreateTablet for tablet 14f04716dbfd4058937365b4822995f4 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=a3fd4c0f8dfd45519cefb9763a7ba464]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:08.038867 18048 tablet_service.cc:1511] Processing CreateTablet for tablet c1aa673d8841432ba7d3a56cf33711aa (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=a3fd4c0f8dfd45519cefb9763a7ba464]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:08.040238 18049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14f04716dbfd4058937365b4822995f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.043726 17974 tablet_service.cc:1511] Processing CreateTablet for tablet c1aa673d8841432ba7d3a56cf33711aa (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=a3fd4c0f8dfd45519cefb9763a7ba464]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:08.043972 17975 tablet_service.cc:1511] Processing CreateTablet for tablet 14f04716dbfd4058937365b4822995f4 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=a3fd4c0f8dfd45519cefb9763a7ba464]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:08.045095 17974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1aa673d8841432ba7d3a56cf33711aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.043991 18048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1aa673d8841432ba7d3a56cf33711aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.046761 17899 tablet_service.cc:1511] Processing CreateTablet for tablet c1aa673d8841432ba7d3a56cf33711aa (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=a3fd4c0f8dfd45519cefb9763a7ba464]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:08.048099 17899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1aa673d8841432ba7d3a56cf33711aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.048578 17975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14f04716dbfd4058937365b4822995f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.045483 17900 tablet_service.cc:1511] Processing CreateTablet for tablet 14f04716dbfd4058937365b4822995f4 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=a3fd4c0f8dfd45519cefb9763a7ba464]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:08.053967 17900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14f04716dbfd4058937365b4822995f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.084447 18099 tablet_bootstrap.cc:492] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6: Bootstrap starting.
I20260709 13:17:08.084507 18100 tablet_bootstrap.cc:492] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa: Bootstrap starting.
I20260709 13:17:08.091472 18099 tablet_bootstrap.cc:654] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.101780 18100 tablet_bootstrap.cc:654] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.108947 18099 tablet_bootstrap.cc:492] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6: No bootstrap required, opened a new log
I20260709 13:17:08.109050 18100 tablet_bootstrap.cc:492] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa: No bootstrap required, opened a new log
I20260709 13:17:08.108922 18102 tablet_bootstrap.cc:492] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e: Bootstrap starting.
I20260709 13:17:08.109632 18099 ts_tablet_manager.cc:1403] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.009s
I20260709 13:17:08.109666 18100 ts_tablet_manager.cc:1403] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.012s
I20260709 13:17:08.112214 18100 raft_consensus.cc:359] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.112753 18100 raft_consensus.cc:385] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.113062 18100 raft_consensus.cc:740] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61e0f20eba074de0a6eb1b1f14c0bafa, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.112783 18099 raft_consensus.cc:359] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } }
I20260709 13:17:08.113731 18099 raft_consensus.cc:385] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.113798 18100 consensus_queue.cc:260] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.114108 18099 raft_consensus.cc:740] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d1296949a5742af94fa7e8b3388cee6, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.115012 18099 consensus_queue.cc:260] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } }
I20260709 13:17:08.116613 18102 tablet_bootstrap.cc:654] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.119729 18084 heartbeater.cc:499] Master 127.6.198.126:41679 was elected leader, sending a full tablet report...
I20260709 13:17:08.120066 18099 ts_tablet_manager.cc:1434] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.006s
I20260709 13:17:08.121413 18099 tablet_bootstrap.cc:492] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6: Bootstrap starting.
I20260709 13:17:08.129841 18099 tablet_bootstrap.cc:654] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.133013 18102 tablet_bootstrap.cc:492] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e: No bootstrap required, opened a new log
I20260709 13:17:08.133562 18102 ts_tablet_manager.cc:1403] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.016s
I20260709 13:17:08.137660 18102 raft_consensus.cc:359] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.138849 18102 raft_consensus.cc:385] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.139318 18102 raft_consensus.cc:740] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6533af55484e4160a399e4886ad5242e, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.140347 18102 consensus_queue.cc:260] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.143069 18099 tablet_bootstrap.cc:492] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6: No bootstrap required, opened a new log
I20260709 13:17:08.143887 18099 ts_tablet_manager.cc:1403] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.011s
I20260709 13:17:08.144291 18010 heartbeater.cc:499] Master 127.6.198.126:41679 was elected leader, sending a full tablet report...
I20260709 13:17:08.144892 18100 ts_tablet_manager.cc:1434] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa: Time spent starting tablet: real 0.035s	user 0.012s	sys 0.021s
I20260709 13:17:08.145990 18100 tablet_bootstrap.cc:492] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa: Bootstrap starting.
I20260709 13:17:08.147832 18099 raft_consensus.cc:359] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.149255 18099 raft_consensus.cc:385] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.149739 18099 raft_consensus.cc:740] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d1296949a5742af94fa7e8b3388cee6, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.150964 18099 consensus_queue.cc:260] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.153303 18100 tablet_bootstrap.cc:654] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.153944 18099 ts_tablet_manager.cc:1434] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:17:08.172379 17935 heartbeater.cc:499] Master 127.6.198.126:41679 was elected leader, sending a full tablet report...
I20260709 13:17:08.172978 18102 ts_tablet_manager.cc:1434] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e: Time spent starting tablet: real 0.039s	user 0.021s	sys 0.017s
I20260709 13:17:08.173892 18102 tablet_bootstrap.cc:492] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e: Bootstrap starting.
I20260709 13:17:08.174158 18100 tablet_bootstrap.cc:492] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa: No bootstrap required, opened a new log
I20260709 13:17:08.174695 18100 ts_tablet_manager.cc:1403] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa: Time spent bootstrapping tablet: real 0.029s	user 0.021s	sys 0.006s
I20260709 13:17:08.177119 18100 raft_consensus.cc:359] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } }
I20260709 13:17:08.177699 18100 raft_consensus.cc:385] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.178047 18100 raft_consensus.cc:740] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61e0f20eba074de0a6eb1b1f14c0bafa, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.178831 18100 consensus_queue.cc:260] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } }
I20260709 13:17:08.180317 18102 tablet_bootstrap.cc:654] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.182631 18100 ts_tablet_manager.cc:1434] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:17:08.186002 18102 tablet_bootstrap.cc:492] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e: No bootstrap required, opened a new log
I20260709 13:17:08.186546 18102 ts_tablet_manager.cc:1403] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.003s
I20260709 13:17:08.189352 18102 raft_consensus.cc:359] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } }
I20260709 13:17:08.189884 18102 raft_consensus.cc:385] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.190135 18102 raft_consensus.cc:740] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6533af55484e4160a399e4886ad5242e, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.190742 18102 consensus_queue.cc:260] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } }
I20260709 13:17:08.192339 18102 ts_tablet_manager.cc:1434] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:17:08.391927 18108 raft_consensus.cc:493] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.392400 18108 raft_consensus.cc:515] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.394455 18108 leader_election.cc:290] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61e0f20eba074de0a6eb1b1f14c0bafa (127.6.198.66:35901), 9d1296949a5742af94fa7e8b3388cee6 (127.6.198.67:36675)
I20260709 13:17:08.404126 17985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1aa673d8841432ba7d3a56cf33711aa" candidate_uuid: "6533af55484e4160a399e4886ad5242e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" is_pre_election: true
I20260709 13:17:08.404896 17985 raft_consensus.cc:2468] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6533af55484e4160a399e4886ad5242e in term 0.
I20260709 13:17:08.405866 18059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1aa673d8841432ba7d3a56cf33711aa" candidate_uuid: "6533af55484e4160a399e4886ad5242e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d1296949a5742af94fa7e8b3388cee6" is_pre_election: true
I20260709 13:17:08.406255 17875 leader_election.cc:304] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61e0f20eba074de0a6eb1b1f14c0bafa, 6533af55484e4160a399e4886ad5242e; no voters: 
I20260709 13:17:08.406591 18059 raft_consensus.cc:2468] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6533af55484e4160a399e4886ad5242e in term 0.
I20260709 13:17:08.407157 18108 raft_consensus.cc:2804] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:08.407500 18108 raft_consensus.cc:493] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.407788 18108 raft_consensus.cc:3060] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.412590 18108 raft_consensus.cc:515] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.414039 18108 leader_election.cc:290] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [CANDIDATE]: Term 1 election: Requested vote from peers 61e0f20eba074de0a6eb1b1f14c0bafa (127.6.198.66:35901), 9d1296949a5742af94fa7e8b3388cee6 (127.6.198.67:36675)
I20260709 13:17:08.414835 17985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1aa673d8841432ba7d3a56cf33711aa" candidate_uuid: "6533af55484e4160a399e4886ad5242e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa"
I20260709 13:17:08.415115 18059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1aa673d8841432ba7d3a56cf33711aa" candidate_uuid: "6533af55484e4160a399e4886ad5242e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d1296949a5742af94fa7e8b3388cee6"
I20260709 13:17:08.415367 17985 raft_consensus.cc:3060] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.415673 18059 raft_consensus.cc:3060] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.420302 17985 raft_consensus.cc:2468] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6533af55484e4160a399e4886ad5242e in term 1.
I20260709 13:17:08.420471 18059 raft_consensus.cc:2468] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6533af55484e4160a399e4886ad5242e in term 1.
I20260709 13:17:08.421276 17875 leader_election.cc:304] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61e0f20eba074de0a6eb1b1f14c0bafa, 6533af55484e4160a399e4886ad5242e; no voters: 
I20260709 13:17:08.422008 18108 raft_consensus.cc:2804] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:08.422930 18108 raft_consensus.cc:697] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 1 LEADER]: Becoming Leader. State: Replica: 6533af55484e4160a399e4886ad5242e, State: Running, Role: LEADER
I20260709 13:17:08.423574 18108 consensus_queue.cc:237] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.431226 17807 catalog_manager.cc:5697] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6533af55484e4160a399e4886ad5242e (127.6.198.65). New cstate: current_term: 1 leader_uuid: "6533af55484e4160a399e4886ad5242e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:08.492865 18105 raft_consensus.cc:493] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.493469 18105 raft_consensus.cc:515] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } }
I20260709 13:17:08.495843 18105 leader_election.cc:290] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6533af55484e4160a399e4886ad5242e (127.6.198.65:37765), 61e0f20eba074de0a6eb1b1f14c0bafa (127.6.198.66:35901)
I20260709 13:17:08.506039 17910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f04716dbfd4058937365b4822995f4" candidate_uuid: "9d1296949a5742af94fa7e8b3388cee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6533af55484e4160a399e4886ad5242e" is_pre_election: true
I20260709 13:17:08.506651 17985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f04716dbfd4058937365b4822995f4" candidate_uuid: "9d1296949a5742af94fa7e8b3388cee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" is_pre_election: true
I20260709 13:17:08.506788 17910 raft_consensus.cc:2468] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d1296949a5742af94fa7e8b3388cee6 in term 0.
I20260709 13:17:08.507326 17985 raft_consensus.cc:2468] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d1296949a5742af94fa7e8b3388cee6 in term 0.
I20260709 13:17:08.508069 18024 leader_election.cc:304] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6533af55484e4160a399e4886ad5242e, 9d1296949a5742af94fa7e8b3388cee6; no voters: 
I20260709 13:17:08.508769 18105 raft_consensus.cc:2804] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:08.509052 18105 raft_consensus.cc:493] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.509292 18105 raft_consensus.cc:3060] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.514103 18105 raft_consensus.cc:515] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } }
I20260709 13:17:08.515554 18105 leader_election.cc:290] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [CANDIDATE]: Term 1 election: Requested vote from peers 6533af55484e4160a399e4886ad5242e (127.6.198.65:37765), 61e0f20eba074de0a6eb1b1f14c0bafa (127.6.198.66:35901)
I20260709 13:17:08.516338 17910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f04716dbfd4058937365b4822995f4" candidate_uuid: "9d1296949a5742af94fa7e8b3388cee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6533af55484e4160a399e4886ad5242e"
I20260709 13:17:08.516592 17985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f04716dbfd4058937365b4822995f4" candidate_uuid: "9d1296949a5742af94fa7e8b3388cee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa"
I20260709 13:17:08.516856 17910 raft_consensus.cc:3060] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.517216 17985 raft_consensus.cc:3060] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.521677 17910 raft_consensus.cc:2468] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d1296949a5742af94fa7e8b3388cee6 in term 1.
I20260709 13:17:08.521752 17985 raft_consensus.cc:2468] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d1296949a5742af94fa7e8b3388cee6 in term 1.
I20260709 13:17:08.522780 18024 leader_election.cc:304] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6533af55484e4160a399e4886ad5242e, 9d1296949a5742af94fa7e8b3388cee6; no voters: 
I20260709 13:17:08.523433 18105 raft_consensus.cc:2804] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:08.524338 18105 raft_consensus.cc:697] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 LEADER]: Becoming Leader. State: Replica: 9d1296949a5742af94fa7e8b3388cee6, State: Running, Role: LEADER
I20260709 13:17:08.524972 18105 consensus_queue.cc:237] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } }
I20260709 13:17:08.531939 17807 catalog_manager.cc:5697] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d1296949a5742af94fa7e8b3388cee6 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "9d1296949a5742af94fa7e8b3388cee6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:08.628944 17807 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45402:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:08.630824 17807 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:08.653119 17899 tablet_service.cc:1511] Processing CreateTablet for tablet 5c8aafdf099d44148d880851a873de56 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=478124e8eee54a51b15a834af1662687]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:08.654467 17899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c8aafdf099d44148d880851a873de56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.654356 17975 tablet_service.cc:1511] Processing CreateTablet for tablet 5c8aafdf099d44148d880851a873de56 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=478124e8eee54a51b15a834af1662687]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:08.655652 17975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c8aafdf099d44148d880851a873de56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.655665 18048 tablet_service.cc:1511] Processing CreateTablet for tablet 5c8aafdf099d44148d880851a873de56 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=478124e8eee54a51b15a834af1662687]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:08.657011 18048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c8aafdf099d44148d880851a873de56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.661196 17974 tablet_service.cc:1511] Processing CreateTablet for tablet 35e46bac39354a7c985b4d83d48bbc31 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=478124e8eee54a51b15a834af1662687]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:08.661446 18049 tablet_service.cc:1511] Processing CreateTablet for tablet 35e46bac39354a7c985b4d83d48bbc31 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=478124e8eee54a51b15a834af1662687]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:08.662468 17974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e46bac39354a7c985b4d83d48bbc31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.662685 18049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e46bac39354a7c985b4d83d48bbc31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.667302 17900 tablet_service.cc:1511] Processing CreateTablet for tablet 35e46bac39354a7c985b4d83d48bbc31 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=478124e8eee54a51b15a834af1662687]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:08.668519 17900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35e46bac39354a7c985b4d83d48bbc31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.672022 18100 tablet_bootstrap.cc:492] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa: Bootstrap starting.
I20260709 13:17:08.679577 18100 tablet_bootstrap.cc:654] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.689059 18100 tablet_bootstrap.cc:492] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa: No bootstrap required, opened a new log
I20260709 13:17:08.689515 18100 ts_tablet_manager.cc:1403] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.000s
I20260709 13:17:08.692493 18100 raft_consensus.cc:359] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.693231 18100 raft_consensus.cc:385] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.693567 18100 raft_consensus.cc:740] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61e0f20eba074de0a6eb1b1f14c0bafa, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.694403 18100 consensus_queue.cc:260] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.696101 18125 tablet_bootstrap.cc:492] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6: Bootstrap starting.
I20260709 13:17:08.696977 18127 tablet_bootstrap.cc:492] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e: Bootstrap starting.
I20260709 13:17:08.703722 18100 ts_tablet_manager.cc:1434] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa: Time spent starting tablet: real 0.014s	user 0.012s	sys 0.002s
I20260709 13:17:08.704811 18100 tablet_bootstrap.cc:492] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa: Bootstrap starting.
I20260709 13:17:08.706560 18125 tablet_bootstrap.cc:654] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.707031 18127 tablet_bootstrap.cc:654] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.711566 18100 tablet_bootstrap.cc:654] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.720398 18125 tablet_bootstrap.cc:492] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6: No bootstrap required, opened a new log
I20260709 13:17:08.720618 18127 tablet_bootstrap.cc:492] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e: No bootstrap required, opened a new log
I20260709 13:17:08.720875 18125 ts_tablet_manager.cc:1403] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.013s
I20260709 13:17:08.721081 18127 ts_tablet_manager.cc:1403] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.007s
I20260709 13:17:08.721379 18100 tablet_bootstrap.cc:492] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa: No bootstrap required, opened a new log
I20260709 13:17:08.721882 18100 ts_tablet_manager.cc:1403] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.007s
I20260709 13:17:08.723971 18127 raft_consensus.cc:359] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.724205 18125 raft_consensus.cc:359] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.724802 18127 raft_consensus.cc:385] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.724649 18100 raft_consensus.cc:359] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.725042 18125 raft_consensus.cc:385] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.725351 18127 raft_consensus.cc:740] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6533af55484e4160a399e4886ad5242e, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.725500 18100 raft_consensus.cc:385] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.725551 18125 raft_consensus.cc:740] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d1296949a5742af94fa7e8b3388cee6, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.726089 18100 raft_consensus.cc:740] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61e0f20eba074de0a6eb1b1f14c0bafa, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.726445 18127 consensus_queue.cc:260] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.726688 18125 consensus_queue.cc:260] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.726855 18100 consensus_queue.cc:260] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.728385 18127 ts_tablet_manager.cc:1434] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:17:08.729609 18125 ts_tablet_manager.cc:1434] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:17:08.730301 18127 tablet_bootstrap.cc:492] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e: Bootstrap starting.
I20260709 13:17:08.730396 18128 raft_consensus.cc:493] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.730669 18125 tablet_bootstrap.cc:492] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6: Bootstrap starting.
I20260709 13:17:08.730979 18128 raft_consensus.cc:515] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.732362 18100 ts_tablet_manager.cc:1434] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:17:08.735468 18128 leader_election.cc:290] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6533af55484e4160a399e4886ad5242e (127.6.198.65:37765), 9d1296949a5742af94fa7e8b3388cee6 (127.6.198.67:36675)
I20260709 13:17:08.739667 18125 tablet_bootstrap.cc:654] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.740665 18127 tablet_bootstrap.cc:654] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.755219 18127 tablet_bootstrap.cc:492] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e: No bootstrap required, opened a new log
I20260709 13:17:08.755818 18127 ts_tablet_manager.cc:1403] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.009s
I20260709 13:17:08.758791 18127 raft_consensus.cc:359] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.759613 18127 raft_consensus.cc:385] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.759975 18127 raft_consensus.cc:740] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6533af55484e4160a399e4886ad5242e, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.760823 18127 consensus_queue.cc:260] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.763494 18127 ts_tablet_manager.cc:1434] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:17:08.764947 18125 tablet_bootstrap.cc:492] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6: No bootstrap required, opened a new log
I20260709 13:17:08.766690 18125 ts_tablet_manager.cc:1403] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.012s
I20260709 13:17:08.769784 18125 raft_consensus.cc:359] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.770182 17910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e46bac39354a7c985b4d83d48bbc31" candidate_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6533af55484e4160a399e4886ad5242e" is_pre_election: true
I20260709 13:17:08.770427 18125 raft_consensus.cc:385] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.770936 17910 raft_consensus.cc:2468] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61e0f20eba074de0a6eb1b1f14c0bafa in term 0.
I20260709 13:17:08.770942 18125 raft_consensus.cc:740] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d1296949a5742af94fa7e8b3388cee6, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.772013 18125 consensus_queue.cc:260] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.772401 17950 leader_election.cc:304] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61e0f20eba074de0a6eb1b1f14c0bafa, 6533af55484e4160a399e4886ad5242e; no voters: 
I20260709 13:17:08.773222 18128 raft_consensus.cc:2804] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:08.773577 18128 raft_consensus.cc:493] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.773891 18128 raft_consensus.cc:3060] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.774322 18125 ts_tablet_manager.cc:1434] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:17:08.775753 18059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e46bac39354a7c985b4d83d48bbc31" candidate_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d1296949a5742af94fa7e8b3388cee6" is_pre_election: true
I20260709 13:17:08.776381 18059 raft_consensus.cc:2468] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61e0f20eba074de0a6eb1b1f14c0bafa in term 0.
I20260709 13:17:08.780380 18128 raft_consensus.cc:515] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.782163 18128 leader_election.cc:290] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [CANDIDATE]: Term 1 election: Requested vote from peers 6533af55484e4160a399e4886ad5242e (127.6.198.65:37765), 9d1296949a5742af94fa7e8b3388cee6 (127.6.198.67:36675)
I20260709 13:17:08.782794 17910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e46bac39354a7c985b4d83d48bbc31" candidate_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6533af55484e4160a399e4886ad5242e"
I20260709 13:17:08.783047 18059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e46bac39354a7c985b4d83d48bbc31" candidate_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d1296949a5742af94fa7e8b3388cee6"
I20260709 13:17:08.783407 17910 raft_consensus.cc:3060] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.783618 18059 raft_consensus.cc:3060] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.786880 18105 raft_consensus.cc:493] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.787395 18105 raft_consensus.cc:515] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.789405 18105 leader_election.cc:290] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6533af55484e4160a399e4886ad5242e (127.6.198.65:37765), 61e0f20eba074de0a6eb1b1f14c0bafa (127.6.198.66:35901)
I20260709 13:17:08.789366 18059 raft_consensus.cc:2468] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61e0f20eba074de0a6eb1b1f14c0bafa in term 1.
I20260709 13:17:08.790428 17985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8aafdf099d44148d880851a873de56" candidate_uuid: "9d1296949a5742af94fa7e8b3388cee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" is_pre_election: true
I20260709 13:17:08.790494 17909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8aafdf099d44148d880851a873de56" candidate_uuid: "9d1296949a5742af94fa7e8b3388cee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6533af55484e4160a399e4886ad5242e" is_pre_election: true
I20260709 13:17:08.790715 17950 leader_election.cc:304] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61e0f20eba074de0a6eb1b1f14c0bafa, 9d1296949a5742af94fa7e8b3388cee6; no voters: 
I20260709 13:17:08.791239 17985 raft_consensus.cc:2468] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d1296949a5742af94fa7e8b3388cee6 in term 0.
I20260709 13:17:08.791247 17909 raft_consensus.cc:2468] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d1296949a5742af94fa7e8b3388cee6 in term 0.
I20260709 13:17:08.791561 18128 raft_consensus.cc:2804] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:08.791960 17910 raft_consensus.cc:2468] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61e0f20eba074de0a6eb1b1f14c0bafa in term 1.
I20260709 13:17:08.792514 18024 leader_election.cc:304] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6533af55484e4160a399e4886ad5242e, 9d1296949a5742af94fa7e8b3388cee6; no voters: 
I20260709 13:17:08.792990 18128 raft_consensus.cc:697] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 LEADER]: Becoming Leader. State: Replica: 61e0f20eba074de0a6eb1b1f14c0bafa, State: Running, Role: LEADER
I20260709 13:17:08.794039 18105 raft_consensus.cc:2804] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:08.793856 18128 consensus_queue.cc:237] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.794435 18105 raft_consensus.cc:493] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.794800 18105 raft_consensus.cc:3060] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.801540 18105 raft_consensus.cc:515] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.802228 17804 catalog_manager.cc:5697] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa reported cstate change: term changed from 0 to 1, leader changed from <none> to 61e0f20eba074de0a6eb1b1f14c0bafa (127.6.198.66). New cstate: current_term: 1 leader_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:08.803479 18105 leader_election.cc:290] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [CANDIDATE]: Term 1 election: Requested vote from peers 6533af55484e4160a399e4886ad5242e (127.6.198.65:37765), 61e0f20eba074de0a6eb1b1f14c0bafa (127.6.198.66:35901)
I20260709 13:17:08.804176 17910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8aafdf099d44148d880851a873de56" candidate_uuid: "9d1296949a5742af94fa7e8b3388cee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6533af55484e4160a399e4886ad5242e"
I20260709 13:17:08.804474 17985 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8aafdf099d44148d880851a873de56" candidate_uuid: "9d1296949a5742af94fa7e8b3388cee6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa"
I20260709 13:17:08.804797 17910 raft_consensus.cc:3060] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.805058 17985 raft_consensus.cc:3060] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.810956 17985 raft_consensus.cc:2468] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d1296949a5742af94fa7e8b3388cee6 in term 1.
I20260709 13:17:08.811465 17910 raft_consensus.cc:2468] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d1296949a5742af94fa7e8b3388cee6 in term 1.
I20260709 13:17:08.811861 18024 leader_election.cc:304] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61e0f20eba074de0a6eb1b1f14c0bafa, 9d1296949a5742af94fa7e8b3388cee6; no voters: 
I20260709 13:17:08.812654 18105 raft_consensus.cc:2804] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:08.813042 18105 raft_consensus.cc:697] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 LEADER]: Becoming Leader. State: Replica: 9d1296949a5742af94fa7e8b3388cee6, State: Running, Role: LEADER
I20260709 13:17:08.813777 18105 consensus_queue.cc:237] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.820144 17804 catalog_manager.cc:5697] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d1296949a5742af94fa7e8b3388cee6 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "9d1296949a5742af94fa7e8b3388cee6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:08.844712 18059 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14f04716dbfd4058937365b4822995f4"
dest_uuid: "9d1296949a5742af94fa7e8b3388cee6"
mode: GRACEFUL
new_leader_uuid: "6533af55484e4160a399e4886ad5242e"
 from {username='slave'} at 127.0.0.1:58494
I20260709 13:17:08.845311 18059 raft_consensus.cc:606] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 LEADER]: Received request to transfer leadership to TS 6533af55484e4160a399e4886ad5242e
I20260709 13:17:08.856521 17910 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1aa673d8841432ba7d3a56cf33711aa"
dest_uuid: "6533af55484e4160a399e4886ad5242e"
mode: GRACEFUL
new_leader_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa"
 from {username='slave'} at 127.0.0.1:43754
I20260709 13:17:08.857040 17910 raft_consensus.cc:606] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 1 LEADER]: Received request to transfer leadership to TS 61e0f20eba074de0a6eb1b1f14c0bafa
I20260709 13:17:08.862435 18059 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c8aafdf099d44148d880851a873de56"
dest_uuid: "9d1296949a5742af94fa7e8b3388cee6"
mode: GRACEFUL
new_leader_uuid: "6533af55484e4160a399e4886ad5242e"
 from {username='slave'} at 127.0.0.1:58494
I20260709 13:17:08.862923 18059 raft_consensus.cc:606] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 LEADER]: Received request to transfer leadership to TS 6533af55484e4160a399e4886ad5242e
I20260709 13:17:08.869689 18108 consensus_queue.cc:1048] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:08.876689 18113 raft_consensus.cc:993] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e: : Instructing follower 61e0f20eba074de0a6eb1b1f14c0bafa to start an election
I20260709 13:17:08.877341 18108 raft_consensus.cc:1081] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 1 LEADER]: Signalling peer 61e0f20eba074de0a6eb1b1f14c0bafa to start an election
I20260709 13:17:08.878459 17985 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35e46bac39354a7c985b4d83d48bbc31"
dest_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa"
mode: GRACEFUL
new_leader_uuid: "6533af55484e4160a399e4886ad5242e"
 from {username='slave'} at 127.0.0.1:56542
I20260709 13:17:08.879654 17985 raft_consensus.cc:606] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 LEADER]: Received request to transfer leadership to TS 6533af55484e4160a399e4886ad5242e
I20260709 13:17:08.880519 17983 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c1aa673d8841432ba7d3a56cf33711aa"
dest_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa"
 from {username='slave'} at 127.0.0.1:56514
I20260709 13:17:08.885277 17983 raft_consensus.cc:493] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:08.885675 17983 raft_consensus.cc:3060] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:08.891959 18113 consensus_queue.cc:1048] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:08.894425 17983 raft_consensus.cc:515] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.900133 17910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1aa673d8841432ba7d3a56cf33711aa" candidate_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6533af55484e4160a399e4886ad5242e"
I20260709 13:17:08.900722 17910 raft_consensus.cc:3055] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:08.901046 17910 raft_consensus.cc:740] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6533af55484e4160a399e4886ad5242e, State: Running, Role: LEADER
I20260709 13:17:08.901149 18058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1aa673d8841432ba7d3a56cf33711aa" candidate_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d1296949a5742af94fa7e8b3388cee6"
I20260709 13:17:08.902976 17910 consensus_queue.cc:260] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.901654 17983 leader_election.cc:290] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [CANDIDATE]: Term 2 election: Requested vote from peers 6533af55484e4160a399e4886ad5242e (127.6.198.65:37765), 9d1296949a5742af94fa7e8b3388cee6 (127.6.198.67:36675)
I20260709 13:17:08.904356 17910 raft_consensus.cc:3060] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:08.910897 17910 raft_consensus.cc:2468] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61e0f20eba074de0a6eb1b1f14c0bafa in term 2.
I20260709 13:17:08.911834 17950 leader_election.cc:304] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61e0f20eba074de0a6eb1b1f14c0bafa, 6533af55484e4160a399e4886ad5242e; no voters: 9d1296949a5742af94fa7e8b3388cee6
I20260709 13:17:08.912537 18128 raft_consensus.cc:2804] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:08.913038 18128 raft_consensus.cc:697] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 LEADER]: Becoming Leader. State: Replica: 61e0f20eba074de0a6eb1b1f14c0bafa, State: Running, Role: LEADER
I20260709 13:17:08.913807 18128 consensus_queue.cc:237] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:08.920199 17806 catalog_manager.cc:5697] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa reported cstate change: term changed from 1 to 2, leader changed from 6533af55484e4160a399e4886ad5242e (127.6.198.65) to 61e0f20eba074de0a6eb1b1f14c0bafa (127.6.198.66). New cstate: current_term: 2 leader_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.098168 18105 consensus_queue.cc:1048] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.109246 18147 consensus_queue.cc:1048] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:09.116420 18105 raft_consensus.cc:993] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6: : Instructing follower 6533af55484e4160a399e4886ad5242e to start an election
I20260709 13:17:09.116864 18147 raft_consensus.cc:1081] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 LEADER]: Signalling peer 6533af55484e4160a399e4886ad5242e to start an election
I20260709 13:17:09.118296 17910 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14f04716dbfd4058937365b4822995f4"
dest_uuid: "6533af55484e4160a399e4886ad5242e"
 from {username='slave'} at 127.0.0.1:43736
I20260709 13:17:09.118901 17910 raft_consensus.cc:493] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:09.119256 17910 raft_consensus.cc:3060] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:09.124476 17910 raft_consensus.cc:515] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } }
I20260709 13:17:09.126070 17910 leader_election.cc:290] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [CANDIDATE]: Term 2 election: Requested vote from peers 9d1296949a5742af94fa7e8b3388cee6 (127.6.198.67:36675), 61e0f20eba074de0a6eb1b1f14c0bafa (127.6.198.66:35901)
I20260709 13:17:09.126863 18059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f04716dbfd4058937365b4822995f4" candidate_uuid: "6533af55484e4160a399e4886ad5242e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d1296949a5742af94fa7e8b3388cee6"
I20260709 13:17:09.127238 17983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14f04716dbfd4058937365b4822995f4" candidate_uuid: "6533af55484e4160a399e4886ad5242e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa"
I20260709 13:17:09.127532 18059 raft_consensus.cc:3055] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:09.127827 17983 raft_consensus.cc:3060] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:09.127940 18059 raft_consensus.cc:740] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d1296949a5742af94fa7e8b3388cee6, State: Running, Role: LEADER
I20260709 13:17:09.128870 18059 consensus_queue.cc:260] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } }
I20260709 13:17:09.130162 18059 raft_consensus.cc:3060] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:09.133242 17983 raft_consensus.cc:2468] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6533af55484e4160a399e4886ad5242e in term 2.
I20260709 13:17:09.134204 17875 leader_election.cc:304] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61e0f20eba074de0a6eb1b1f14c0bafa, 6533af55484e4160a399e4886ad5242e; no voters: 
I20260709 13:17:09.134927 18113 raft_consensus.cc:2804] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:09.135157 18059 raft_consensus.cc:2468] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6533af55484e4160a399e4886ad5242e in term 2.
I20260709 13:17:09.135336 18113 raft_consensus.cc:697] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [term 2 LEADER]: Becoming Leader. State: Replica: 6533af55484e4160a399e4886ad5242e, State: Running, Role: LEADER
I20260709 13:17:09.136271 18113 consensus_queue.cc:237] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } }
I20260709 13:17:09.142736 17806 catalog_manager.cc:5697] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e reported cstate change: term changed from 1 to 2, leader changed from 9d1296949a5742af94fa7e8b3388cee6 (127.6.198.67) to 6533af55484e4160a399e4886ad5242e (127.6.198.65). New cstate: current_term: 2 leader_uuid: "6533af55484e4160a399e4886ad5242e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.206374 18128 consensus_queue.cc:1048] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.212908 18136 raft_consensus.cc:993] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa: : Instructing follower 6533af55484e4160a399e4886ad5242e to start an election
I20260709 13:17:09.213634 18128 raft_consensus.cc:1081] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 LEADER]: Signalling peer 6533af55484e4160a399e4886ad5242e to start an election
I20260709 13:17:09.215939 17909 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35e46bac39354a7c985b4d83d48bbc31"
dest_uuid: "6533af55484e4160a399e4886ad5242e"
 from {username='slave'} at 127.0.0.1:43738
I20260709 13:17:09.216476 17909 raft_consensus.cc:493] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:09.216851 17909 raft_consensus.cc:3060] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:09.219655 18128 consensus_queue.cc:1048] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.224277 17909 raft_consensus.cc:515] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:09.227669 17983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e46bac39354a7c985b4d83d48bbc31" candidate_uuid: "6533af55484e4160a399e4886ad5242e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa"
I20260709 13:17:09.228148 17909 leader_election.cc:290] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [CANDIDATE]: Term 2 election: Requested vote from peers 61e0f20eba074de0a6eb1b1f14c0bafa (127.6.198.66:35901), 9d1296949a5742af94fa7e8b3388cee6 (127.6.198.67:36675)
I20260709 13:17:09.229110 17983 raft_consensus.cc:3055] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:09.229151 18059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e46bac39354a7c985b4d83d48bbc31" candidate_uuid: "6533af55484e4160a399e4886ad5242e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d1296949a5742af94fa7e8b3388cee6"
I20260709 13:17:09.230867 17983 raft_consensus.cc:740] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 61e0f20eba074de0a6eb1b1f14c0bafa, State: Running, Role: LEADER
I20260709 13:17:09.231560 18059 raft_consensus.cc:3060] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:09.232148 17983 consensus_queue.cc:260] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:09.233456 17983 raft_consensus.cc:3060] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:09.237236 18059 raft_consensus.cc:2468] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6533af55484e4160a399e4886ad5242e in term 2.
I20260709 13:17:09.238198 17875 leader_election.cc:304] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6533af55484e4160a399e4886ad5242e, 9d1296949a5742af94fa7e8b3388cee6; no voters: 
I20260709 13:17:09.238899 18113 raft_consensus.cc:2804] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:09.239388 18113 raft_consensus.cc:697] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 2 LEADER]: Becoming Leader. State: Replica: 6533af55484e4160a399e4886ad5242e, State: Running, Role: LEADER
I20260709 13:17:09.239450 17983 raft_consensus.cc:2468] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6533af55484e4160a399e4886ad5242e in term 2.
I20260709 13:17:09.240262 18113 consensus_queue.cc:237] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:09.246049 17804 catalog_manager.cc:5697] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e reported cstate change: term changed from 1 to 2, leader changed from 61e0f20eba074de0a6eb1b1f14c0bafa (127.6.198.66) to 6533af55484e4160a399e4886ad5242e (127.6.198.65). New cstate: current_term: 2 leader_uuid: "6533af55484e4160a399e4886ad5242e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.328116 18147 consensus_queue.cc:1048] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.334019 17909 raft_consensus.cc:1275] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 2 FOLLOWER]: Refusing update from remote peer 61e0f20eba074de0a6eb1b1f14c0bafa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:09.335608 18128 consensus_queue.cc:1048] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [LEADER]: Connected to new peer: Peer: permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:09.339084 18117 consensus_queue.cc:1048] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.346513 18059 raft_consensus.cc:3060] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:09.354624 18059 raft_consensus.cc:1275] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6 [term 2 FOLLOWER]: Refusing update from remote peer 61e0f20eba074de0a6eb1b1f14c0bafa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:09.355589 18147 raft_consensus.cc:993] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6: : Instructing follower 6533af55484e4160a399e4886ad5242e to start an election
I20260709 13:17:09.356089 18105 raft_consensus.cc:1081] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 LEADER]: Signalling peer 6533af55484e4160a399e4886ad5242e to start an election
I20260709 13:17:09.355973 18128 consensus_queue.cc:1048] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:09.357856 17909 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c8aafdf099d44148d880851a873de56"
dest_uuid: "6533af55484e4160a399e4886ad5242e"
 from {username='slave'} at 127.0.0.1:43736
I20260709 13:17:09.358435 17909 raft_consensus.cc:493] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:09.358839 17909 raft_consensus.cc:3060] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:09.366915 17909 raft_consensus.cc:515] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:09.368624 17909 leader_election.cc:290] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [CANDIDATE]: Term 2 election: Requested vote from peers 61e0f20eba074de0a6eb1b1f14c0bafa (127.6.198.66:35901), 9d1296949a5742af94fa7e8b3388cee6 (127.6.198.67:36675)
I20260709 13:17:09.369517 17983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8aafdf099d44148d880851a873de56" candidate_uuid: "6533af55484e4160a399e4886ad5242e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa"
I20260709 13:17:09.370049 17983 raft_consensus.cc:3060] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:09.370110 18059 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c8aafdf099d44148d880851a873de56" candidate_uuid: "6533af55484e4160a399e4886ad5242e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9d1296949a5742af94fa7e8b3388cee6"
I20260709 13:17:09.370824 18059 raft_consensus.cc:3055] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:09.371148 18059 raft_consensus.cc:740] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9d1296949a5742af94fa7e8b3388cee6, State: Running, Role: LEADER
I20260709 13:17:09.371877 18059 consensus_queue.cc:260] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:09.372776 18059 raft_consensus.cc:3060] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:09.375025 17983 raft_consensus.cc:2468] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6533af55484e4160a399e4886ad5242e in term 2.
I20260709 13:17:09.376029 17875 leader_election.cc:304] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61e0f20eba074de0a6eb1b1f14c0bafa, 6533af55484e4160a399e4886ad5242e; no voters: 
I20260709 13:17:09.376770 18113 raft_consensus.cc:2804] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:09.377247 18113 raft_consensus.cc:697] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [term 2 LEADER]: Becoming Leader. State: Replica: 6533af55484e4160a399e4886ad5242e, State: Running, Role: LEADER
I20260709 13:17:09.377925 18113 consensus_queue.cc:237] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:09.378561 18059 raft_consensus.cc:2468] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6533af55484e4160a399e4886ad5242e in term 2.
I20260709 13:17:09.384183 17805 catalog_manager.cc:5697] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e reported cstate change: term changed from 1 to 2, leader changed from 9d1296949a5742af94fa7e8b3388cee6 (127.6.198.67) to 6533af55484e4160a399e4886ad5242e (127.6.198.65). New cstate: current_term: 2 leader_uuid: "6533af55484e4160a399e4886ad5242e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.543490 17983 raft_consensus.cc:1275] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 FOLLOWER]: Refusing update from remote peer 6533af55484e4160a399e4886ad5242e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:09.544673 18113 consensus_queue.cc:1048] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:09.551999 18059 raft_consensus.cc:1275] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 2 FOLLOWER]: Refusing update from remote peer 6533af55484e4160a399e4886ad5242e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:09.553762 18113 consensus_queue.cc:1048] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:09.591998  6937 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:17:09.593667  6937 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:17:09.594064  6937 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:17:09.605358 17909 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35e46bac39354a7c985b4d83d48bbc31"
dest_uuid: "6533af55484e4160a399e4886ad5242e"
mode: GRACEFUL
new_leader_uuid: "9d1296949a5742af94fa7e8b3388cee6"
 from {username='slave'} at 127.0.0.1:43770
I20260709 13:17:09.605878 17909 raft_consensus.cc:606] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 2 LEADER]: Received request to transfer leadership to TS 9d1296949a5742af94fa7e8b3388cee6
I20260709 13:17:09.606948  6937 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:17:09.678164 17983 raft_consensus.cc:1275] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 FOLLOWER]: Refusing update from remote peer 6533af55484e4160a399e4886ad5242e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:09.679371 18113 consensus_queue.cc:1048] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:09.687157 18059 raft_consensus.cc:1275] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 2 FOLLOWER]: Refusing update from remote peer 6533af55484e4160a399e4886ad5242e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:09.688553 18154 consensus_queue.cc:1048] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:09.695750 18141 raft_consensus.cc:993] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e: : Instructing follower 9d1296949a5742af94fa7e8b3388cee6 to start an election
I20260709 13:17:09.696194 18141 raft_consensus.cc:1081] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 2 LEADER]: Signalling peer 9d1296949a5742af94fa7e8b3388cee6 to start an election
I20260709 13:17:09.697702 18059 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "35e46bac39354a7c985b4d83d48bbc31"
dest_uuid: "9d1296949a5742af94fa7e8b3388cee6"
 from {username='slave'} at 127.0.0.1:58478
I20260709 13:17:09.698289 18059 raft_consensus.cc:493] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:09.698741 18059 raft_consensus.cc:3060] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:09.703682 18059 raft_consensus.cc:515] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:09.705191 18059 leader_election.cc:290] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [CANDIDATE]: Term 3 election: Requested vote from peers 6533af55484e4160a399e4886ad5242e (127.6.198.65:37765), 61e0f20eba074de0a6eb1b1f14c0bafa (127.6.198.66:35901)
I20260709 13:17:09.706090 17909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e46bac39354a7c985b4d83d48bbc31" candidate_uuid: "9d1296949a5742af94fa7e8b3388cee6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6533af55484e4160a399e4886ad5242e"
I20260709 13:17:09.706748 17909 raft_consensus.cc:3055] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:09.706979 17983 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35e46bac39354a7c985b4d83d48bbc31" candidate_uuid: "9d1296949a5742af94fa7e8b3388cee6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa"
I20260709 13:17:09.707099 17909 raft_consensus.cc:740] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6533af55484e4160a399e4886ad5242e, State: Running, Role: LEADER
I20260709 13:17:09.707690 17983 raft_consensus.cc:3060] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:09.708413 17909 consensus_queue.cc:260] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:09.709800 17909 raft_consensus.cc:3060] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:09.712898 17983 raft_consensus.cc:2468] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d1296949a5742af94fa7e8b3388cee6 in term 3.
I20260709 13:17:09.713968 18024 leader_election.cc:304] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61e0f20eba074de0a6eb1b1f14c0bafa, 9d1296949a5742af94fa7e8b3388cee6; no voters: 
I20260709 13:17:09.714658 18105 raft_consensus.cc:2804] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:09.714713 17909 raft_consensus.cc:2468] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d1296949a5742af94fa7e8b3388cee6 in term 3.
I20260709 13:17:09.715188 18105 raft_consensus.cc:697] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 3 LEADER]: Becoming Leader. State: Replica: 9d1296949a5742af94fa7e8b3388cee6, State: Running, Role: LEADER
I20260709 13:17:09.716063 18105 consensus_queue.cc:237] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } }
I20260709 13:17:09.721467 17805 catalog_manager.cc:5697] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 reported cstate change: term changed from 2 to 3, leader changed from 6533af55484e4160a399e4886ad5242e (127.6.198.65) to 9d1296949a5742af94fa7e8b3388cee6 (127.6.198.67). New cstate: current_term: 3 leader_uuid: "9d1296949a5742af94fa7e8b3388cee6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6533af55484e4160a399e4886ad5242e" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 37765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61e0f20eba074de0a6eb1b1f14c0bafa" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d1296949a5742af94fa7e8b3388cee6" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 36675 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:09.740116  6937 tablet_server.cc:179] TabletServer@127.6.198.65:0 shutting down...
I20260709 13:17:09.759533  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:09.760201  6937 tablet_replica.cc:333] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e: stopping tablet replica
I20260709 13:17:09.760798  6937 raft_consensus.cc:2243] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:09.761422  6937 pending_rounds.cc:70] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e: Trying to abort 1 pending ops.
I20260709 13:17:09.761651  6937 pending_rounds.cc:77] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305638008340054016 op_type: NO_OP noop_request { }
I20260709 13:17:09.761946  6937 raft_consensus.cc:2889] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:09.762218  6937 raft_consensus.cc:2272] T 5c8aafdf099d44148d880851a873de56 P 6533af55484e4160a399e4886ad5242e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:09.764355  6937 tablet_replica.cc:333] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e: stopping tablet replica
I20260709 13:17:09.764878  6937 raft_consensus.cc:2243] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:09.765293  6937 raft_consensus.cc:2272] T 35e46bac39354a7c985b4d83d48bbc31 P 6533af55484e4160a399e4886ad5242e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:09.767359  6937 tablet_replica.cc:333] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e: stopping tablet replica
I20260709 13:17:09.767859  6937 raft_consensus.cc:2243] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:09.768594  6937 raft_consensus.cc:2272] T 14f04716dbfd4058937365b4822995f4 P 6533af55484e4160a399e4886ad5242e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:09.770700  6937 tablet_replica.cc:333] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e: stopping tablet replica
I20260709 13:17:09.771188  6937 raft_consensus.cc:2243] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:09.771697  6937 raft_consensus.cc:2272] T c1aa673d8841432ba7d3a56cf33711aa P 6533af55484e4160a399e4886ad5242e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:09.784756  6937 tablet_server.cc:196] TabletServer@127.6.198.65:0 shutdown complete.
I20260709 13:17:09.796628  6937 tablet_server.cc:179] TabletServer@127.6.198.66:0 shutting down...
I20260709 13:17:09.812645  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:09.813277  6937 tablet_replica.cc:333] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa: stopping tablet replica
I20260709 13:17:09.813822  6937 raft_consensus.cc:2243] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:09.814273  6937 raft_consensus.cc:2272] T 5c8aafdf099d44148d880851a873de56 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:09.816723  6937 tablet_replica.cc:333] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa: stopping tablet replica
I20260709 13:17:09.817273  6937 raft_consensus.cc:2243] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:09.817694  6937 raft_consensus.cc:2272] T 35e46bac39354a7c985b4d83d48bbc31 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:09.819911  6937 tablet_replica.cc:333] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa: stopping tablet replica
I20260709 13:17:09.820397  6937 raft_consensus.cc:2243] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:09.820802  6937 raft_consensus.cc:2272] T 14f04716dbfd4058937365b4822995f4 P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:09.823175  6937 tablet_replica.cc:333] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa: stopping tablet replica
I20260709 13:17:09.823664  6937 raft_consensus.cc:2243] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:09.824452  6937 raft_consensus.cc:2272] T c1aa673d8841432ba7d3a56cf33711aa P 61e0f20eba074de0a6eb1b1f14c0bafa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:09.846457  6937 tablet_server.cc:196] TabletServer@127.6.198.66:0 shutdown complete.
I20260709 13:17:09.858227  6937 tablet_server.cc:179] TabletServer@127.6.198.67:0 shutting down...
I20260709 13:17:09.874524  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:09.875136  6937 tablet_replica.cc:333] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6: stopping tablet replica
I20260709 13:17:09.875677  6937 raft_consensus.cc:2243] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:09.876127  6937 raft_consensus.cc:2272] T 5c8aafdf099d44148d880851a873de56 P 9d1296949a5742af94fa7e8b3388cee6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:09.878451  6937 tablet_replica.cc:333] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6: stopping tablet replica
I20260709 13:17:09.878985  6937 raft_consensus.cc:2243] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:09.879577  6937 pending_rounds.cc:70] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6: Trying to abort 1 pending ops.
I20260709 13:17:09.879737  6937 pending_rounds.cc:77] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305638009723990016 op_type: NO_OP noop_request { }
I20260709 13:17:09.880019  6937 raft_consensus.cc:2889] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:09.880334  6937 raft_consensus.cc:2272] T 35e46bac39354a7c985b4d83d48bbc31 P 9d1296949a5742af94fa7e8b3388cee6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:09.882539  6937 tablet_replica.cc:333] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6: stopping tablet replica
I20260709 13:17:09.883080  6937 raft_consensus.cc:2243] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:09.883435  6937 raft_consensus.cc:2272] T c1aa673d8841432ba7d3a56cf33711aa P 9d1296949a5742af94fa7e8b3388cee6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:09.885399  6937 tablet_replica.cc:333] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6: stopping tablet replica
I20260709 13:17:09.885813  6937 raft_consensus.cc:2243] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:09.886220  6937 raft_consensus.cc:2272] T 14f04716dbfd4058937365b4822995f4 P 9d1296949a5742af94fa7e8b3388cee6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:09.908499  6937 tablet_server.cc:196] TabletServer@127.6.198.67:0 shutdown complete.
I20260709 13:17:09.919516  6937 master.cc:562] Master@127.6.198.126:41679 shutting down...
I20260709 13:17:09.933960  6937 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:09.934514  6937 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:09.934783  6937 tablet_replica.cc:333] T 00000000000000000000000000000000 P f5831f2d82844288b78d489a742ddc77: stopping tablet replica
I20260709 13:17:09.953759  6937 master.cc:584] Master@127.6.198.126:41679 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2508 ms)
[----------] 6 tests from LeaderRebalancerTest (310829 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:17:09.980055  6937 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.198.126:44927
I20260709 13:17:09.980988  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:09.985607 18158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:09.986593 18157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:09.987160 18160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:09.987821  6937 server_base.cc:1061] running on GCE node
I20260709 13:17:09.988540  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:09.988742  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:09.988898  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603029988881 us; error 0 us; skew 500 ppm
I20260709 13:17:09.989411  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:09.991619  6937 webserver.cc:533] Webserver started at http://127.6.198.126:35129/ using document root <none> and password file <none>
I20260709 13:17:09.992086  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:09.992285  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:09.992554  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:09.993535  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/master-0-root/instance:
uuid: "97ea9e0a79d04bd691f7c46dd5df9755"
format_stamp: "Formatted at 2026-07-09 13:17:09 on dist-test-slave-tqzw"
I20260709 13:17:09.997665  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:17:10.000980 18165 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:10.001840  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:17:10.002076  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/master-0-root
uuid: "97ea9e0a79d04bd691f7c46dd5df9755"
format_stamp: "Formatted at 2026-07-09 13:17:09 on dist-test-slave-tqzw"
I20260709 13:17:10.002308  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:10.010530  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:10.011375  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:10.046125  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.126:44927
I20260709 13:17:10.046221 18216 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.126:44927 every 8 connection(s)
I20260709 13:17:10.050028 18217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.060070 18217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755: Bootstrap starting.
I20260709 13:17:10.064303 18217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.068053 18217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755: No bootstrap required, opened a new log
I20260709 13:17:10.069947 18217 raft_consensus.cc:359] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ea9e0a79d04bd691f7c46dd5df9755" member_type: VOTER }
I20260709 13:17:10.070319 18217 raft_consensus.cc:385] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:10.070590 18217 raft_consensus.cc:740] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97ea9e0a79d04bd691f7c46dd5df9755, State: Initialized, Role: FOLLOWER
I20260709 13:17:10.071152 18217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ea9e0a79d04bd691f7c46dd5df9755" member_type: VOTER }
I20260709 13:17:10.071591 18217 raft_consensus.cc:399] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:10.071828 18217 raft_consensus.cc:493] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:10.072152 18217 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:10.076444 18217 raft_consensus.cc:515] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ea9e0a79d04bd691f7c46dd5df9755" member_type: VOTER }
I20260709 13:17:10.076920 18217 leader_election.cc:304] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97ea9e0a79d04bd691f7c46dd5df9755; no voters: 
I20260709 13:17:10.078019 18217 leader_election.cc:290] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:10.078399 18220 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:10.079741 18220 raft_consensus.cc:697] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [term 1 LEADER]: Becoming Leader. State: Replica: 97ea9e0a79d04bd691f7c46dd5df9755, State: Running, Role: LEADER
I20260709 13:17:10.080374 18220 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ea9e0a79d04bd691f7c46dd5df9755" member_type: VOTER }
I20260709 13:17:10.080932 18217 sys_catalog.cc:565] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:10.083060 18221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97ea9e0a79d04bd691f7c46dd5df9755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ea9e0a79d04bd691f7c46dd5df9755" member_type: VOTER } }
I20260709 13:17:10.083130 18222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97ea9e0a79d04bd691f7c46dd5df9755. Latest consensus state: current_term: 1 leader_uuid: "97ea9e0a79d04bd691f7c46dd5df9755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97ea9e0a79d04bd691f7c46dd5df9755" member_type: VOTER } }
I20260709 13:17:10.083760 18221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:10.083801 18222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:10.087225 18225 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:10.094817 18225 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:10.095755  6937 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:10.103114 18225 catalog_manager.cc:1383] Generated new cluster ID: 5157f270204f4117aea53ea6d6ed1433
I20260709 13:17:10.103355 18225 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:10.144510 18225 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:10.145718 18225 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:10.156068 18225 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755: Generated new TSK 0
I20260709 13:17:10.156592 18225 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:10.162062  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:10.167521 18238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:10.168994 18239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:10.171140  6937 server_base.cc:1061] running on GCE node
W20260709 13:17:10.172298 18241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:10.173146  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:10.173338  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:10.173461  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603030173450 us; error 0 us; skew 500 ppm
I20260709 13:17:10.173904  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:10.176087  6937 webserver.cc:533] Webserver started at http://127.6.198.65:36073/ using document root <none> and password file <none>
I20260709 13:17:10.176517  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:10.176666  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:10.176868  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:10.177826  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-0-root/instance:
uuid: "88766a58641e46449f46227d9e09bd68"
format_stamp: "Formatted at 2026-07-09 13:17:10 on dist-test-slave-tqzw"
I20260709 13:17:10.181972  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:17:10.185165 18246 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:10.185878  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:17:10.186165  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-0-root
uuid: "88766a58641e46449f46227d9e09bd68"
format_stamp: "Formatted at 2026-07-09 13:17:10 on dist-test-slave-tqzw"
I20260709 13:17:10.186472  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:10.200335  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:10.201411  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:10.202767  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:10.205101  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:10.205329  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:10.205569  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:10.205737  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:10.242005  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.65:38301
I20260709 13:17:10.242105 18308 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.65:38301 every 8 connection(s)
I20260709 13:17:10.246923  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:10.253935 18313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:10.256645 18314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:10.258031 18316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:10.258838  6937 server_base.cc:1061] running on GCE node
I20260709 13:17:10.259598 18309 heartbeater.cc:344] Connected to a master server at 127.6.198.126:44927
I20260709 13:17:10.259893  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:17:10.260012 18309 heartbeater.cc:461] Registering TS with master...
W20260709 13:17:10.260192  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:10.260407  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603030260388 us; error 0 us; skew 500 ppm
I20260709 13:17:10.260875 18309 heartbeater.cc:507] Master 127.6.198.126:44927 requested a full tablet report, sending...
I20260709 13:17:10.261170  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:10.263242 18182 ts_manager.cc:194] Registered new tserver with Master: 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301)
I20260709 13:17:10.264408  6937 webserver.cc:533] Webserver started at http://127.6.198.66:38393/ using document root <none> and password file <none>
I20260709 13:17:10.265049  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:10.265245  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:10.265506  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:10.265535 18182 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50596
I20260709 13:17:10.266716  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-1-root/instance:
uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
format_stamp: "Formatted at 2026-07-09 13:17:10 on dist-test-slave-tqzw"
I20260709 13:17:10.270996  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:17:10.273988 18321 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:10.274748  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:17:10.275030  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-1-root
uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
format_stamp: "Formatted at 2026-07-09 13:17:10 on dist-test-slave-tqzw"
I20260709 13:17:10.275311  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:10.288249  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:10.289186  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:10.290668  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:10.292727  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:10.292943  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:10.293165  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:10.293331  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:10.330078  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.66:35643
I20260709 13:17:10.330194 18383 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.66:35643 every 8 connection(s)
I20260709 13:17:10.334792  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:10.341522 18388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:10.342165 18387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:10.344913 18390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:10.345767 18384 heartbeater.cc:344] Connected to a master server at 127.6.198.126:44927
I20260709 13:17:10.346083  6937 server_base.cc:1061] running on GCE node
I20260709 13:17:10.346189 18384 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:10.347256 18384 heartbeater.cc:507] Master 127.6.198.126:44927 requested a full tablet report, sending...
I20260709 13:17:10.347265  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:10.347711  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:10.347925  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603030347907 us; error 0 us; skew 500 ppm
I20260709 13:17:10.348624  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:10.349860 18182 ts_manager.cc:194] Registered new tserver with Master: 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:10.351480  6937 webserver.cc:533] Webserver started at http://127.6.198.67:44301/ using document root <none> and password file <none>
I20260709 13:17:10.351835 18182 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50608
I20260709 13:17:10.352124  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:10.352430  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:10.352754  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:10.354180  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-2-root/instance:
uuid: "0a3115e6843944079e3fad8d05197029"
format_stamp: "Formatted at 2026-07-09 13:17:10 on dist-test-slave-tqzw"
I20260709 13:17:10.358470  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:17:10.361421 18395 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:10.362166  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:17:10.362474  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-2-root
uuid: "0a3115e6843944079e3fad8d05197029"
format_stamp: "Formatted at 2026-07-09 13:17:10 on dist-test-slave-tqzw"
I20260709 13:17:10.362703  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602718985547-6937-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:10.380460  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:10.381362  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:10.382864  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:10.385020  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:10.385188  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:10.385479  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:10.385636  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:10.422711  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.67:41093
I20260709 13:17:10.422820 18457 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.67:41093 every 8 connection(s)
I20260709 13:17:10.435842 18458 heartbeater.cc:344] Connected to a master server at 127.6.198.126:44927
I20260709 13:17:10.436311 18458 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:10.436985 18458 heartbeater.cc:507] Master 127.6.198.126:44927 requested a full tablet report, sending...
I20260709 13:17:10.438865 18182 ts_manager.cc:194] Registered new tserver with Master: 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:10.439208  6937 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013313044s
I20260709 13:17:10.440305 18182 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50612
I20260709 13:17:10.462865 18182 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50614:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:10.465402 18182 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:10.538092 18274 tablet_service.cc:1511] Processing CreateTablet for tablet 86bd5ede009f4c36813af6db8e97baa6 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:10.539541 18274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86bd5ede009f4c36813af6db8e97baa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.539237 18349 tablet_service.cc:1511] Processing CreateTablet for tablet 86bd5ede009f4c36813af6db8e97baa6 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:10.540150 18423 tablet_service.cc:1511] Processing CreateTablet for tablet 86bd5ede009f4c36813af6db8e97baa6 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:10.540565 18349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86bd5ede009f4c36813af6db8e97baa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.541461 18423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86bd5ede009f4c36813af6db8e97baa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.543380 18273 tablet_service.cc:1511] Processing CreateTablet for tablet 0d7e9ac119d140b091ea674bbe3f7a71 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:10.543879 18422 tablet_service.cc:1511] Processing CreateTablet for tablet 0d7e9ac119d140b091ea674bbe3f7a71 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:10.544770 18273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d7e9ac119d140b091ea674bbe3f7a71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.544400 18347 tablet_service.cc:1511] Processing CreateTablet for tablet 9ab85571c50949298324235b64c91bac (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:10.545732 18347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ab85571c50949298324235b64c91bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.546000 18421 tablet_service.cc:1511] Processing CreateTablet for tablet 9ab85571c50949298324235b64c91bac (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:10.547385 18272 tablet_service.cc:1511] Processing CreateTablet for tablet 9ab85571c50949298324235b64c91bac (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:10.548118 18420 tablet_service.cc:1511] Processing CreateTablet for tablet ad42bcd434774d8c8cbc5dd5188ba040 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:10.548465 18345 tablet_service.cc:1511] Processing CreateTablet for tablet c8480c976dc449d0b38cc0e892e699c2 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:10.549400 18418 tablet_service.cc:1511] Processing CreateTablet for tablet c338bb3d5e064f07bc473134a4090cb4 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:10.549911 18271 tablet_service.cc:1511] Processing CreateTablet for tablet ad42bcd434774d8c8cbc5dd5188ba040 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:10.550446 18419 tablet_service.cc:1511] Processing CreateTablet for tablet c8480c976dc449d0b38cc0e892e699c2 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:10.551179 18344 tablet_service.cc:1511] Processing CreateTablet for tablet c338bb3d5e064f07bc473134a4090cb4 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:10.551821 18419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8480c976dc449d0b38cc0e892e699c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.552067 18270 tablet_service.cc:1511] Processing CreateTablet for tablet c8480c976dc449d0b38cc0e892e699c2 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:10.554659 18418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c338bb3d5e064f07bc473134a4090cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.549825 18345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8480c976dc449d0b38cc0e892e699c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.555536 18269 tablet_service.cc:1511] Processing CreateTablet for tablet c338bb3d5e064f07bc473134a4090cb4 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:10.556932 18420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad42bcd434774d8c8cbc5dd5188ba040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.543380 18348 tablet_service.cc:1511] Processing CreateTablet for tablet 0d7e9ac119d140b091ea674bbe3f7a71 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:10.557778 18421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ab85571c50949298324235b64c91bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.557962 18348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d7e9ac119d140b091ea674bbe3f7a71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.558715 18344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c338bb3d5e064f07bc473134a4090cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.547385 18346 tablet_service.cc:1511] Processing CreateTablet for tablet ad42bcd434774d8c8cbc5dd5188ba040 (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:10.559803 18346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad42bcd434774d8c8cbc5dd5188ba040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.562700 18422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d7e9ac119d140b091ea674bbe3f7a71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.564100 18267 tablet_service.cc:1511] Processing CreateTablet for tablet a84920b1dc0243028f50c02f24fb1f8b (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:10.566736 18266 tablet_service.cc:1511] Processing CreateTablet for tablet 1d220a74f8fc451ca4c52f0d3143131f (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:10.562723 18268 tablet_service.cc:1511] Processing CreateTablet for tablet c5ac76c3dcf5452db28d5619ff1c913b (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:10.566708 18343 tablet_service.cc:1511] Processing CreateTablet for tablet c5ac76c3dcf5452db28d5619ff1c913b (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:10.570145 18343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ac76c3dcf5452db28d5619ff1c913b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.571702 18417 tablet_service.cc:1511] Processing CreateTablet for tablet c5ac76c3dcf5452db28d5619ff1c913b (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:10.572988 18417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ac76c3dcf5452db28d5619ff1c913b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.573071 18342 tablet_service.cc:1511] Processing CreateTablet for tablet a84920b1dc0243028f50c02f24fb1f8b (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:10.574370 18342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a84920b1dc0243028f50c02f24fb1f8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.548724 18272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ab85571c50949298324235b64c91bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.576845 18270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8480c976dc449d0b38cc0e892e699c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.578006 18269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c338bb3d5e064f07bc473134a4090cb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.579401 18266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d220a74f8fc451ca4c52f0d3143131f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.580761 18271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad42bcd434774d8c8cbc5dd5188ba040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.582056 18267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a84920b1dc0243028f50c02f24fb1f8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.584467 18268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5ac76c3dcf5452db28d5619ff1c913b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.599399 18416 tablet_service.cc:1511] Processing CreateTablet for tablet a84920b1dc0243028f50c02f24fb1f8b (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:10.601250 18416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a84920b1dc0243028f50c02f24fb1f8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.601806 18415 tablet_service.cc:1511] Processing CreateTablet for tablet 1d220a74f8fc451ca4c52f0d3143131f (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:10.603088 18415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d220a74f8fc451ca4c52f0d3143131f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.599764 18341 tablet_service.cc:1511] Processing CreateTablet for tablet 1d220a74f8fc451ca4c52f0d3143131f (DEFAULT_TABLE table=test-workload [id=36e3686a46814f4e9660c99b7ae137c2]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:10.622809 18341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d220a74f8fc451ca4c52f0d3143131f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:10.703886 18473 tablet_bootstrap.cc:492] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:10.719708 18474 tablet_bootstrap.cc:492] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:10.727147 18473 tablet_bootstrap.cc:654] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.729777 18474 tablet_bootstrap.cc:654] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.732012 18475 tablet_bootstrap.cc:492] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:10.738022 18475 tablet_bootstrap.cc:654] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.738734 18473 tablet_bootstrap.cc:492] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:10.739354 18473 ts_tablet_manager.cc:1403] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.036s	user 0.011s	sys 0.007s
I20260709 13:17:10.741242 18474 tablet_bootstrap.cc:492] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:10.741714 18474 ts_tablet_manager.cc:1403] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.022s	user 0.002s	sys 0.011s
I20260709 13:17:10.742329 18475 tablet_bootstrap.cc:492] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:10.742172 18473 raft_consensus.cc:359] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } }
I20260709 13:17:10.742841 18475 ts_tablet_manager.cc:1403] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.011s	user 0.005s	sys 0.005s
I20260709 13:17:10.743009 18473 raft_consensus.cc:385] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:10.743377 18473 raft_consensus.cc:740] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:10.744238 18473 consensus_queue.cc:260] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } }
I20260709 13:17:10.744491 18474 raft_consensus.cc:359] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.745183 18474 raft_consensus.cc:385] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:10.745527 18474 raft_consensus.cc:740] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:10.745767 18475 raft_consensus.cc:359] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.746707 18475 raft_consensus.cc:385] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:10.747071 18475 raft_consensus.cc:740] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:10.747339 18458 heartbeater.cc:499] Master 127.6.198.126:44927 was elected leader, sending a full tablet report...
I20260709 13:17:10.747704 18473 ts_tablet_manager.cc:1434] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:17:10.747828 18475 consensus_queue.cc:260] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.748726 18473 tablet_bootstrap.cc:492] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:10.746448 18474 consensus_queue.cc:260] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.752038 18384 heartbeater.cc:499] Master 127.6.198.126:44927 was elected leader, sending a full tablet report...
I20260709 13:17:10.754521 18473 tablet_bootstrap.cc:654] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.759009 18474 ts_tablet_manager.cc:1434] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.010s
I20260709 13:17:10.759485 18309 heartbeater.cc:499] Master 127.6.198.126:44927 was elected leader, sending a full tablet report...
I20260709 13:17:10.760177 18474 tablet_bootstrap.cc:492] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:10.764331 18473 tablet_bootstrap.cc:492] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:10.764879 18473 ts_tablet_manager.cc:1403] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.016s	user 0.002s	sys 0.009s
I20260709 13:17:10.766566 18474 tablet_bootstrap.cc:654] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.767567 18473 raft_consensus.cc:359] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.768155 18473 raft_consensus.cc:385] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:10.768395 18473 raft_consensus.cc:740] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:10.769038 18473 consensus_queue.cc:260] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.771171 18473 ts_tablet_manager.cc:1434] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:17:10.771989 18475 ts_tablet_manager.cc:1434] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:17:10.773574 18475 tablet_bootstrap.cc:492] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:10.772117 18473 tablet_bootstrap.cc:492] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:10.779997 18474 tablet_bootstrap.cc:492] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:10.780560 18474 ts_tablet_manager.cc:1403] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.021s	user 0.006s	sys 0.012s
I20260709 13:17:10.781167 18473 tablet_bootstrap.cc:654] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.780583 18475 tablet_bootstrap.cc:654] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.783808 18474 raft_consensus.cc:359] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.784739 18474 raft_consensus.cc:385] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:10.785102 18474 raft_consensus.cc:740] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:10.786015 18474 consensus_queue.cc:260] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.788779 18474 ts_tablet_manager.cc:1434] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:17:10.789811 18474 tablet_bootstrap.cc:492] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:10.789863 18475 tablet_bootstrap.cc:492] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:10.790427 18475 ts_tablet_manager.cc:1403] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.007s
I20260709 13:17:10.793226 18475 raft_consensus.cc:359] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.793900 18475 raft_consensus.cc:385] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:10.794176 18475 raft_consensus.cc:740] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:10.795536 18475 consensus_queue.cc:260] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.797158 18474 tablet_bootstrap.cc:654] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.797643 18475 ts_tablet_manager.cc:1434] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 13:17:10.798614 18475 tablet_bootstrap.cc:492] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:10.800443 18473 tablet_bootstrap.cc:492] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:10.800966 18473 ts_tablet_manager.cc:1403] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.011s
I20260709 13:17:10.803611 18473 raft_consensus.cc:359] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.804296 18473 raft_consensus.cc:385] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:10.804600 18473 raft_consensus.cc:740] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:10.805287 18473 consensus_queue.cc:260] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.806092 18475 tablet_bootstrap.cc:654] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.807739 18473 ts_tablet_manager.cc:1434] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:17:10.808776 18473 tablet_bootstrap.cc:492] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:10.812362 18474 tablet_bootstrap.cc:492] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:10.812988 18474 ts_tablet_manager.cc:1403] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.003s
I20260709 13:17:10.814633 18473 tablet_bootstrap.cc:654] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.815533 18475 tablet_bootstrap.cc:492] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:10.816057 18475 ts_tablet_manager.cc:1403] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.001s
I20260709 13:17:10.815976 18474 raft_consensus.cc:359] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.816829 18474 raft_consensus.cc:385] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:10.817128 18474 raft_consensus.cc:740] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:10.817849 18474 consensus_queue.cc:260] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.819711 18479 raft_consensus.cc:493] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.820247 18474 ts_tablet_manager.cc:1434] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:17:10.820209 18479 raft_consensus.cc:515] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.821352 18474 tablet_bootstrap.cc:492] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:10.827847 18474 tablet_bootstrap.cc:654] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.834748 18474 tablet_bootstrap.cc:492] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:10.835238 18474 ts_tablet_manager.cc:1403] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 13:17:10.837878 18479 leader_election.cc:290] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:10.838060 18474 raft_consensus.cc:359] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } }
I20260709 13:17:10.838359 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad42bcd434774d8c8cbc5dd5188ba040" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:10.838842 18474 raft_consensus.cc:385] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:10.839187 18474 raft_consensus.cc:740] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:10.839293 18359 raft_consensus.cc:2468] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:10.840008 18474 consensus_queue.cc:260] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } }
I20260709 13:17:10.841400 18399 leader_election.cc:304] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 
I20260709 13:17:10.842545 18474 ts_tablet_manager.cc:1434] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:17:10.843556 18474 tablet_bootstrap.cc:492] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:10.846411 18479 raft_consensus.cc:2804] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:10.846925 18479 raft_consensus.cc:493] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:10.847364 18479 raft_consensus.cc:3060] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:10.849496 18474 tablet_bootstrap.cc:654] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.849786 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad42bcd434774d8c8cbc5dd5188ba040" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:10.850510 18284 raft_consensus.cc:2468] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:10.819105 18475 raft_consensus.cc:359] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.852908 18475 raft_consensus.cc:385] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:10.853277 18475 raft_consensus.cc:740] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:10.854689 18475 consensus_queue.cc:260] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.861177 18475 ts_tablet_manager.cc:1434] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 13:17:10.862525 18475 tablet_bootstrap.cc:492] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:10.869076 18479 raft_consensus.cc:515] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.872370 18473 tablet_bootstrap.cc:492] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:10.872529 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad42bcd434774d8c8cbc5dd5188ba040" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:10.873150 18473 ts_tablet_manager.cc:1403] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.065s	user 0.021s	sys 0.028s
I20260709 13:17:10.873207 18284 raft_consensus.cc:3060] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:10.879251 18491 raft_consensus.cc:493] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.879746 18491 raft_consensus.cc:515] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.880163 18474 tablet_bootstrap.cc:492] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:10.880683 18474 ts_tablet_manager.cc:1403] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.037s	user 0.018s	sys 0.016s
I20260709 13:17:10.881079 18284 raft_consensus.cc:2468] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:10.882972 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab85571c50949298324235b64c91bac" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:10.883648 18475 tablet_bootstrap.cc:654] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.883706 18474 raft_consensus.cc:359] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
W20260709 13:17:10.884394 18399 leader_election.cc:343] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:10.884563 18474 raft_consensus.cc:385] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:10.884982 18474 raft_consensus.cc:740] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:10.877170 18473 raft_consensus.cc:359] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.882652 18283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab85571c50949298324235b64c91bac" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:10.881881 18491 leader_election.cc:290] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:10.886328 18473 raft_consensus.cc:385] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:10.886719 18473 raft_consensus.cc:740] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:10.885742 18398 leader_election.cc:304] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:10.887626 18491 raft_consensus.cc:2804] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:17:10.887809 18398 leader_election.cc:343] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:10.887626 18473 consensus_queue.cc:260] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.888520 18398 leader_election.cc:304] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a3115e6843944079e3fad8d05197029; no voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68
I20260709 13:17:10.889142 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad42bcd434774d8c8cbc5dd5188ba040" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:10.889801 18359 raft_consensus.cc:3060] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:10.892477 18479 leader_election.cc:290] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:10.894636 18479 raft_consensus.cc:2749] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:10.885741 18474 consensus_queue.cc:260] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.898777 18474 ts_tablet_manager.cc:1434] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.002s
I20260709 13:17:10.898805 18480 raft_consensus.cc:493] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.901437 18480 raft_consensus.cc:515] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.902814 18474 tablet_bootstrap.cc:492] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:10.903537 18491 raft_consensus.cc:697] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Becoming Leader. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:10.904888 18491 consensus_queue.cc:237] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.907330 18480 leader_election.cc:290] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:10.913717 18474 tablet_bootstrap.cc:654] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:10.926976 18481 raft_consensus.cc:493] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.927484 18481 raft_consensus.cc:515] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.932025 18481 leader_election.cc:290] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:10.969848 18359 raft_consensus.cc:2468] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:10.978691 18480 raft_consensus.cc:493] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:10.979230 18480 raft_consensus.cc:515] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:10.981705 18480 leader_election.cc:290] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
W20260709 13:17:10.989668 18398 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.65:38301, user_credentials={real_user=slave}} blocked reactor thread for 101886us
I20260709 13:17:11.004791 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8480c976dc449d0b38cc0e892e699c2" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
W20260709 13:17:11.006780 18250 leader_election.cc:343] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:11.029099 18474 tablet_bootstrap.cc:492] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:11.029675 18474 ts_tablet_manager.cc:1403] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.128s	user 0.028s	sys 0.047s
I20260709 13:17:11.031610 18433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8480c976dc449d0b38cc0e892e699c2" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
I20260709 13:17:11.033113 18474 raft_consensus.cc:359] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
W20260709 13:17:11.033643 18250 leader_election.cc:343] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:11.033919 18474 raft_consensus.cc:385] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.033255 18180 catalog_manager.cc:5697] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a3115e6843944079e3fad8d05197029 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "0a3115e6843944079e3fad8d05197029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:11.034289 18474 raft_consensus.cc:740] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.034276 18250 leader_election.cc:304] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88766a58641e46449f46227d9e09bd68; no voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c
I20260709 13:17:11.035545 18481 raft_consensus.cc:2749] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:11.035403 18474 consensus_queue.cc:260] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.038655 18433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e9ac119d140b091ea674bbe3f7a71" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
I20260709 13:17:11.038926 18481 raft_consensus.cc:493] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.039295 18433 raft_consensus.cc:2468] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 0.
I20260709 13:17:11.039364 18481 raft_consensus.cc:515] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } }
I20260709 13:17:11.040983 18325 leader_election.cc:304] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 
I20260709 13:17:11.041437 18481 leader_election.cc:290] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.041832 18480 raft_consensus.cc:2804] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:11.042178 18491 raft_consensus.cc:493] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.042266 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86bd5ede009f4c36813af6db8e97baa6" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:11.042784 18491 raft_consensus.cc:515] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } }
I20260709 13:17:11.042218 18480 raft_consensus.cc:493] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.043718 18480 raft_consensus.cc:3060] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.045523 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86bd5ede009f4c36813af6db8e97baa6" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:11.046856 18433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c338bb3d5e064f07bc473134a4090cb4" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
W20260709 13:17:11.046881 18399 leader_election.cc:343] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:11.049500 18325 leader_election.cc:343] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:11.051353 18250 leader_election.cc:343] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:11.051505 18491 leader_election.cc:290] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301)
I20260709 13:17:11.052762 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86bd5ede009f4c36813af6db8e97baa6" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:11.053547 18284 raft_consensus.cc:2468] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:11.054929 18433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86bd5ede009f4c36813af6db8e97baa6" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
I20260709 13:17:11.055428 18398 leader_election.cc:304] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 5f3f6d22d9624fc2a34acafd98a6892c
I20260709 13:17:11.055792 18433 raft_consensus.cc:2468] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 0.
I20260709 13:17:11.057039 18491 raft_consensus.cc:2804] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:11.057725 18491 raft_consensus.cc:493] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.054936 18480 raft_consensus.cc:515] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.058183 18491 raft_consensus.cc:3060] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.061376 18433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e9ac119d140b091ea674bbe3f7a71" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:11.062122 18433 raft_consensus.cc:3060] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.063133 18250 leader_election.cc:304] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 5f3f6d22d9624fc2a34acafd98a6892c
I20260709 13:17:11.064401 18481 raft_consensus.cc:2804] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:11.064766 18481 raft_consensus.cc:493] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.065097 18481 raft_consensus.cc:3060] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.068991 18491 raft_consensus.cc:515] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } }
I20260709 13:17:11.073212 18283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c338bb3d5e064f07bc473134a4090cb4" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:11.073282 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e9ac119d140b091ea674bbe3f7a71" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:11.074793 18475 tablet_bootstrap.cc:492] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:11.071012 18480 leader_election.cc:290] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.075330 18475 ts_tablet_manager.cc:1403] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.213s	user 0.049s	sys 0.049s
I20260709 13:17:11.073376 18473 ts_tablet_manager.cc:1434] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.200s	user 0.040s	sys 0.116s
W20260709 13:17:11.076166 18324 leader_election.cc:343] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:11.076514 18473 tablet_bootstrap.cc:492] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:11.077180 18324 leader_election.cc:304] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68
W20260709 13:17:11.078384 18324 leader_election.cc:343] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:11.078589 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86bd5ede009f4c36813af6db8e97baa6" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:11.076494 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e9ac119d140b091ea674bbe3f7a71" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:11.079948 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86bd5ede009f4c36813af6db8e97baa6" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
W20260709 13:17:11.080001 18399 leader_election.cc:343] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:11.080297 18324 leader_election.cc:343] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:11.080806 18491 leader_election.cc:290] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Requested vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301)
I20260709 13:17:11.081054 18475 raft_consensus.cc:359] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } }
I20260709 13:17:11.081885 18475 raft_consensus.cc:385] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.082180 18475 raft_consensus.cc:740] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.075342 18433 raft_consensus.cc:2468] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 1.
I20260709 13:17:11.082952 18481 raft_consensus.cc:515] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } }
I20260709 13:17:11.083117 18475 consensus_queue.cc:260] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } }
I20260709 13:17:11.083762 18325 leader_election.cc:304] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 88766a58641e46449f46227d9e09bd68
I20260709 13:17:11.084540 18282 raft_consensus.cc:2393] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a3115e6843944079e3fad8d05197029 in current term 1: Already voted for candidate 88766a58641e46449f46227d9e09bd68 in this term.
I20260709 13:17:11.085326 18481 leader_election.cc:290] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 election: Requested vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.081192 18480 raft_consensus.cc:2749] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:11.087430 18398 leader_election.cc:304] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a3115e6843944079e3fad8d05197029; no voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68
I20260709 13:17:11.087733 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86bd5ede009f4c36813af6db8e97baa6" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:11.088562 18359 raft_consensus.cc:3060] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.089114 18491 raft_consensus.cc:2749] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:11.089268 18505 raft_consensus.cc:2804] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:11.095459 18473 tablet_bootstrap.cc:654] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:11.091852 18474 ts_tablet_manager.cc:1434] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.062s	user 0.009s	sys 0.021s
I20260709 13:17:11.098686 18505 raft_consensus.cc:697] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Becoming Leader. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:11.099624 18505 consensus_queue.cc:237] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.093117 18475 ts_tablet_manager.cc:1434] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.003s
I20260709 13:17:11.101289 18359 raft_consensus.cc:2468] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 1.
I20260709 13:17:11.102208 18475 tablet_bootstrap.cc:492] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:11.103856 18474 tablet_bootstrap.cc:492] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:11.091359 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86bd5ede009f4c36813af6db8e97baa6" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:11.105929 18432 raft_consensus.cc:2393] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 88766a58641e46449f46227d9e09bd68 in current term 1: Already voted for candidate 0a3115e6843944079e3fad8d05197029 in this term.
I20260709 13:17:11.107494 18503 raft_consensus.cc:493] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.108033 18503 raft_consensus.cc:515] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.112002 18358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a84920b1dc0243028f50c02f24fb1f8b" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:11.113562 18474 tablet_bootstrap.cc:654] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:11.113608 18433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a84920b1dc0243028f50c02f24fb1f8b" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
W20260709 13:17:11.115027 18250 leader_election.cc:343] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:11.116078 18250 leader_election.cc:343] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:11.116477 18250 leader_election.cc:304] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88766a58641e46449f46227d9e09bd68; no voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c
I20260709 13:17:11.117290 18481 raft_consensus.cc:2749] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:11.110172 18503 leader_election.cc:290] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.119428 18250 leader_election.cc:304] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 0a3115e6843944079e3fad8d05197029
I20260709 13:17:11.120543 18503 raft_consensus.cc:2804] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:11.120749 18475 tablet_bootstrap.cc:654] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:11.121228 18503 raft_consensus.cc:697] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Becoming Leader. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:11.122092 18503 consensus_queue.cc:237] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } }
W20260709 13:17:11.124440 18230 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:11.133548 18475 tablet_bootstrap.cc:492] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:11.134052 18475 ts_tablet_manager.cc:1403] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 13:17:11.136816 18475 raft_consensus.cc:359] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.137534 18475 raft_consensus.cc:385] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.136358 18182 catalog_manager.cc:5697] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88766a58641e46449f46227d9e09bd68 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "88766a58641e46449f46227d9e09bd68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:11.137847 18475 raft_consensus.cc:740] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.144703 18475 consensus_queue.cc:260] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.146389 18475 ts_tablet_manager.cc:1434] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.001s
I20260709 13:17:11.147270 18475 tablet_bootstrap.cc:492] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:11.157670 18475 tablet_bootstrap.cc:654] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:11.166922 18475 tablet_bootstrap.cc:492] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:11.167131 18473 tablet_bootstrap.cc:492] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:11.167584 18475 ts_tablet_manager.cc:1403] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.009s
I20260709 13:17:11.168903 18474 tablet_bootstrap.cc:492] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:11.169414 18474 ts_tablet_manager.cc:1403] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.066s	user 0.017s	sys 0.041s
I20260709 13:17:11.167698 18473 ts_tablet_manager.cc:1403] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.091s	user 0.013s	sys 0.015s
I20260709 13:17:11.170717 18475 raft_consensus.cc:359] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.171612 18475 raft_consensus.cc:385] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.172027 18475 raft_consensus.cc:740] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.172443 18473 raft_consensus.cc:359] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.173219 18473 raft_consensus.cc:385] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.172871 18475 consensus_queue.cc:260] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.173032 18474 raft_consensus.cc:359] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.173620 18473 raft_consensus.cc:740] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.174021 18474 raft_consensus.cc:385] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.161517 18178 catalog_manager.cc:5697] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66). New cstate: current_term: 1 leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:11.174397 18474 raft_consensus.cc:740] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.174585 18473 consensus_queue.cc:260] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.212416 18474 consensus_queue.cc:260] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.214115 18475 ts_tablet_manager.cc:1434] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.000s
I20260709 13:17:11.216461 18473 ts_tablet_manager.cc:1434] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 13:17:11.217077 18475 tablet_bootstrap.cc:492] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:11.217577 18474 ts_tablet_manager.cc:1434] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.003s
I20260709 13:17:11.220340 18474 tablet_bootstrap.cc:492] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:11.222821 18481 raft_consensus.cc:493] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.223984 18481 raft_consensus.cc:515] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.228183 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ac76c3dcf5452db28d5619ff1c913b" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:11.229524 18475 tablet_bootstrap.cc:654] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:11.229547 18359 raft_consensus.cc:2468] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 0.
I20260709 13:17:11.230563 18473 tablet_bootstrap.cc:492] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:11.231742 18481 leader_election.cc:290] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.249294 18474 tablet_bootstrap.cc:654] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:11.268370 18473 tablet_bootstrap.cc:654] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:11.231835 18250 leader_election.cc:304] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:11.272294 18481 raft_consensus.cc:2804] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:11.272691 18481 raft_consensus.cc:493] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.273100 18481 raft_consensus.cc:3060] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.274873 18474 tablet_bootstrap.cc:492] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:11.274878 18475 tablet_bootstrap.cc:492] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:11.275473 18474 ts_tablet_manager.cc:1403] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.056s	user 0.016s	sys 0.018s
I20260709 13:17:11.275810 18475 ts_tablet_manager.cc:1403] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.044s
W20260709 13:17:11.275700 18250 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:35643, user_credentials={real_user=slave}} blocked reactor thread for 44158.8us
I20260709 13:17:11.278714 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ac76c3dcf5452db28d5619ff1c913b" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
W20260709 13:17:11.280606 18250 leader_election.cc:343] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:11.280809 18475 raft_consensus.cc:359] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.281703 18475 raft_consensus.cc:385] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.282061 18475 raft_consensus.cc:740] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.283166 18475 consensus_queue.cc:260] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.284369 18474 raft_consensus.cc:359] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.285125 18474 raft_consensus.cc:385] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.284969 18481 raft_consensus.cc:515] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.285526 18474 raft_consensus.cc:740] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.286532 18475 ts_tablet_manager.cc:1434] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:17:11.286553 18474 consensus_queue.cc:260] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.287189 18481 leader_election.cc:290] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 election: Requested vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.288302 18475 tablet_bootstrap.cc:492] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:11.289513 18474 ts_tablet_manager.cc:1434] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.004s
I20260709 13:17:11.293701 18473 tablet_bootstrap.cc:492] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:11.296813 18473 ts_tablet_manager.cc:1403] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.079s	user 0.016s	sys 0.007s
I20260709 13:17:11.296908 18475 tablet_bootstrap.cc:654] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:11.297560 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ac76c3dcf5452db28d5619ff1c913b" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:11.298287 18359 raft_consensus.cc:3060] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.299814 18474 tablet_bootstrap.cc:492] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:11.305001 18473 raft_consensus.cc:359] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.306162 18505 raft_consensus.cc:493] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.307934 18473 raft_consensus.cc:385] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.308311 18473 raft_consensus.cc:740] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.308151 18505 raft_consensus.cc:515] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.308838 18359 raft_consensus.cc:2468] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 1.
I20260709 13:17:11.309201 18473 consensus_queue.cc:260] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.311596 18505 leader_election.cc:290] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.307598 18475 tablet_bootstrap.cc:492] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:11.312531 18475 ts_tablet_manager.cc:1403] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.025s	user 0.005s	sys 0.014s
I20260709 13:17:11.313812 18503 raft_consensus.cc:493] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.314200 18474 tablet_bootstrap.cc:654] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:11.314554 18503 raft_consensus.cc:515] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.316872 18503 leader_election.cc:290] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.312342 18433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab85571c50949298324235b64c91bac" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
I20260709 13:17:11.316972 18475 raft_consensus.cc:359] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.317696 18433 raft_consensus.cc:2468] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 0.
I20260709 13:17:11.317873 18475 raft_consensus.cc:385] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.318250 18475 raft_consensus.cc:740] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.311905 18473 ts_tablet_manager.cc:1434] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.003s
I20260709 13:17:11.319290 18325 leader_election.cc:304] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 
I20260709 13:17:11.313063 18481 raft_consensus.cc:493] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.313243 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ac76c3dcf5452db28d5619ff1c913b" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:11.320091 18473 tablet_bootstrap.cc:492] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:11.313483 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab85571c50949298324235b64c91bac" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:11.321188 18474 tablet_bootstrap.cc:492] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:11.321512 18282 raft_consensus.cc:2468] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 0.
I20260709 13:17:11.321703 18474 ts_tablet_manager.cc:1403] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.004s
I20260709 13:17:11.322641 18475 consensus_queue.cc:260] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.321201 18505 raft_consensus.cc:2804] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:11.323717 18505 raft_consensus.cc:493] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.324100 18505 raft_consensus.cc:3060] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.325029 18475 ts_tablet_manager.cc:1434] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.003s
I20260709 13:17:11.326550 18475 tablet_bootstrap.cc:492] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:11.326670 18250 leader_election.cc:304] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:11.321223 18481 raft_consensus.cc:515] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.327558 18503 raft_consensus.cc:2804] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:11.331182 18474 raft_consensus.cc:359] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.331899 18474 raft_consensus.cc:385] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.332221 18474 raft_consensus.cc:740] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.332948 18474 consensus_queue.cc:260] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.334285 18475 tablet_bootstrap.cc:654] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:11.335400 18481 leader_election.cc:290] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.342744 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab85571c50949298324235b64c91bac" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:11.334287 18473 tablet_bootstrap.cc:654] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:11.348982 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab85571c50949298324235b64c91bac" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
I20260709 13:17:11.355525 18432 raft_consensus.cc:2468] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 0.
I20260709 13:17:11.350715 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d220a74f8fc451ca4c52f0d3143131f" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:11.351866 18433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d220a74f8fc451ca4c52f0d3143131f" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
I20260709 13:17:11.359869 18505 raft_consensus.cc:515] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
W20260709 13:17:11.352077 18250 leader_election.cc:343] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:11.352512 18491 consensus_queue.cc:1048] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:11.361251 18359 raft_consensus.cc:2393] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88766a58641e46449f46227d9e09bd68 in current term 1: Already voted for candidate 5f3f6d22d9624fc2a34acafd98a6892c in this term.
I20260709 13:17:11.363018 18505 leader_election.cc:290] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.363476 18503 raft_consensus.cc:697] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Becoming Leader. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:11.364409 18433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab85571c50949298324235b64c91bac" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:11.365002 18433 raft_consensus.cc:3060] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.364720 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab85571c50949298324235b64c91bac" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:11.365978 18282 raft_consensus.cc:3060] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.366045 18474 ts_tablet_manager.cc:1434] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.044s	user 0.018s	sys 0.015s
I20260709 13:17:11.368562 18503 consensus_queue.cc:237] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
W20260709 13:17:11.371285 18250 leader_election.cc:343] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:11.372227 18250 leader_election.cc:304] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 5f3f6d22d9624fc2a34acafd98a6892c
I20260709 13:17:11.372532 18475 tablet_bootstrap.cc:492] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:11.373016 18475 ts_tablet_manager.cc:1403] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.047s	user 0.008s	sys 0.020s
W20260709 13:17:11.373533 18250 leader_election.cc:343] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:11.374020 18250 leader_election.cc:304] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88766a58641e46449f46227d9e09bd68; no voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c
I20260709 13:17:11.374778 18481 raft_consensus.cc:2749] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:11.376542 18475 raft_consensus.cc:359] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.377338 18475 raft_consensus.cc:385] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.377595 18475 raft_consensus.cc:740] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.378466 18475 consensus_queue.cc:260] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.384486 18282 raft_consensus.cc:2468] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 1.
I20260709 13:17:11.385221 18520 raft_consensus.cc:2764] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:17:11.389693 18324 leader_election.cc:304] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:11.390254 18473 tablet_bootstrap.cc:492] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:11.390653 18505 raft_consensus.cc:2804] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:11.391255 18505 raft_consensus.cc:697] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Becoming Leader. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:11.392059 18475 ts_tablet_manager.cc:1434] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.002s
I20260709 13:17:11.390305 18178 catalog_manager.cc:5697] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88766a58641e46449f46227d9e09bd68 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "88766a58641e46449f46227d9e09bd68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:11.392342 18505 consensus_queue.cc:237] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.391052 18473 ts_tablet_manager.cc:1403] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.071s	user 0.014s	sys 0.007s
I20260709 13:17:11.397126 18433 raft_consensus.cc:2468] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 1.
I20260709 13:17:11.401710 18473 raft_consensus.cc:359] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.402814 18473 raft_consensus.cc:385] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.403291 18473 raft_consensus.cc:740] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.404546 18473 consensus_queue.cc:260] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.408648 18473 ts_tablet_manager.cc:1434] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:17:11.410115 18473 tablet_bootstrap.cc:492] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:11.417887 18473 tablet_bootstrap.cc:654] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:11.423035 18180 catalog_manager.cc:5697] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66). New cstate: current_term: 1 leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:11.423875 18505 raft_consensus.cc:493] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.424576 18505 raft_consensus.cc:515] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.428578 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8480c976dc449d0b38cc0e892e699c2" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:11.429064 18494 consensus_queue.cc:1048] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:11.429289 18282 raft_consensus.cc:2468] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 0.
I20260709 13:17:11.430591 18473 tablet_bootstrap.cc:492] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:11.430660 18324 leader_election.cc:304] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:11.431308 18473 ts_tablet_manager.cc:1403] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.013s
I20260709 13:17:11.431756 18506 raft_consensus.cc:2804] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:11.432202 18506 raft_consensus.cc:493] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.432534 18506 raft_consensus.cc:3060] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.435456 18473 raft_consensus.cc:359] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.436458 18473 raft_consensus.cc:385] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.436780 18473 raft_consensus.cc:740] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.437523 18473 consensus_queue.cc:260] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.439679 18473 ts_tablet_manager.cc:1434] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:17:11.442876 18506 raft_consensus.cc:515] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.444912 18433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8480c976dc449d0b38cc0e892e699c2" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
I20260709 13:17:11.445719 18473 tablet_bootstrap.cc:492] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:11.445668 18433 raft_consensus.cc:2468] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 0.
I20260709 13:17:11.443701 18505 leader_election.cc:290] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.449640 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8480c976dc449d0b38cc0e892e699c2" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:11.451077 18433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8480c976dc449d0b38cc0e892e699c2" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:11.451776 18433 raft_consensus.cc:3060] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.453258 18473 tablet_bootstrap.cc:654] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:11.454412 18284 raft_consensus.cc:3060] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.458868 18473 tablet_bootstrap.cc:492] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:11.459357 18473 ts_tablet_manager.cc:1403] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.014s	user 0.005s	sys 0.007s
I20260709 13:17:11.462188 18473 raft_consensus.cc:359] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.462819 18473 raft_consensus.cc:385] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:11.463092 18473 raft_consensus.cc:740] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:11.463949 18473 consensus_queue.cc:260] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.464820 18506 leader_election.cc:290] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.465579 18473 ts_tablet_manager.cc:1434] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:17:11.467923 18284 raft_consensus.cc:2468] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 1.
I20260709 13:17:11.469249 18324 leader_election.cc:304] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:11.470132 18505 raft_consensus.cc:2804] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:11.470655 18505 raft_consensus.cc:697] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Becoming Leader. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:11.471522 18505 consensus_queue.cc:237] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.467959 18506 raft_consensus.cc:493] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.473661 18506 raft_consensus.cc:515] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.467396 18433 raft_consensus.cc:2468] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 1.
I20260709 13:17:11.482904 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a84920b1dc0243028f50c02f24fb1f8b" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:11.483530 18433 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a84920b1dc0243028f50c02f24fb1f8b" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
I20260709 13:17:11.483899 18284 raft_consensus.cc:2468] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 0.
I20260709 13:17:11.484229 18433 raft_consensus.cc:2468] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 0.
I20260709 13:17:11.485064 18324 leader_election.cc:304] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:11.483582 18491 raft_consensus.cc:493] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.486024 18480 raft_consensus.cc:2804] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:11.486529 18480 raft_consensus.cc:493] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.486058 18491 raft_consensus.cc:515] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.486887 18480 raft_consensus.cc:3060] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.489643 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ac76c3dcf5452db28d5619ff1c913b" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:11.490216 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ac76c3dcf5452db28d5619ff1c913b" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:11.491091 18359 raft_consensus.cc:2393] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a3115e6843944079e3fad8d05197029 in current term 1: Already voted for candidate 88766a58641e46449f46227d9e09bd68 in this term.
I20260709 13:17:11.492542 18399 leader_election.cc:304] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a3115e6843944079e3fad8d05197029; no voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68
I20260709 13:17:11.493477 18494 raft_consensus.cc:3060] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.495294 18495 raft_consensus.cc:493] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.495805 18495 raft_consensus.cc:515] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.495895 18480 raft_consensus.cc:515] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.497885 18495 leader_election.cc:290] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:11.498802 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c338bb3d5e064f07bc473134a4090cb4" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:11.499468 18284 raft_consensus.cc:2468] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:11.499341 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a84920b1dc0243028f50c02f24fb1f8b" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:11.500679 18282 raft_consensus.cc:3060] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.500837 18398 leader_election.cc:304] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:11.502086 18480 leader_election.cc:290] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.502560 18520 raft_consensus.cc:493] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.503463 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c338bb3d5e064f07bc473134a4090cb4" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:11.503240 18520 raft_consensus.cc:515] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.504272 18359 raft_consensus.cc:2468] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:11.505019 18495 raft_consensus.cc:2804] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:11.505419 18495 raft_consensus.cc:493] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.505743 18495 raft_consensus.cc:3060] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.507565 18520 leader_election.cc:290] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.508671 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e9ac119d140b091ea674bbe3f7a71" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
I20260709 13:17:11.509610 18432 raft_consensus.cc:2393] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88766a58641e46449f46227d9e09bd68 in current term 1: Already voted for candidate 5f3f6d22d9624fc2a34acafd98a6892c in this term.
I20260709 13:17:11.512472 18494 raft_consensus.cc:2749] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:11.512771 18520 consensus_queue.cc:1048] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:11.513154 18491 leader_election.cc:290] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:11.514330 18506 leader_election.cc:290] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:11.515497 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a84920b1dc0243028f50c02f24fb1f8b" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:11.516100 18432 raft_consensus.cc:3060] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.518314 18282 raft_consensus.cc:2468] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 1.
I20260709 13:17:11.519976 18324 leader_election.cc:304] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:11.520187 18495 raft_consensus.cc:515] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.521253 18506 raft_consensus.cc:2804] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:11.521898 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e9ac119d140b091ea674bbe3f7a71" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:11.522446 18506 raft_consensus.cc:697] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Becoming Leader. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:11.526783 18506 consensus_queue.cc:237] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.524020 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c338bb3d5e064f07bc473134a4090cb4" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:11.527904 18357 raft_consensus.cc:3060] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.530464 18178 catalog_manager.cc:5697] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66). New cstate: current_term: 1 leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:11.533965 18432 raft_consensus.cc:2468] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 1.
I20260709 13:17:11.525746 18250 leader_election.cc:304] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88766a58641e46449f46227d9e09bd68; no voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c
I20260709 13:17:11.523917 18495 leader_election.cc:290] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:11.535477 18520 raft_consensus.cc:3060] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.535638 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c338bb3d5e064f07bc473134a4090cb4" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:11.536284 18282 raft_consensus.cc:3060] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.536278 18357 raft_consensus.cc:2468] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:11.539572 18399 leader_election.cc:304] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 
I20260709 13:17:11.545501 18282 raft_consensus.cc:2468] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:11.546556 18495 raft_consensus.cc:2804] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:11.547258 18495 raft_consensus.cc:697] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Becoming Leader. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:11.548049 18495 consensus_queue.cc:237] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.558070 18503 consensus_queue.cc:1048] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:11.563165 18520 raft_consensus.cc:2749] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:11.568969 18178 catalog_manager.cc:5697] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a3115e6843944079e3fad8d05197029 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "0a3115e6843944079e3fad8d05197029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:11.572461 18180 catalog_manager.cc:5697] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66). New cstate: current_term: 1 leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:11.645151 18506 consensus_queue.cc:1048] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:11.646098 18495 raft_consensus.cc:493] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:11.646638 18495 raft_consensus.cc:515] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.649168 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d220a74f8fc451ca4c52f0d3143131f" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:11.649567 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d220a74f8fc451ca4c52f0d3143131f" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:11.649807 18284 raft_consensus.cc:2468] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:11.650288 18357 raft_consensus.cc:2468] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:11.651206 18398 leader_election.cc:304] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:11.652083 18491 raft_consensus.cc:2804] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:11.652423 18491 raft_consensus.cc:493] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:11.652748 18491 raft_consensus.cc:3060] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.658690 18480 consensus_queue.cc:1048] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:11.651578 18495 leader_election.cc:290] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:11.664741 18491 raft_consensus.cc:515] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.669263 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d220a74f8fc451ca4c52f0d3143131f" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:11.669852 18282 raft_consensus.cc:3060] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.669915 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d220a74f8fc451ca4c52f0d3143131f" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:11.670701 18357 raft_consensus.cc:3060] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:11.672425 18491 leader_election.cc:290] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:11.676807 18282 raft_consensus.cc:2468] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:11.677841 18398 leader_election.cc:304] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:11.677929 18357 raft_consensus.cc:2468] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:11.678547 18491 raft_consensus.cc:2804] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:11.679052 18491 raft_consensus.cc:697] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Becoming Leader. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:11.679864 18491 consensus_queue.cc:237] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:11.687230 18178 catalog_manager.cc:5697] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a3115e6843944079e3fad8d05197029 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "0a3115e6843944079e3fad8d05197029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:11.815697 18480 consensus_queue.cc:1048] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:11.825625 18505 consensus_queue.cc:1048] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:11.928035 18520 consensus_queue.cc:1048] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:11.945550 18520 consensus_queue.cc:1048] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:11.964339 18357 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d7e9ac119d140b091ea674bbe3f7a71"
dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
mode: GRACEFUL
new_leader_uuid: "88766a58641e46449f46227d9e09bd68"
 from {username='slave'} at 127.0.0.1:44084
I20260709 13:17:11.964901 18357 raft_consensus.cc:606] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Received request to transfer leadership to TS 88766a58641e46449f46227d9e09bd68
I20260709 13:17:11.969640 18357 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ab85571c50949298324235b64c91bac"
dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
mode: GRACEFUL
new_leader_uuid: "88766a58641e46449f46227d9e09bd68"
 from {username='slave'} at 127.0.0.1:44084
I20260709 13:17:11.970146 18357 raft_consensus.cc:606] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Received request to transfer leadership to TS 88766a58641e46449f46227d9e09bd68
I20260709 13:17:11.973457 18505 consensus_queue.cc:1048] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:11.988322 18480 consensus_queue.cc:1048] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:11.990393 18432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad42bcd434774d8c8cbc5dd5188ba040"
dest_uuid: "0a3115e6843944079e3fad8d05197029"
mode: GRACEFUL
new_leader_uuid: "88766a58641e46449f46227d9e09bd68"
 from {username='slave'} at 127.0.0.1:48146
I20260709 13:17:11.990963 18432 raft_consensus.cc:606] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Received request to transfer leadership to TS 88766a58641e46449f46227d9e09bd68
I20260709 13:17:11.999912 18432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c338bb3d5e064f07bc473134a4090cb4"
dest_uuid: "0a3115e6843944079e3fad8d05197029"
mode: GRACEFUL
new_leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
 from {username='slave'} at 127.0.0.1:48146
I20260709 13:17:12.000473 18432 raft_consensus.cc:606] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Received request to transfer leadership to TS 5f3f6d22d9624fc2a34acafd98a6892c
I20260709 13:17:12.001331 18491 consensus_queue.cc:1048] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:12.010212 18491 raft_consensus.cc:993] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029: : Instructing follower 5f3f6d22d9624fc2a34acafd98a6892c to start an election
I20260709 13:17:12.010684 18494 raft_consensus.cc:1081] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Signalling peer 5f3f6d22d9624fc2a34acafd98a6892c to start an election
I20260709 13:17:12.013907 18357 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c338bb3d5e064f07bc473134a4090cb4"
dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
 from {username='slave'} at 127.0.0.1:44066
I20260709 13:17:12.014505 18357 raft_consensus.cc:493] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:12.014861 18357 raft_consensus.cc:3060] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.020053 18494 consensus_queue.cc:1048] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:12.023064 18357 raft_consensus.cc:515] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.025969 18283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c338bb3d5e064f07bc473134a4090cb4" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:12.026082 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c338bb3d5e064f07bc473134a4090cb4" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:12.026724 18432 raft_consensus.cc:3055] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:12.027055 18432 raft_consensus.cc:740] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:12.027489 18281 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5ac76c3dcf5452db28d5619ff1c913b"
dest_uuid: "88766a58641e46449f46227d9e09bd68"
mode: GRACEFUL
new_leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
 from {username='slave'} at 127.0.0.1:37594
I20260709 13:17:12.028085 18281 raft_consensus.cc:606] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Received request to transfer leadership to TS 5f3f6d22d9624fc2a34acafd98a6892c
I20260709 13:17:12.027866 18432 consensus_queue.cc:260] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.028743 18432 raft_consensus.cc:3060] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.035717 18432 raft_consensus.cc:2468] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 2.
I20260709 13:17:12.036762 18325 leader_election.cc:304] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 88766a58641e46449f46227d9e09bd68
I20260709 13:17:12.037550 18357 leader_election.cc:290] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 2 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:12.037591 18480 raft_consensus.cc:2804] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:12.038228 18480 raft_consensus.cc:697] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Becoming Leader. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:12.039070 18480 consensus_queue.cc:237] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.045620 18180 catalog_manager.cc:5697] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c reported cstate change: term changed from 1 to 2, leader changed from 0a3115e6843944079e3fad8d05197029 (127.6.198.67) to 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66). New cstate: current_term: 2 leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:12.067806 18491 consensus_queue.cc:1048] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:12.078739 18494 consensus_queue.cc:1048] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:12.117933 18480 consensus_queue.cc:1048] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:12.120141 18480 consensus_queue.cc:1048] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:12.204069 18506 raft_consensus.cc:993] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c: : Instructing follower 88766a58641e46449f46227d9e09bd68 to start an election
I20260709 13:17:12.204578 18505 raft_consensus.cc:1081] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Signalling peer 88766a58641e46449f46227d9e09bd68 to start an election
I20260709 13:17:12.205986 18284 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d7e9ac119d140b091ea674bbe3f7a71"
dest_uuid: "88766a58641e46449f46227d9e09bd68"
 from {username='slave'} at 127.0.0.1:37584
I20260709 13:17:12.206480 18284 raft_consensus.cc:493] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:12.206733 18284 raft_consensus.cc:3060] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.211761 18284 raft_consensus.cc:515] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.213203 18284 leader_election.cc:290] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 2 election: Requested vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:12.214017 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e9ac119d140b091ea674bbe3f7a71" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:12.214248 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d7e9ac119d140b091ea674bbe3f7a71" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:12.214607 18357 raft_consensus.cc:3055] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:12.214890 18432 raft_consensus.cc:3060] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.214944 18357 raft_consensus.cc:740] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:12.215847 18357 consensus_queue.cc:260] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.216861 18357 raft_consensus.cc:3060] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.219732 18432 raft_consensus.cc:2468] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 2.
I20260709 13:17:12.220830 18250 leader_election.cc:304] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:12.221597 18503 raft_consensus.cc:2804] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:12.221989 18503 raft_consensus.cc:697] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Becoming Leader. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:12.222308 18357 raft_consensus.cc:2468] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 2.
I20260709 13:17:12.222772 18503 consensus_queue.cc:237] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.228832 18178 catalog_manager.cc:5697] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 reported cstate change: term changed from 1 to 2, leader changed from 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66) to 88766a58641e46449f46227d9e09bd68 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "88766a58641e46449f46227d9e09bd68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:12.322039 18491 raft_consensus.cc:993] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029: : Instructing follower 88766a58641e46449f46227d9e09bd68 to start an election
I20260709 13:17:12.322609 18494 raft_consensus.cc:1081] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Signalling peer 88766a58641e46449f46227d9e09bd68 to start an election
I20260709 13:17:12.324410 18284 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad42bcd434774d8c8cbc5dd5188ba040"
dest_uuid: "88766a58641e46449f46227d9e09bd68"
 from {username='slave'} at 127.0.0.1:37574
I20260709 13:17:12.324890 18284 raft_consensus.cc:493] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:12.325531 18284 raft_consensus.cc:3060] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.332294 18284 raft_consensus.cc:515] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.334019 18284 leader_election.cc:290] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 2 election: Requested vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:12.334800 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad42bcd434774d8c8cbc5dd5188ba040" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:12.335073 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad42bcd434774d8c8cbc5dd5188ba040" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:12.335487 18357 raft_consensus.cc:3060] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.335633 18432 raft_consensus.cc:3055] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:12.336076 18432 raft_consensus.cc:740] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:12.336962 18432 consensus_queue.cc:260] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.338222 18432 raft_consensus.cc:3060] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.342851 18357 raft_consensus.cc:2468] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 2.
I20260709 13:17:12.344089 18250 leader_election.cc:304] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:12.344841 18503 raft_consensus.cc:2804] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:12.345129 18432 raft_consensus.cc:2468] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 2.
I20260709 13:17:12.345345 18503 raft_consensus.cc:697] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Becoming Leader. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:12.346499 18503 consensus_queue.cc:237] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.353044 18506 raft_consensus.cc:993] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c: : Instructing follower 88766a58641e46449f46227d9e09bd68 to start an election
I20260709 13:17:12.352806 18178 catalog_manager.cc:5697] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 reported cstate change: term changed from 1 to 2, leader changed from 0a3115e6843944079e3fad8d05197029 (127.6.198.67) to 88766a58641e46449f46227d9e09bd68 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "88766a58641e46449f46227d9e09bd68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:12.353550 18505 raft_consensus.cc:1081] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Signalling peer 88766a58641e46449f46227d9e09bd68 to start an election
I20260709 13:17:12.355109 18284 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ab85571c50949298324235b64c91bac"
dest_uuid: "88766a58641e46449f46227d9e09bd68"
 from {username='slave'} at 127.0.0.1:37584
I20260709 13:17:12.355583 18284 raft_consensus.cc:493] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:12.355926 18284 raft_consensus.cc:3060] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.363034 18521 raft_consensus.cc:993] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68: : Instructing follower 5f3f6d22d9624fc2a34acafd98a6892c to start an election
I20260709 13:17:12.363574 18521 raft_consensus.cc:1081] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Signalling peer 5f3f6d22d9624fc2a34acafd98a6892c to start an election
I20260709 13:17:12.363478 18284 raft_consensus.cc:515] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.365882 18357 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5ac76c3dcf5452db28d5619ff1c913b"
dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
 from {username='slave'} at 127.0.0.1:44072
I20260709 13:17:12.366148 18284 leader_election.cc:290] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 2 election: Requested vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:12.366715 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab85571c50949298324235b64c91bac" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:12.366632 18357 raft_consensus.cc:493] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:12.367072 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ab85571c50949298324235b64c91bac" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:12.367375 18359 raft_consensus.cc:3055] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:12.367609 18357 raft_consensus.cc:3060] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.367703 18432 raft_consensus.cc:3060] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.367743 18359 raft_consensus.cc:740] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:12.368999 18359 consensus_queue.cc:260] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.370446 18359 raft_consensus.cc:3060] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.373010 18432 raft_consensus.cc:2468] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 2.
I20260709 13:17:12.373950 18250 leader_election.cc:304] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:12.374663 18503 raft_consensus.cc:2804] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:12.375090 18503 raft_consensus.cc:697] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Becoming Leader. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:12.375613 18357 raft_consensus.cc:515] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.376271 18503 consensus_queue.cc:237] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.378173 18359 raft_consensus.cc:2468] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 2.
I20260709 13:17:12.378628 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ac76c3dcf5452db28d5619ff1c913b" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:12.379170 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5ac76c3dcf5452db28d5619ff1c913b" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:12.379379 18284 raft_consensus.cc:3055] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:12.379822 18432 raft_consensus.cc:3060] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.379874 18284 raft_consensus.cc:740] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:12.380875 18284 consensus_queue.cc:260] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.381990 18284 raft_consensus.cc:3060] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.385303 18432 raft_consensus.cc:2468] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 2.
I20260709 13:17:12.385159 18178 catalog_manager.cc:5697] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 reported cstate change: term changed from 1 to 2, leader changed from 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66) to 88766a58641e46449f46227d9e09bd68 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "88766a58641e46449f46227d9e09bd68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:12.386526 18325 leader_election.cc:304] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 
I20260709 13:17:12.385794 18357 leader_election.cc:290] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 2 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:12.387351 18505 raft_consensus.cc:2804] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:12.387874 18505 raft_consensus.cc:697] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Becoming Leader. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:12.388736 18505 consensus_queue.cc:237] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:12.388854 18284 raft_consensus.cc:2468] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 2.
I20260709 13:17:12.395938 18178 catalog_manager.cc:5697] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c reported cstate change: term changed from 1 to 2, leader changed from 88766a58641e46449f46227d9e09bd68 (127.6.198.65) to 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66). New cstate: current_term: 2 leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:12.557814 18432 raft_consensus.cc:1275] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Refusing update from remote peer 5f3f6d22d9624fc2a34acafd98a6892c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:12.558971 18505 consensus_queue.cc:1048] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:12.565814 18284 raft_consensus.cc:3060] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:12.574028 18284 raft_consensus.cc:1275] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Refusing update from remote peer 5f3f6d22d9624fc2a34acafd98a6892c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:12.575218 18505 consensus_queue.cc:1048] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:12.619290 18432 raft_consensus.cc:1275] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Refusing update from remote peer 88766a58641e46449f46227d9e09bd68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:12.620518 18503 consensus_queue.cc:1048] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:12.626935 18357 raft_consensus.cc:1275] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Refusing update from remote peer 88766a58641e46449f46227d9e09bd68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:12.628438 18503 consensus_queue.cc:1048] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:12.753240 18432 raft_consensus.cc:1275] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Refusing update from remote peer 88766a58641e46449f46227d9e09bd68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:12.754596 18503 consensus_queue.cc:1048] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:12.762894 18357 raft_consensus.cc:1275] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Refusing update from remote peer 88766a58641e46449f46227d9e09bd68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:12.764288 18521 consensus_queue.cc:1048] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:12.782263 18432 raft_consensus.cc:1275] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Refusing update from remote peer 5f3f6d22d9624fc2a34acafd98a6892c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:12.783387 18505 consensus_queue.cc:1048] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:12.790576 18284 raft_consensus.cc:1275] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Refusing update from remote peer 5f3f6d22d9624fc2a34acafd98a6892c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:12.792260 18505 consensus_queue.cc:1048] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:12.851473 18357 raft_consensus.cc:1275] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Refusing update from remote peer 88766a58641e46449f46227d9e09bd68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:12.852660 18521 consensus_queue.cc:1048] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:12.860730 18432 raft_consensus.cc:1275] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Refusing update from remote peer 88766a58641e46449f46227d9e09bd68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:12.862035 18503 consensus_queue.cc:1048] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:15.070281 18180 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46078:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:15.072701 18180 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:15.122565 18273 tablet_service.cc:1511] Processing CreateTablet for tablet 65f8d06d5edc4472ae7573fef6c1ad5a (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:15.123612 18341 tablet_service.cc:1511] Processing CreateTablet for tablet 65f8d06d5edc4472ae7573fef6c1ad5a (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:15.124269 18273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65f8d06d5edc4472ae7573fef6c1ad5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.125146 18341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65f8d06d5edc4472ae7573fef6c1ad5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.125900 18416 tablet_service.cc:1511] Processing CreateTablet for tablet 65f8d06d5edc4472ae7573fef6c1ad5a (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:15.126852 18267 tablet_service.cc:1511] Processing CreateTablet for tablet 87ecfb9a373b4fc49adadc26edb303a6 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:15.127291 18416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65f8d06d5edc4472ae7573fef6c1ad5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.128276 18267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ecfb9a373b4fc49adadc26edb303a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.131369 18268 tablet_service.cc:1511] Processing CreateTablet for tablet cb850b6ebc584c56b8b97b4b6f5b0819 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:15.131855 18344 tablet_service.cc:1511] Processing CreateTablet for tablet 87ecfb9a373b4fc49adadc26edb303a6 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:15.132771 18268 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb850b6ebc584c56b8b97b4b6f5b0819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.133205 18344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ecfb9a373b4fc49adadc26edb303a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.135969 18417 tablet_service.cc:1511] Processing CreateTablet for tablet 87ecfb9a373b4fc49adadc26edb303a6 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:15.137399 18417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87ecfb9a373b4fc49adadc26edb303a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.143702 18342 tablet_service.cc:1511] Processing CreateTablet for tablet cb850b6ebc584c56b8b97b4b6f5b0819 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:15.145167 18342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb850b6ebc584c56b8b97b4b6f5b0819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.164453 18415 tablet_service.cc:1511] Processing CreateTablet for tablet cb850b6ebc584c56b8b97b4b6f5b0819 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:15.165959 18415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb850b6ebc584c56b8b97b4b6f5b0819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.166682 18269 tablet_service.cc:1511] Processing CreateTablet for tablet ade415aef06a450eb50a7907a22d5849 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:15.168182 18269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade415aef06a450eb50a7907a22d5849. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.169188 18345 tablet_service.cc:1511] Processing CreateTablet for tablet ade415aef06a450eb50a7907a22d5849 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:15.170578 18345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade415aef06a450eb50a7907a22d5849. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:15.175063 18230 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:15.179890 18418 tablet_service.cc:1511] Processing CreateTablet for tablet ade415aef06a450eb50a7907a22d5849 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:15.181335 18418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade415aef06a450eb50a7907a22d5849. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.182047 18266 tablet_service.cc:1511] Processing CreateTablet for tablet de81f3e8dc484f52a9e49a1a8b2f8945 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:15.183367 18266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de81f3e8dc484f52a9e49a1a8b2f8945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.186137 18419 tablet_service.cc:1511] Processing CreateTablet for tablet de81f3e8dc484f52a9e49a1a8b2f8945 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:15.187498 18419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de81f3e8dc484f52a9e49a1a8b2f8945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.189805 18272 tablet_service.cc:1511] Processing CreateTablet for tablet ed0588646d884ab19a290e8ac8fb025c (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:15.191151 18272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed0588646d884ab19a290e8ac8fb025c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.197562 18422 tablet_service.cc:1511] Processing CreateTablet for tablet ed0588646d884ab19a290e8ac8fb025c (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:15.198274 18347 tablet_service.cc:1511] Processing CreateTablet for tablet de81f3e8dc484f52a9e49a1a8b2f8945 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:15.199021 18422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed0588646d884ab19a290e8ac8fb025c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.199661 18347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de81f3e8dc484f52a9e49a1a8b2f8945. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.227406 18574 tablet_bootstrap.cc:492] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:15.235384 18574 tablet_bootstrap.cc:654] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.240417 18575 tablet_bootstrap.cc:492] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:15.244545 18273 tablet_service.cc:1511] Processing CreateTablet for tablet 8cdb694dd8674ed88ba584fcf49328c9 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:15.252285 18273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cdb694dd8674ed88ba584fcf49328c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.254933 18415 tablet_service.cc:1511] Processing CreateTablet for tablet 8cdb694dd8674ed88ba584fcf49328c9 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:15.258576 18415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cdb694dd8674ed88ba584fcf49328c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.263527 18575 tablet_bootstrap.cc:654] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.264163 18267 tablet_service.cc:1511] Processing CreateTablet for tablet 9706990d796b40b7ad65514a7ce20f56 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:15.249742 18342 tablet_service.cc:1511] Processing CreateTablet for tablet ed0588646d884ab19a290e8ac8fb025c (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:15.266680 18342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed0588646d884ab19a290e8ac8fb025c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.268611 18574 tablet_bootstrap.cc:492] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:15.269187 18574 ts_tablet_manager.cc:1403] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.042s	user 0.000s	sys 0.011s
I20260709 13:17:15.271371 18344 tablet_service.cc:1511] Processing CreateTablet for tablet 8cdb694dd8674ed88ba584fcf49328c9 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:15.272676 18344 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cdb694dd8674ed88ba584fcf49328c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.274438 18267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9706990d796b40b7ad65514a7ce20f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.277801 18347 tablet_service.cc:1511] Processing CreateTablet for tablet 9706990d796b40b7ad65514a7ce20f56 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:15.279152 18347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9706990d796b40b7ad65514a7ce20f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.281301 18576 tablet_bootstrap.cc:492] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:15.287808 18574 raft_consensus.cc:359] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.288640 18574 raft_consensus.cc:385] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.288936 18574 raft_consensus.cc:740] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.291013 18419 tablet_service.cc:1511] Processing CreateTablet for tablet 9706990d796b40b7ad65514a7ce20f56 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:15.292528 18419 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9706990d796b40b7ad65514a7ce20f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.295658 18574 consensus_queue.cc:260] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.298117 18341 tablet_service.cc:1511] Processing CreateTablet for tablet c170e078318c43fbbaca55225e42b6b7 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:15.299031 18575 tablet_bootstrap.cc:492] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:15.299592 18341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c170e078318c43fbbaca55225e42b6b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.303174 18266 tablet_service.cc:1511] Processing CreateTablet for tablet c170e078318c43fbbaca55225e42b6b7 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:15.304584 18266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c170e078318c43fbbaca55225e42b6b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.304682 18574 ts_tablet_manager.cc:1434] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.005s
I20260709 13:17:15.305922 18574 tablet_bootstrap.cc:492] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:15.299662 18575 ts_tablet_manager.cc:1403] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.059s	user 0.011s	sys 0.014s
I20260709 13:17:15.318397 18576 tablet_bootstrap.cc:654] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.320294 18415 tablet_service.cc:1511] Processing CreateTablet for tablet c170e078318c43fbbaca55225e42b6b7 (DEFAULT_TABLE table=soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:15.322829 18415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c170e078318c43fbbaca55225e42b6b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:15.321029 18575 raft_consensus.cc:359] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.330200 18575 raft_consensus.cc:385] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.330597 18575 raft_consensus.cc:740] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.331521 18575 consensus_queue.cc:260] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.338610 18574 tablet_bootstrap.cc:654] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.338627 18575 ts_tablet_manager.cc:1434] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.001s
I20260709 13:17:15.346400 18575 tablet_bootstrap.cc:492] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:15.363337 18575 tablet_bootstrap.cc:654] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.371534 18574 tablet_bootstrap.cc:492] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:15.372119 18574 ts_tablet_manager.cc:1403] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.066s	user 0.009s	sys 0.006s
I20260709 13:17:15.375298 18574 raft_consensus.cc:359] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.376725 18574 raft_consensus.cc:385] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.377024 18574 raft_consensus.cc:740] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.378106 18576 tablet_bootstrap.cc:492] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:15.378753 18576 ts_tablet_manager.cc:1403] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.098s	user 0.016s	sys 0.019s
I20260709 13:17:15.378207 18574 consensus_queue.cc:260] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.380539 18574 ts_tablet_manager.cc:1434] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:15.381868 18574 tablet_bootstrap.cc:492] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:15.382123 18576 raft_consensus.cc:359] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.383023 18576 raft_consensus.cc:385] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.383315 18576 raft_consensus.cc:740] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.384178 18576 consensus_queue.cc:260] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.387189 18576 ts_tablet_manager.cc:1434] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:15.388332 18576 tablet_bootstrap.cc:492] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:15.390965 18574 tablet_bootstrap.cc:654] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.396579 18576 tablet_bootstrap.cc:654] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.397347 18575 tablet_bootstrap.cc:492] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:15.398043 18575 ts_tablet_manager.cc:1403] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.052s	user 0.007s	sys 0.010s
I20260709 13:17:15.401989 18575 raft_consensus.cc:359] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.402779 18575 raft_consensus.cc:385] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.403074 18575 raft_consensus.cc:740] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.404068 18575 consensus_queue.cc:260] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.405010 18574 tablet_bootstrap.cc:492] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:15.405608 18574 ts_tablet_manager.cc:1403] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.000s
I20260709 13:17:15.408725 18574 raft_consensus.cc:359] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.409525 18574 raft_consensus.cc:385] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.409859 18574 raft_consensus.cc:740] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.410776 18574 consensus_queue.cc:260] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.413707 18575 ts_tablet_manager.cc:1434] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:17:15.416519 18574 ts_tablet_manager.cc:1434] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:17:15.417881 18574 tablet_bootstrap.cc:492] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:15.424707 18574 tablet_bootstrap.cc:654] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.421453 18575 tablet_bootstrap.cc:492] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:15.440364 18575 tablet_bootstrap.cc:654] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.442556 18576 tablet_bootstrap.cc:492] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:15.443022 18574 tablet_bootstrap.cc:492] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:15.443269 18576 ts_tablet_manager.cc:1403] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.055s	user 0.034s	sys 0.014s
I20260709 13:17:15.443629 18574 ts_tablet_manager.cc:1403] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.015s
I20260709 13:17:15.445771 18576 raft_consensus.cc:359] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.446451 18576 raft_consensus.cc:385] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.446760 18576 raft_consensus.cc:740] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.447575 18574 raft_consensus.cc:359] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.447589 18576 consensus_queue.cc:260] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.448426 18574 raft_consensus.cc:385] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.449045 18574 raft_consensus.cc:740] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.449205 18575 tablet_bootstrap.cc:492] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:15.449738 18575 ts_tablet_manager.cc:1403] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.005s
I20260709 13:17:15.450001 18574 consensus_queue.cc:260] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.450696 18576 ts_tablet_manager.cc:1434] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:17:15.452545 18574 ts_tablet_manager.cc:1434] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:15.453377 18575 raft_consensus.cc:359] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.453840 18574 tablet_bootstrap.cc:492] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:15.454118 18575 raft_consensus.cc:385] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.454512 18575 raft_consensus.cc:740] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.455552 18575 consensus_queue.cc:260] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.459102 18576 tablet_bootstrap.cc:492] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:15.461318 18575 ts_tablet_manager.cc:1434] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:17:15.462312 18574 tablet_bootstrap.cc:654] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.462638 18575 tablet_bootstrap.cc:492] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:15.475307 18575 tablet_bootstrap.cc:654] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.480870 18576 tablet_bootstrap.cc:654] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.489204 18574 tablet_bootstrap.cc:492] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:15.489738 18574 ts_tablet_manager.cc:1403] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.036s	user 0.018s	sys 0.015s
I20260709 13:17:15.490530 18576 tablet_bootstrap.cc:492] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:15.490671 18575 tablet_bootstrap.cc:492] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:15.491061 18576 ts_tablet_manager.cc:1403] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.000s
I20260709 13:17:15.491192 18575 ts_tablet_manager.cc:1403] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.004s
I20260709 13:17:15.492784 18574 raft_consensus.cc:359] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.493548 18574 raft_consensus.cc:385] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.493667 18575 raft_consensus.cc:359] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.493880 18574 raft_consensus.cc:740] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.494402 18575 raft_consensus.cc:385] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.494149 18576 raft_consensus.cc:359] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.494777 18575 raft_consensus.cc:740] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.494928 18576 raft_consensus.cc:385] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.494899 18574 consensus_queue.cc:260] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.495486 18576 raft_consensus.cc:740] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.495618 18575 consensus_queue.cc:260] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.497617 18574 ts_tablet_manager.cc:1434] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:17:15.497203 18576 consensus_queue.cc:260] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.499337 18574 tablet_bootstrap.cc:492] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:15.500234 18576 ts_tablet_manager.cc:1434] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:17:15.501691 18576 tablet_bootstrap.cc:492] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:15.502311 18575 ts_tablet_manager.cc:1434] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 13:17:15.503479 18575 tablet_bootstrap.cc:492] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:15.507339 18574 tablet_bootstrap.cc:654] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.508648 18576 tablet_bootstrap.cc:654] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.517987 18569 raft_consensus.cc:493] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.519073 18575 tablet_bootstrap.cc:654] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.518730 18569 raft_consensus.cc:515] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.523880 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade415aef06a450eb50a7907a22d5849" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
W20260709 13:17:15.526008 18399 leader_election.cc:343] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:15.526775 18574 tablet_bootstrap.cc:492] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:15.527073 18569 leader_election.cc:290] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:15.527521 18574 ts_tablet_manager.cc:1403] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.015s
I20260709 13:17:15.531566 18574 raft_consensus.cc:359] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.532325 18574 raft_consensus.cc:385] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.532477 18575 tablet_bootstrap.cc:492] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:15.532648 18574 raft_consensus.cc:740] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.533150 18575 ts_tablet_manager.cc:1403] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.008s
I20260709 13:17:15.533687 18574 consensus_queue.cc:260] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.535663 18574 ts_tablet_manager.cc:1434] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:17:15.536779 18574 tablet_bootstrap.cc:492] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:15.536571 18575 raft_consensus.cc:359] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.537670 18575 raft_consensus.cc:385] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.538192 18575 raft_consensus.cc:740] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.539173 18575 consensus_queue.cc:260] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.540098 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade415aef06a450eb50a7907a22d5849" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:15.541215 18284 raft_consensus.cc:2468] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:15.543254 18398 leader_election.cc:304] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 5f3f6d22d9624fc2a34acafd98a6892c
I20260709 13:17:15.549811 18575 ts_tablet_manager.cc:1434] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:17:15.551139 18575 tablet_bootstrap.cc:492] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:15.551213 18569 raft_consensus.cc:493] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.554714 18569 raft_consensus.cc:515] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.551599 18574 tablet_bootstrap.cc:654] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.559208 18594 raft_consensus.cc:2804] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:15.559691 18594 raft_consensus.cc:493] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.560026 18594 raft_consensus.cc:3060] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.563649 18580 raft_consensus.cc:493] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.563954 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ecfb9a373b4fc49adadc26edb303a6" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:15.564479 18574 tablet_bootstrap.cc:492] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:15.564314 18580 raft_consensus.cc:515] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.564651 18357 raft_consensus.cc:2468] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:15.565090 18574 ts_tablet_manager.cc:1403] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.004s
I20260709 13:17:15.566193 18399 leader_election.cc:304] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 
I20260709 13:17:15.567768 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb850b6ebc584c56b8b97b4b6f5b0819" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
I20260709 13:17:15.568485 18432 raft_consensus.cc:2468] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 0.
I20260709 13:17:15.568650 18574 raft_consensus.cc:359] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.569563 18574 raft_consensus.cc:385] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.569589 18569 leader_election.cc:290] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:15.569963 18574 raft_consensus.cc:740] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.570031 18325 leader_election.cc:304] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 
I20260709 13:17:15.570441 18569 raft_consensus.cc:2804] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:15.570876 18569 raft_consensus.cc:493] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.572654 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb850b6ebc584c56b8b97b4b6f5b0819" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:15.583864 18284 raft_consensus.cc:2468] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 0.
I20260709 13:17:15.583272 18569 raft_consensus.cc:3060] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.572738 18580 leader_election.cc:290] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:15.574723 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ecfb9a373b4fc49adadc26edb303a6" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:15.586552 18282 raft_consensus.cc:2468] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:15.586668 18576 tablet_bootstrap.cc:492] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:15.587280 18576 ts_tablet_manager.cc:1403] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.086s	user 0.029s	sys 0.037s
I20260709 13:17:15.571787 18559 raft_consensus.cc:2804] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:15.588403 18559 raft_consensus.cc:493] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.588765 18559 raft_consensus.cc:3060] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.590008 18576 raft_consensus.cc:359] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.590761 18576 raft_consensus.cc:385] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.590608 18595 raft_consensus.cc:493] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.581171 18594 raft_consensus.cc:515] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.591306 18576 raft_consensus.cc:740] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.591545 18595 raft_consensus.cc:515] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.592170 18576 consensus_queue.cc:260] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.593798 18595 leader_election.cc:290] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:15.594833 18576 ts_tablet_manager.cc:1434] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:17:15.594971 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade415aef06a450eb50a7907a22d5849" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:15.580073 18575 tablet_bootstrap.cc:654] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.571159 18574 consensus_queue.cc:260] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.595952 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f8d06d5edc4472ae7573fef6c1ad5a" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:15.596146 18576 tablet_bootstrap.cc:492] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:15.596611 18359 raft_consensus.cc:2468] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:15.597951 18574 ts_tablet_manager.cc:1434] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:17:15.598752 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade415aef06a450eb50a7907a22d5849" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:15.599174 18574 tablet_bootstrap.cc:492] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:15.599350 18282 raft_consensus.cc:3060] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:15.603121 18399 leader_election.cc:343] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:15.603850 18399 leader_election.cc:304] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 
I20260709 13:17:15.604807 18595 raft_consensus.cc:2804] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:15.605301 18595 raft_consensus.cc:493] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.605527 18594 leader_election.cc:290] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:15.605762 18595 raft_consensus.cc:3060] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.607314 18594 raft_consensus.cc:493] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.607739 18594 raft_consensus.cc:515] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.605715 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f8d06d5edc4472ae7573fef6c1ad5a" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:15.610267 18284 raft_consensus.cc:2468] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:15.616159 18576 tablet_bootstrap.cc:654] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.629788 18282 raft_consensus.cc:2468] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:15.610306 18594 leader_election.cc:290] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:15.649329 18559 raft_consensus.cc:515] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.651328 18398 leader_election.cc:304] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 5f3f6d22d9624fc2a34acafd98a6892c
I20260709 13:17:15.655530 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb850b6ebc584c56b8b97b4b6f5b0819" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:15.656188 18282 raft_consensus.cc:3060] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.660142 18594 raft_consensus.cc:2804] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:15.634555 18569 raft_consensus.cc:515] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.636555 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0588646d884ab19a290e8ac8fb025c" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:15.645790 18574 tablet_bootstrap.cc:654] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.660967 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb850b6ebc584c56b8b97b4b6f5b0819" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:15.661671 18284 raft_consensus.cc:2468] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:15.661842 18432 raft_consensus.cc:3060] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.665824 18576 tablet_bootstrap.cc:492] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:15.666567 18576 ts_tablet_manager.cc:1403] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.071s	user 0.021s	sys 0.012s
I20260709 13:17:15.661003 18559 leader_election.cc:290] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:15.689842 18575 tablet_bootstrap.cc:492] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
W20260709 13:17:15.692732 18399 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.6.198.66:35643, user_credentials={real_user=slave}} blocked reactor thread for 88960.6us
I20260709 13:17:15.693078 18576 raft_consensus.cc:359] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.694646 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0588646d884ab19a290e8ac8fb025c" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:15.697371 18576 raft_consensus.cc:385] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.694941 18398 leader_election.cc:304] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:15.697723 18576 raft_consensus.cc:740] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.698302 18601 raft_consensus.cc:2804] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:15.698920 18601 raft_consensus.cc:493] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.699579 18282 raft_consensus.cc:2468] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 1.
I20260709 13:17:15.695991 18569 leader_election.cc:290] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:15.697925 18432 raft_consensus.cc:2468] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 1.
I20260709 13:17:15.694681 18595 raft_consensus.cc:515] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.700562 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ecfb9a373b4fc49adadc26edb303a6" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:15.701589 18284 raft_consensus.cc:3060] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.701699 18325 leader_election.cc:304] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 
I20260709 13:17:15.702244 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ecfb9a373b4fc49adadc26edb303a6" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:15.702802 18595 leader_election.cc:290] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:15.702760 18559 raft_consensus.cc:2804] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:15.703040 18359 raft_consensus.cc:3060] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.660670 18594 raft_consensus.cc:697] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Becoming Leader. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:15.703766 18559 raft_consensus.cc:697] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Becoming Leader. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:15.704474 18594 consensus_queue.cc:237] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.704593 18559 consensus_queue.cc:237] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.696709 18575 ts_tablet_manager.cc:1403] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.146s	user 0.031s	sys 0.030s
I20260709 13:17:15.707597 18281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f8d06d5edc4472ae7573fef6c1ad5a" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:15.708292 18281 raft_consensus.cc:3060] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.699841 18601 raft_consensus.cc:3060] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.712369 18575 raft_consensus.cc:359] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.713332 18359 raft_consensus.cc:2468] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:15.713428 18575 raft_consensus.cc:385] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.717017 18575 raft_consensus.cc:740] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.720083 18575 consensus_queue.cc:260] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.722699 18281 raft_consensus.cc:2468] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:15.724184 18398 leader_election.cc:304] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:15.725181 18575 ts_tablet_manager.cc:1434] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.002s
I20260709 13:17:15.725379 18595 raft_consensus.cc:2804] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:15.725951 18595 raft_consensus.cc:697] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Becoming Leader. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:15.726104 18284 raft_consensus.cc:2468] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:15.726827 18595 consensus_queue.cc:237] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.727577 18398 leader_election.cc:304] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:15.726325 18575 tablet_bootstrap.cc:492] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:15.728752 18599 raft_consensus.cc:2804] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:15.729236 18599 raft_consensus.cc:697] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Becoming Leader. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:15.730098 18599 consensus_queue.cc:237] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.698822 18576 consensus_queue.cc:260] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.737232 18576 ts_tablet_manager.cc:1434] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.070s	user 0.004s	sys 0.009s
I20260709 13:17:15.739969 18576 tablet_bootstrap.cc:492] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
W20260709 13:17:15.743563 18399 leader_election.cc:343] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:15.745126 18575 tablet_bootstrap.cc:654] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.745955 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f8d06d5edc4472ae7573fef6c1ad5a" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:15.746608 18359 raft_consensus.cc:3060] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.755617 18576 tablet_bootstrap.cc:654] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.757869 18574 tablet_bootstrap.cc:492] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:15.758509 18574 ts_tablet_manager.cc:1403] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.160s	user 0.016s	sys 0.023s
I20260709 13:17:15.758376 18587 raft_consensus.cc:493] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.760634 18587 raft_consensus.cc:515] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.763427 18359 raft_consensus.cc:2468] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:15.765019 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de81f3e8dc484f52a9e49a1a8b2f8945" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:15.765843 18357 raft_consensus.cc:2468] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 0.
I20260709 13:17:15.767455 18250 leader_election.cc:304] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:15.768294 18521 raft_consensus.cc:2804] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:15.768675 18521 raft_consensus.cc:493] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.768543 18574 raft_consensus.cc:359] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.769320 18521 raft_consensus.cc:3060] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.769605 18574 raft_consensus.cc:385] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.770020 18574 raft_consensus.cc:740] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.772902 18574 consensus_queue.cc:260] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.776258 18574 ts_tablet_manager.cc:1434] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:17:15.778966 18574 tablet_bootstrap.cc:492] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029: Bootstrap starting.
I20260709 13:17:15.776292 18587 leader_election.cc:290] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:15.780102 18576 tablet_bootstrap.cc:492] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:15.780691 18576 ts_tablet_manager.cc:1403] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.043s	user 0.024s	sys 0.005s
I20260709 13:17:15.784554 18576 raft_consensus.cc:359] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.785265 18576 raft_consensus.cc:385] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.785472 18575 tablet_bootstrap.cc:492] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:15.785559 18576 raft_consensus.cc:740] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.786098 18575 ts_tablet_manager.cc:1403] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.000s
I20260709 13:17:15.786517 18576 consensus_queue.cc:260] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.785689 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de81f3e8dc484f52a9e49a1a8b2f8945" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
I20260709 13:17:15.788301 18432 raft_consensus.cc:2468] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 0.
I20260709 13:17:15.788950 18576 ts_tablet_manager.cc:1434] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:15.789319 18575 raft_consensus.cc:359] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.790557 18575 raft_consensus.cc:385] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.791117 18575 raft_consensus.cc:740] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.792241 18521 raft_consensus.cc:515] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.792851 18576 tablet_bootstrap.cc:492] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:15.793088 18574 tablet_bootstrap.cc:654] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.788852 18182 catalog_manager.cc:5697] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66). New cstate: current_term: 1 leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.793133 18575 consensus_queue.cc:260] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.796238 18521 leader_election.cc:290] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 election: Requested vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:15.800333 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de81f3e8dc484f52a9e49a1a8b2f8945" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:15.812069 18359 raft_consensus.cc:3060] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.812621 18576 tablet_bootstrap.cc:654] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.812918 18594 raft_consensus.cc:493] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.813575 18594 raft_consensus.cc:515] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.800755 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de81f3e8dc484f52a9e49a1a8b2f8945" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:15.814975 18432 raft_consensus.cc:3060] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.816556 18594 leader_election.cc:290] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:15.816764 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdb694dd8674ed88ba584fcf49328c9" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:15.814143 18180 catalog_manager.cc:5697] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a3115e6843944079e3fad8d05197029 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "0a3115e6843944079e3fad8d05197029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:15.822974 18399 leader_election.cc:343] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:15.823884 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdb694dd8674ed88ba584fcf49328c9" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
W20260709 13:17:15.825331 18398 leader_election.cc:343] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:15.825754 18398 leader_election.cc:304] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a3115e6843944079e3fad8d05197029; no voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68
I20260709 13:17:15.827443 18594 raft_consensus.cc:2749] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:15.801664 18601 raft_consensus.cc:515] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.832175 18601 leader_election.cc:290] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:15.833714 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0588646d884ab19a290e8ac8fb025c" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:15.834357 18357 raft_consensus.cc:3060] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.808418 18575 ts_tablet_manager.cc:1434] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:17:15.838155 18575 tablet_bootstrap.cc:492] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:15.838992 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0588646d884ab19a290e8ac8fb025c" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:15.839972 18284 raft_consensus.cc:3060] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.844549 18432 raft_consensus.cc:2468] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 1.
I20260709 13:17:15.847647 18359 raft_consensus.cc:2468] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 1.
I20260709 13:17:15.847390 18250 leader_election.cc:304] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:15.850180 18521 raft_consensus.cc:2804] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:15.850735 18284 raft_consensus.cc:2468] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:15.850898 18521 raft_consensus.cc:697] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Becoming Leader. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:15.852257 18357 raft_consensus.cc:2468] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:15.853677 18399 leader_election.cc:304] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 
I20260709 13:17:15.855362 18574 tablet_bootstrap.cc:492] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029: No bootstrap required, opened a new log
I20260709 13:17:15.855832 18574 ts_tablet_manager.cc:1403] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029: Time spent bootstrapping tablet: real 0.077s	user 0.011s	sys 0.000s
I20260709 13:17:15.856340 18601 raft_consensus.cc:2804] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:15.857234 18601 raft_consensus.cc:697] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Becoming Leader. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:15.858523 18601 consensus_queue.cc:237] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.859331 18574 raft_consensus.cc:359] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.860419 18574 raft_consensus.cc:385] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.852262 18521 consensus_queue.cc:237] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.860730 18574 raft_consensus.cc:740] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.867357 18574 consensus_queue.cc:260] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.871258 18575 tablet_bootstrap.cc:654] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.872432 18574 ts_tablet_manager.cc:1434] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:17:15.874429 18559 raft_consensus.cc:493] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.875355 18559 raft_consensus.cc:515] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.877280 18559 leader_election.cc:290] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:15.879642 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade415aef06a450eb50a7907a22d5849" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
I20260709 13:17:15.880431 18576 tablet_bootstrap.cc:492] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:15.880964 18576 ts_tablet_manager.cc:1403] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.088s	user 0.022s	sys 0.038s
I20260709 13:17:15.878504 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade415aef06a450eb50a7907a22d5849" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:15.882405 18284 raft_consensus.cc:2393] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5f3f6d22d9624fc2a34acafd98a6892c in current term 1: Already voted for candidate 0a3115e6843944079e3fad8d05197029 in this term.
I20260709 13:17:15.883508 18324 leader_election.cc:304] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68
I20260709 13:17:15.884341 18178 catalog_manager.cc:5697] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88766a58641e46449f46227d9e09bd68 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "88766a58641e46449f46227d9e09bd68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:15.884909 18576 raft_consensus.cc:359] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.885706 18576 raft_consensus.cc:385] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.885998 18576 raft_consensus.cc:740] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.884857 18559 raft_consensus.cc:3060] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.890374 18576 consensus_queue.cc:260] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.892748 18576 ts_tablet_manager.cc:1434] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.004s
I20260709 13:17:15.893736 18576 tablet_bootstrap.cc:492] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:15.895124 18559 raft_consensus.cc:2749] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:15.900950 18576 tablet_bootstrap.cc:654] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.912843 18575 tablet_bootstrap.cc:492] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:15.913313 18575 ts_tablet_manager.cc:1403] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.075s	user 0.021s	sys 0.003s
I20260709 13:17:15.914433 18180 catalog_manager.cc:5697] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a3115e6843944079e3fad8d05197029 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "0a3115e6843944079e3fad8d05197029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.916015 18575 raft_consensus.cc:359] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.916673 18575 raft_consensus.cc:385] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.916949 18575 raft_consensus.cc:740] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.918020 18576 tablet_bootstrap.cc:492] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:15.917615 18575 consensus_queue.cc:260] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.919121 18576 ts_tablet_manager.cc:1403] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.005s
I20260709 13:17:15.920516 18575 ts_tablet_manager.cc:1434] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:17:15.917563 18180 catalog_manager.cc:5697] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a3115e6843944079e3fad8d05197029 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "0a3115e6843944079e3fad8d05197029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.918521 18587 raft_consensus.cc:493] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:15.921864 18575 tablet_bootstrap.cc:492] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68: Bootstrap starting.
I20260709 13:17:15.921772 18587 raft_consensus.cc:515] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.922253 18180 catalog_manager.cc:5697] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a3115e6843944079e3fad8d05197029 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "0a3115e6843944079e3fad8d05197029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:15.923830 18587 leader_election.cc:290] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:15.925470 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9706990d796b40b7ad65514a7ce20f56" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
I20260709 13:17:15.926061 18432 raft_consensus.cc:2468] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 0.
I20260709 13:17:15.927187 18250 leader_election.cc:304] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:15.926973 18576 raft_consensus.cc:359] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.927863 18576 raft_consensus.cc:385] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.928123 18576 raft_consensus.cc:740] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.928746 18576 consensus_queue.cc:260] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.929992 18587 raft_consensus.cc:2804] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:15.930395 18587 raft_consensus.cc:493] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:15.930727 18576 ts_tablet_manager.cc:1434] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:17:15.930827 18575 tablet_bootstrap.cc:654] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.926949 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9706990d796b40b7ad65514a7ce20f56" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:15.931731 18576 tablet_bootstrap.cc:492] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c: Bootstrap starting.
I20260709 13:17:15.931661 18357 raft_consensus.cc:2468] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 0.
I20260709 13:17:15.930742 18587 raft_consensus.cc:3060] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.937743 18576 tablet_bootstrap.cc:654] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:15.942119 18575 tablet_bootstrap.cc:492] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68: No bootstrap required, opened a new log
I20260709 13:17:15.942587 18575 ts_tablet_manager.cc:1403] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 13:17:15.945138 18575 raft_consensus.cc:359] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.945771 18575 raft_consensus.cc:385] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.946038 18575 raft_consensus.cc:740] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.946667 18575 consensus_queue.cc:260] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.948664 18575 ts_tablet_manager.cc:1434] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:17:15.951586 18576 tablet_bootstrap.cc:492] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c: No bootstrap required, opened a new log
I20260709 13:17:15.952047 18576 ts_tablet_manager.cc:1403] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 13:17:15.952466 18587 raft_consensus.cc:515] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.953930 18576 raft_consensus.cc:359] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.954466 18576 raft_consensus.cc:385] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:15.954741 18576 raft_consensus.cc:740] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Initialized, Role: FOLLOWER
I20260709 13:17:15.955312 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9706990d796b40b7ad65514a7ce20f56" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:15.955430 18576 consensus_queue.cc:260] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.955945 18357 raft_consensus.cc:3060] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.957496 18576 ts_tablet_manager.cc:1434] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:17:15.963207 18587 leader_election.cc:290] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 election: Requested vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:15.963400 18357 raft_consensus.cc:2468] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 1.
I20260709 13:17:15.964313 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9706990d796b40b7ad65514a7ce20f56" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:15.964560 18250 leader_election.cc:304] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:15.964888 18432 raft_consensus.cc:3060] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:15.965309 18587 raft_consensus.cc:2804] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:15.965672 18587 raft_consensus.cc:697] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Becoming Leader. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:15.966491 18587 consensus_queue.cc:237] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:15.969750 18432 raft_consensus.cc:2468] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 1.
I20260709 13:17:15.975044 18180 catalog_manager.cc:5697] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88766a58641e46449f46227d9e09bd68 (127.6.198.65). New cstate: current_term: 1 leader_uuid: "88766a58641e46449f46227d9e09bd68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.011369 18601 raft_consensus.cc:493] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.011852 18601 raft_consensus.cc:515] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.013873 18601 leader_election.cc:290] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:16.014642 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c170e078318c43fbbaca55225e42b6b7" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:16.014912 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c170e078318c43fbbaca55225e42b6b7" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" is_pre_election: true
I20260709 13:17:16.015277 18284 raft_consensus.cc:2468] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:16.015542 18357 raft_consensus.cc:2468] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 0.
I20260709 13:17:16.016283 18398 leader_election.cc:304] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:16.017102 18601 raft_consensus.cc:2804] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:16.017438 18601 raft_consensus.cc:493] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:16.017722 18601 raft_consensus.cc:3060] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:16.022387 18601 raft_consensus.cc:515] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.023830 18601 leader_election.cc:290] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:16.024560 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c170e078318c43fbbaca55225e42b6b7" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:16.025067 18284 raft_consensus.cc:3060] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:16.024835 18357 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c170e078318c43fbbaca55225e42b6b7" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:16.025528 18357 raft_consensus.cc:3060] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:16.031252 18284 raft_consensus.cc:2468] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:16.031575 18357 raft_consensus.cc:2468] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 1.
I20260709 13:17:16.032606 18399 leader_election.cc:304] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 
I20260709 13:17:16.033483 18594 raft_consensus.cc:2804] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:16.033963 18594 raft_consensus.cc:697] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Becoming Leader. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:16.034765 18594 consensus_queue.cc:237] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.041857 18180 catalog_manager.cc:5697] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a3115e6843944079e3fad8d05197029 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "0a3115e6843944079e3fad8d05197029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:16.046384 18559 raft_consensus.cc:493] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:16.046819 18559 raft_consensus.cc:515] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.048662 18559 leader_election.cc:290] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:16.049152 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdb694dd8674ed88ba584fcf49328c9" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68" is_pre_election: true
I20260709 13:17:16.049536 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdb694dd8674ed88ba584fcf49328c9" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029" is_pre_election: true
I20260709 13:17:16.049767 18284 raft_consensus.cc:2468] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 0.
I20260709 13:17:16.050057 18432 raft_consensus.cc:2468] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 0.
I20260709 13:17:16.050732 18324 leader_election.cc:304] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:16.051373 18559 raft_consensus.cc:2804] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:16.051712 18559 raft_consensus.cc:493] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:16.051928 18559 raft_consensus.cc:3060] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:16.056365 18559 raft_consensus.cc:515] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.057758 18559 leader_election.cc:290] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:16.058422 18284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdb694dd8674ed88ba584fcf49328c9" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:16.058606 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cdb694dd8674ed88ba584fcf49328c9" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:16.058869 18284 raft_consensus.cc:3060] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:16.059151 18432 raft_consensus.cc:3060] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:16.063009 18284 raft_consensus.cc:2468] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 1.
I20260709 13:17:16.063616 18432 raft_consensus.cc:2468] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 1.
I20260709 13:17:16.063971 18324 leader_election.cc:304] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:16.064759 18559 raft_consensus.cc:2804] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:16.065120 18559 raft_consensus.cc:697] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Becoming Leader. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:16.065778 18559 consensus_queue.cc:237] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.072748 18180 catalog_manager.cc:5697] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66). New cstate: current_term: 1 leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.159727 18594 consensus_queue.cc:1048] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.176347 18594 consensus_queue.cc:1048] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.188935 18559 consensus_queue.cc:1048] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.196302 18595 consensus_queue.cc:1048] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.212255 18601 consensus_queue.cc:1048] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.223664 18632 consensus_queue.cc:1048] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:16.224974 18594 consensus_queue.cc:1048] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:16.236236 18594 consensus_queue.cc:1048] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:16.247419 18569 consensus_queue.cc:1048] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.274550 18432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "65f8d06d5edc4472ae7573fef6c1ad5a"
dest_uuid: "0a3115e6843944079e3fad8d05197029"
mode: GRACEFUL
new_leader_uuid: "88766a58641e46449f46227d9e09bd68"
 from {username='slave'} at 127.0.0.1:48146
I20260709 13:17:16.275133 18432 raft_consensus.cc:606] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Received request to transfer leadership to TS 88766a58641e46449f46227d9e09bd68
I20260709 13:17:16.281014 18595 consensus_queue.cc:1048] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.283890 18432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87ecfb9a373b4fc49adadc26edb303a6"
dest_uuid: "0a3115e6843944079e3fad8d05197029"
mode: GRACEFUL
new_leader_uuid: "88766a58641e46449f46227d9e09bd68"
 from {username='slave'} at 127.0.0.1:48146
I20260709 13:17:16.284451 18432 raft_consensus.cc:606] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Received request to transfer leadership to TS 88766a58641e46449f46227d9e09bd68
I20260709 13:17:16.291850 18357 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb850b6ebc584c56b8b97b4b6f5b0819"
dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
mode: GRACEFUL
new_leader_uuid: "88766a58641e46449f46227d9e09bd68"
 from {username='slave'} at 127.0.0.1:44084
I20260709 13:17:16.292373 18357 raft_consensus.cc:606] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Received request to transfer leadership to TS 88766a58641e46449f46227d9e09bd68
I20260709 13:17:16.299090 18432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ade415aef06a450eb50a7907a22d5849"
dest_uuid: "0a3115e6843944079e3fad8d05197029"
mode: GRACEFUL
new_leader_uuid: "88766a58641e46449f46227d9e09bd68"
 from {username='slave'} at 127.0.0.1:48146
I20260709 13:17:16.299574 18432 raft_consensus.cc:606] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Received request to transfer leadership to TS 88766a58641e46449f46227d9e09bd68
I20260709 13:17:16.304589 18282 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "de81f3e8dc484f52a9e49a1a8b2f8945"
dest_uuid: "88766a58641e46449f46227d9e09bd68"
mode: GRACEFUL
new_leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
 from {username='slave'} at 127.0.0.1:37594
I20260709 13:17:16.305178 18282 raft_consensus.cc:606] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Received request to transfer leadership to TS 5f3f6d22d9624fc2a34acafd98a6892c
I20260709 13:17:16.310113 18432 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed0588646d884ab19a290e8ac8fb025c"
dest_uuid: "0a3115e6843944079e3fad8d05197029"
mode: GRACEFUL
new_leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
 from {username='slave'} at 127.0.0.1:48146
I20260709 13:17:16.310570 18432 raft_consensus.cc:606] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Received request to transfer leadership to TS 5f3f6d22d9624fc2a34acafd98a6892c
I20260709 13:17:16.317875 18282 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9706990d796b40b7ad65514a7ce20f56"
dest_uuid: "88766a58641e46449f46227d9e09bd68"
mode: GRACEFUL
new_leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
 from {username='slave'} at 127.0.0.1:37594
I20260709 13:17:16.318317 18282 raft_consensus.cc:606] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Received request to transfer leadership to TS 5f3f6d22d9624fc2a34acafd98a6892c
I20260709 13:17:16.348668 18587 consensus_queue.cc:1048] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.351574 18587 consensus_queue.cc:1048] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.355796 18521 raft_consensus.cc:993] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68: : Instructing follower 5f3f6d22d9624fc2a34acafd98a6892c to start an election
I20260709 13:17:16.357214 18587 raft_consensus.cc:1081] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Signalling peer 5f3f6d22d9624fc2a34acafd98a6892c to start an election
I20260709 13:17:16.365247 18357 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "de81f3e8dc484f52a9e49a1a8b2f8945"
dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
 from {username='slave'} at 127.0.0.1:44072
I20260709 13:17:16.366818 18357 raft_consensus.cc:493] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:16.367237 18357 raft_consensus.cc:3060] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.370555 18641 consensus_queue.cc:1048] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.373095 18641 consensus_queue.cc:1048] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.384395 18357 raft_consensus.cc:515] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.399682 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de81f3e8dc484f52a9e49a1a8b2f8945" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:16.400368 18282 raft_consensus.cc:3055] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:16.400713 18282 raft_consensus.cc:740] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:16.401751 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de81f3e8dc484f52a9e49a1a8b2f8945" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:16.401543 18282 consensus_queue.cc:260] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.402315 18432 raft_consensus.cc:3060] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.400727 18357 leader_election.cc:290] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 2 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:16.404335 18282 raft_consensus.cc:3060] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.409058 18641 raft_consensus.cc:993] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68: : Instructing follower 5f3f6d22d9624fc2a34acafd98a6892c to start an election
I20260709 13:17:16.409570 18641 raft_consensus.cc:1081] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Signalling peer 5f3f6d22d9624fc2a34acafd98a6892c to start an election
I20260709 13:17:16.411652 18432 raft_consensus.cc:2468] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 2.
I20260709 13:17:16.411686 18282 raft_consensus.cc:2468] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 2.
I20260709 13:17:16.411396 18358 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9706990d796b40b7ad65514a7ce20f56"
dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
 from {username='slave'} at 127.0.0.1:44072
I20260709 13:17:16.412696 18358 raft_consensus.cc:493] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:16.413079 18358 raft_consensus.cc:3060] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.413107 18324 leader_election.cc:304] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:16.414330 18632 raft_consensus.cc:2804] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:16.414839 18632 raft_consensus.cc:697] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Becoming Leader. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:16.415625 18632 consensus_queue.cc:237] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.419960 18358 raft_consensus.cc:515] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.422019 18358 leader_election.cc:290] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 2 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:16.422297 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9706990d796b40b7ad65514a7ce20f56" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:16.422945 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9706990d796b40b7ad65514a7ce20f56" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:16.423465 18432 raft_consensus.cc:3060] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.422997 18282 raft_consensus.cc:3055] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:16.425624 18282 raft_consensus.cc:740] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:16.426540 18282 consensus_queue.cc:260] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.428203 18282 raft_consensus.cc:3060] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.432289 18178 catalog_manager.cc:5697] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c reported cstate change: term changed from 1 to 2, leader changed from 88766a58641e46449f46227d9e09bd68 (127.6.198.65) to 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66). New cstate: current_term: 2 leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.434836 18432 raft_consensus.cc:2468] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 2.
I20260709 13:17:16.435190 18282 raft_consensus.cc:2468] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 2.
I20260709 13:17:16.435967 18325 leader_election.cc:304] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 
I20260709 13:17:16.436810 18632 raft_consensus.cc:2804] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:16.437211 18632 raft_consensus.cc:697] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Becoming Leader. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:16.437959 18632 consensus_queue.cc:237] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.446480 18178 catalog_manager.cc:5697] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c reported cstate change: term changed from 1 to 2, leader changed from 88766a58641e46449f46227d9e09bd68 (127.6.198.65) to 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66). New cstate: current_term: 2 leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.469331 18632 consensus_queue.cc:1048] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.479547 18632 consensus_queue.cc:1048] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.543430 18595 consensus_queue.cc:1048] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.553555 18595 consensus_queue.cc:1048] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:16.623459 18594 raft_consensus.cc:993] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029: : Instructing follower 88766a58641e46449f46227d9e09bd68 to start an election
I20260709 13:17:16.623961 18595 raft_consensus.cc:1081] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Signalling peer 88766a58641e46449f46227d9e09bd68 to start an election
I20260709 13:17:16.625281 18282 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87ecfb9a373b4fc49adadc26edb303a6"
dest_uuid: "88766a58641e46449f46227d9e09bd68"
 from {username='slave'} at 127.0.0.1:37574
I20260709 13:17:16.625684 18282 raft_consensus.cc:493] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:16.625916 18282 raft_consensus.cc:3060] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.630825 18282 raft_consensus.cc:515] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.632223 18282 leader_election.cc:290] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 2 election: Requested vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:16.633031 18358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ecfb9a373b4fc49adadc26edb303a6" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:16.633348 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87ecfb9a373b4fc49adadc26edb303a6" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:16.633697 18358 raft_consensus.cc:3060] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.633875 18432 raft_consensus.cc:3055] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:16.634229 18432 raft_consensus.cc:740] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:16.635005 18432 consensus_queue.cc:260] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.636183 18432 raft_consensus.cc:3060] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.640954 18358 raft_consensus.cc:2468] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 2.
I20260709 13:17:16.642334 18250 leader_election.cc:304] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:16.643004 18432 raft_consensus.cc:2468] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 2.
I20260709 13:17:16.643165 18521 raft_consensus.cc:2804] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:16.643751 18521 raft_consensus.cc:697] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Becoming Leader. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:16.644531 18521 consensus_queue.cc:237] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.651652 18182 catalog_manager.cc:5697] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 reported cstate change: term changed from 1 to 2, leader changed from 0a3115e6843944079e3fad8d05197029 (127.6.198.67) to 88766a58641e46449f46227d9e09bd68 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "88766a58641e46449f46227d9e09bd68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.652781 18631 raft_consensus.cc:993] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c: : Instructing follower 88766a58641e46449f46227d9e09bd68 to start an election
I20260709 13:17:16.653295 18632 raft_consensus.cc:1081] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Signalling peer 88766a58641e46449f46227d9e09bd68 to start an election
I20260709 13:17:16.654925 18282 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb850b6ebc584c56b8b97b4b6f5b0819"
dest_uuid: "88766a58641e46449f46227d9e09bd68"
 from {username='slave'} at 127.0.0.1:37584
I20260709 13:17:16.655450 18282 raft_consensus.cc:493] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:16.655798 18282 raft_consensus.cc:3060] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.661762 18282 raft_consensus.cc:515] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.663530 18282 leader_election.cc:290] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 2 election: Requested vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:16.664255 18358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb850b6ebc584c56b8b97b4b6f5b0819" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:16.664565 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb850b6ebc584c56b8b97b4b6f5b0819" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:16.665005 18358 raft_consensus.cc:3055] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:16.665061 18432 raft_consensus.cc:3060] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.669405 18432 raft_consensus.cc:2468] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 2.
I20260709 13:17:16.669775 18358 raft_consensus.cc:740] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:16.670442 18250 leader_election.cc:304] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:16.670578 18358 consensus_queue.cc:260] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.671190 18521 raft_consensus.cc:2804] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:16.671612 18521 raft_consensus.cc:697] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Becoming Leader. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:16.671720 18358 raft_consensus.cc:3060] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.672340 18521 consensus_queue.cc:237] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.677505 18358 raft_consensus.cc:2468] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 2.
I20260709 13:17:16.679366 18182 catalog_manager.cc:5697] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 reported cstate change: term changed from 1 to 2, leader changed from 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66) to 88766a58641e46449f46227d9e09bd68 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "88766a58641e46449f46227d9e09bd68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.717247 18594 raft_consensus.cc:993] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029: : Instructing follower 88766a58641e46449f46227d9e09bd68 to start an election
I20260709 13:17:16.717725 18595 raft_consensus.cc:1081] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Signalling peer 88766a58641e46449f46227d9e09bd68 to start an election
I20260709 13:17:16.719168 18282 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "65f8d06d5edc4472ae7573fef6c1ad5a"
dest_uuid: "88766a58641e46449f46227d9e09bd68"
 from {username='slave'} at 127.0.0.1:37574
I20260709 13:17:16.719700 18282 raft_consensus.cc:493] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:16.720009 18282 raft_consensus.cc:3060] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.724985 18282 raft_consensus.cc:515] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.726567 18282 leader_election.cc:290] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 2 election: Requested vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:16.727639 18358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f8d06d5edc4472ae7573fef6c1ad5a" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:16.727847 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65f8d06d5edc4472ae7573fef6c1ad5a" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:16.728291 18358 raft_consensus.cc:3060] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.728386 18432 raft_consensus.cc:3055] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:16.728757 18432 raft_consensus.cc:740] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:16.729524 18432 consensus_queue.cc:260] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.730706 18432 raft_consensus.cc:3060] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.735296 18358 raft_consensus.cc:2468] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 2.
I20260709 13:17:16.736497 18250 leader_election.cc:304] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:16.736904 18432 raft_consensus.cc:2468] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 2.
I20260709 13:17:16.737288 18521 raft_consensus.cc:2804] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:16.738253 18521 raft_consensus.cc:697] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Becoming Leader. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:16.739169 18521 consensus_queue.cc:237] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.745281 18182 catalog_manager.cc:5697] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 reported cstate change: term changed from 1 to 2, leader changed from 0a3115e6843944079e3fad8d05197029 (127.6.198.67) to 88766a58641e46449f46227d9e09bd68 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "88766a58641e46449f46227d9e09bd68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.823161 18594 raft_consensus.cc:993] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029: : Instructing follower 88766a58641e46449f46227d9e09bd68 to start an election
I20260709 13:17:16.823621 18594 raft_consensus.cc:1081] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Signalling peer 88766a58641e46449f46227d9e09bd68 to start an election
I20260709 13:17:16.824998 18282 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ade415aef06a450eb50a7907a22d5849"
dest_uuid: "88766a58641e46449f46227d9e09bd68"
 from {username='slave'} at 127.0.0.1:37574
I20260709 13:17:16.825378 18282 raft_consensus.cc:493] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:16.825599 18282 raft_consensus.cc:3060] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.830394 18282 raft_consensus.cc:515] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.831923 18282 leader_election.cc:290] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 2 election: Requested vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:16.832748 18358 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade415aef06a450eb50a7907a22d5849" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:16.833001 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade415aef06a450eb50a7907a22d5849" candidate_uuid: "88766a58641e46449f46227d9e09bd68" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:16.833400 18358 raft_consensus.cc:3060] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.833496 18432 raft_consensus.cc:3055] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:16.833855 18432 raft_consensus.cc:740] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:16.834614 18432 consensus_queue.cc:260] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.835762 18432 raft_consensus.cc:3060] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.840214 18358 raft_consensus.cc:2468] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 2.
I20260709 13:17:16.841563 18250 leader_election.cc:304] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:16.842048 18432 raft_consensus.cc:2468] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88766a58641e46449f46227d9e09bd68 in term 2.
I20260709 13:17:16.842260 18521 raft_consensus.cc:2804] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:16.842829 18521 raft_consensus.cc:697] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Becoming Leader. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:16.843626 18521 consensus_queue.cc:237] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.850435 18182 catalog_manager.cc:5697] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 reported cstate change: term changed from 1 to 2, leader changed from 0a3115e6843944079e3fad8d05197029 (127.6.198.67) to 88766a58641e46449f46227d9e09bd68 (127.6.198.65). New cstate: current_term: 2 leader_uuid: "88766a58641e46449f46227d9e09bd68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.852548 18594 raft_consensus.cc:993] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029: : Instructing follower 5f3f6d22d9624fc2a34acafd98a6892c to start an election
I20260709 13:17:16.853072 18595 raft_consensus.cc:1081] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Signalling peer 5f3f6d22d9624fc2a34acafd98a6892c to start an election
I20260709 13:17:16.854666 18357 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed0588646d884ab19a290e8ac8fb025c"
dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
 from {username='slave'} at 127.0.0.1:44066
I20260709 13:17:16.855185 18357 raft_consensus.cc:493] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:16.855518 18357 raft_consensus.cc:3060] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.861572 18357 raft_consensus.cc:515] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.863193 18357 leader_election.cc:290] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 2 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 0a3115e6843944079e3fad8d05197029 (127.6.198.67:41093)
I20260709 13:17:16.864173 18282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0588646d884ab19a290e8ac8fb025c" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:16.864236 18432 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0588646d884ab19a290e8ac8fb025c" candidate_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a3115e6843944079e3fad8d05197029"
I20260709 13:17:16.864846 18282 raft_consensus.cc:3060] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.864939 18432 raft_consensus.cc:3055] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:16.865389 18432 raft_consensus.cc:740] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:16.866115 18432 consensus_queue.cc:260] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.867023 18432 raft_consensus.cc:3060] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:16.869500 18282 raft_consensus.cc:2468] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 2.
I20260709 13:17:16.870714 18324 leader_election.cc:304] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f3f6d22d9624fc2a34acafd98a6892c, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:16.871519 18432 raft_consensus.cc:2468] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f3f6d22d9624fc2a34acafd98a6892c in term 2.
I20260709 13:17:16.871649 18632 raft_consensus.cc:2804] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:16.872229 18632 raft_consensus.cc:697] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Becoming Leader. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:16.872977 18632 consensus_queue.cc:237] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:16.880012 18182 catalog_manager.cc:5697] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c reported cstate change: term changed from 1 to 2, leader changed from 0a3115e6843944079e3fad8d05197029 (127.6.198.67) to 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66). New cstate: current_term: 2 leader_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:16.914865 18432 raft_consensus.cc:1275] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Refusing update from remote peer 5f3f6d22d9624fc2a34acafd98a6892c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:16.916014 18632 consensus_queue.cc:1048] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:16.925710 18282 raft_consensus.cc:1275] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Refusing update from remote peer 5f3f6d22d9624fc2a34acafd98a6892c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:16.927296 18631 consensus_queue.cc:1048] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:16.944453 18282 raft_consensus.cc:1275] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Refusing update from remote peer 5f3f6d22d9624fc2a34acafd98a6892c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:16.945657 18631 consensus_queue.cc:1048] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:16.947974 18432 raft_consensus.cc:1275] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Refusing update from remote peer 5f3f6d22d9624fc2a34acafd98a6892c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:16.949344 18631 consensus_queue.cc:1048] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:17.043004 18357 raft_consensus.cc:1275] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Refusing update from remote peer 88766a58641e46449f46227d9e09bd68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:17.044098 18641 consensus_queue.cc:1048] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:17.051397 18432 raft_consensus.cc:1275] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Refusing update from remote peer 88766a58641e46449f46227d9e09bd68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:17.052971 18521 consensus_queue.cc:1048] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:17.134889 18357 raft_consensus.cc:1275] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Refusing update from remote peer 88766a58641e46449f46227d9e09bd68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:17.136140 18521 consensus_queue.cc:1048] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:17.146579 18432 raft_consensus.cc:1275] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Refusing update from remote peer 88766a58641e46449f46227d9e09bd68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:17.149317 18432 raft_consensus.cc:1275] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Refusing update from remote peer 88766a58641e46449f46227d9e09bd68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:17.148030 18521 consensus_queue.cc:1048] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:17:17.151422 18641 consensus_queue.cc:1048] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:17.162955 18357 raft_consensus.cc:1275] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Refusing update from remote peer 88766a58641e46449f46227d9e09bd68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:17.164337 18641 consensus_queue.cc:1048] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:17.355527 18433 raft_consensus.cc:1275] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Refusing update from remote peer 88766a58641e46449f46227d9e09bd68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:17.357061 18649 consensus_queue.cc:1048] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:17.358873 18357 raft_consensus.cc:1275] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Refusing update from remote peer 88766a58641e46449f46227d9e09bd68: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:17.360402 18521 consensus_queue.cc:1048] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:17.373823 18433 raft_consensus.cc:1275] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Refusing update from remote peer 5f3f6d22d9624fc2a34acafd98a6892c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:17.375145 18632 consensus_queue.cc:1048] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:17.392926 18282 raft_consensus.cc:1275] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Refusing update from remote peer 5f3f6d22d9624fc2a34acafd98a6892c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:17.395270 18631 consensus_queue.cc:1048] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:19.324087 18182 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46078:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:17:19.342226 18182 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46078:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:17:19.379887  6937 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:17:19.610601 18680 log.cc:927] Time spent T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:17:19.634981 18284 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb850b6ebc584c56b8b97b4b6f5b0819"
dest_uuid: "88766a58641e46449f46227d9e09bd68"
mode: GRACEFUL
new_leader_uuid: "0a3115e6843944079e3fad8d05197029"
 from {username='slave'} at 127.0.0.1:37600
I20260709 13:17:19.635758 18284 raft_consensus.cc:606] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Received request to transfer leadership to TS 0a3115e6843944079e3fad8d05197029
I20260709 13:17:19.611933 18700 tablet.cc:1724] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.684324 18711 tablet.cc:1724] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.687098 18716 tablet.cc:1724] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:17:19.691219 18459 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:65f8d06d5edc4472ae7573fef6c1ad5a)
W20260709 13:17:19.700497 18459 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:65f8d06d5edc4472ae7573fef6c1ad5a)
W20260709 13:17:19.696908 18706 log.cc:927] Time spent T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 13:17:19.709193 18684 log.cc:927] Time spent T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029: Append to log took a long time: real 0.179s	user 0.003s	sys 0.001s
I20260709 13:17:19.692787 18713 tablet.cc:1724] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.724332 18717 tablet.cc:1724] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:17:19.735615 18686 log.cc:927] Time spent T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68: Append to log took a long time: real 0.220s	user 0.003s	sys 0.000s
I20260709 13:17:19.752100 18720 tablet.cc:1724] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:17:19.744542 18385 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:65f8d06d5edc4472ae7573fef6c1ad5a)
I20260709 13:17:19.756614 18704 raft_consensus.cc:993] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68: : Instructing follower 0a3115e6843944079e3fad8d05197029 to start an election
I20260709 13:17:19.778252 18642 raft_consensus.cc:1081] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Signalling peer 0a3115e6843944079e3fad8d05197029 to start an election
W20260709 13:17:19.770897 18398 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:38301, user_credentials={real_user=slave}} blocked reactor thread for 52652.4us
I20260709 13:17:19.791194 18726 tablet.cc:1724] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.795212 18725 tablet.cc:1724] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.801620 18427 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb850b6ebc584c56b8b97b4b6f5b0819"
dest_uuid: "0a3115e6843944079e3fad8d05197029"
 from {username='slave'} at 127.0.0.1:48136
I20260709 13:17:19.802290 18427 raft_consensus.cc:493] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:19.802683 18427 raft_consensus.cc:3060] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:19.802031 18724 tablet.cc:1724] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.806285 18359 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed0588646d884ab19a290e8ac8fb025c"
dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
mode: GRACEFUL
new_leader_uuid: "0a3115e6843944079e3fad8d05197029"
 from {username='slave'} at 127.0.0.1:44098
I20260709 13:17:19.816206 18427 raft_consensus.cc:515] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:19.813544 18713 tablet.cc:1724] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.827549 18724 tablet.cc:1724] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.854614 18711 tablet.cc:1724] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.855360 18717 tablet.cc:1724] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.862593 18359 raft_consensus.cc:606] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Received request to transfer leadership to TS 0a3115e6843944079e3fad8d05197029
I20260709 13:17:19.837610 18722 tablet.cc:1724] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.854631 18700 tablet.cc:1724] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.870493 18716 tablet.cc:1724] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.874886  6937 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:17:19.875568  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:19.869853 18727 tablet.cc:1724] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.879096 18427 leader_election.cc:290] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 3 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:19.892544 18728 tablet.cc:1724] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.894747 18725 tablet.cc:1724] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.900115 18713 tablet.cc:1724] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.905215 18724 tablet.cc:1724] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.911546 18281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb850b6ebc584c56b8b97b4b6f5b0819" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:19.912182 18281 raft_consensus.cc:3055] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:19.912503 18281 raft_consensus.cc:740] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:19.913317 18281 consensus_queue.cc:260] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:19.914585 18281 raft_consensus.cc:3060] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:19.914799 18726 tablet.cc:1724] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.915367 18720 tablet.cc:1724] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.906785 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb850b6ebc584c56b8b97b4b6f5b0819" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:19.918038 18280 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86bd5ede009f4c36813af6db8e97baa6"
dest_uuid: "88766a58641e46449f46227d9e09bd68"
mode: GRACEFUL
new_leader_uuid: "0a3115e6843944079e3fad8d05197029"
 from {username='slave'} at 127.0.0.1:37600
I20260709 13:17:19.923724 18359 raft_consensus.cc:3060] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:19.923765 18280 raft_consensus.cc:606] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Received request to transfer leadership to TS 0a3115e6843944079e3fad8d05197029
I20260709 13:17:19.927189 18356 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c338bb3d5e064f07bc473134a4090cb4"
dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
mode: GRACEFUL
new_leader_uuid: "0a3115e6843944079e3fad8d05197029"
 from {username='slave'} at 127.0.0.1:44098
I20260709 13:17:19.927794 18356 raft_consensus.cc:606] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Received request to transfer leadership to TS 0a3115e6843944079e3fad8d05197029
I20260709 13:17:19.931293 18642 raft_consensus.cc:993] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68: : Instructing follower 0a3115e6843944079e3fad8d05197029 to start an election
I20260709 13:17:19.931450  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:19.931772 18642 raft_consensus.cc:1081] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Signalling peer 0a3115e6843944079e3fad8d05197029 to start an election
I20260709 13:17:19.931919 18359 raft_consensus.cc:2468] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 3.
I20260709 13:17:19.932153  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:19.933647 18399 leader_election.cc:304] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 
I20260709 13:17:19.935091 18729 raft_consensus.cc:2804] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:19.935839 18729 raft_consensus.cc:697] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 3 LEADER]: Becoming Leader. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:19.937449 18729 consensus_queue.cc:237] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:19.937639 18722 tablet.cc:1724] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.936506 18429 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86bd5ede009f4c36813af6db8e97baa6"
dest_uuid: "0a3115e6843944079e3fad8d05197029"
 from {username='slave'} at 127.0.0.1:48136
I20260709 13:17:19.945067 18727 tablet.cc:1724] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.947017 18429 raft_consensus.cc:493] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:19.948933 18429 raft_consensus.cc:3060] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:19.909965 18711 tablet.cc:1724] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.906157 18700 tablet.cc:1724] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:19.954187 18281 raft_consensus.cc:2468] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 3.
I20260709 13:17:19.962073 18429 raft_consensus.cc:515] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } }
I20260709 13:17:19.979554 18429 leader_election.cc:290] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 2 election: Requested vote from peers 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643), 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301)
I20260709 13:17:19.980613 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86bd5ede009f4c36813af6db8e97baa6" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:19.981594 18359 raft_consensus.cc:3060] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:19.980775 18281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86bd5ede009f4c36813af6db8e97baa6" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:19.983191 18281 raft_consensus.cc:3055] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:19.983559 18281 raft_consensus.cc:740] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 88766a58641e46449f46227d9e09bd68, State: Running, Role: LEADER
I20260709 13:17:19.984503 18281 consensus_queue.cc:260] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } }
I20260709 13:17:19.986078 18281 raft_consensus.cc:3060] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:19.990633 18359 raft_consensus.cc:2468] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 2.
I20260709 13:17:19.992008 18399 leader_election.cc:304] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 5f3f6d22d9624fc2a34acafd98a6892c; no voters: 
I20260709 13:17:19.992774 18729 raft_consensus.cc:2804] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:19.993288 18729 raft_consensus.cc:697] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 2 LEADER]: Becoming Leader. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:19.993309 18281 raft_consensus.cc:2468] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 2.
I20260709 13:17:19.994247 18729 consensus_queue.cc:237] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } }
I20260709 13:17:19.997730 18178 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755: soft_deleted_table [id=5ade366e0a654ce6b483be0286f4f012] alter complete (version 1)
I20260709 13:17:20.011801 18182 catalog_manager.cc:5697] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 reported cstate change: term changed from 1 to 2, leader changed from 88766a58641e46449f46227d9e09bd68 (127.6.198.65) to 0a3115e6843944079e3fad8d05197029 (127.6.198.67). New cstate: current_term: 2 leader_uuid: "0a3115e6843944079e3fad8d05197029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:20.013578 18182 catalog_manager.cc:5697] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 reported cstate change: term changed from 2 to 3, leader changed from 88766a58641e46449f46227d9e09bd68 (127.6.198.65) to 0a3115e6843944079e3fad8d05197029 (127.6.198.67). New cstate: current_term: 3 leader_uuid: "0a3115e6843944079e3fad8d05197029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:20.152675 18631 raft_consensus.cc:993] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c: : Instructing follower 0a3115e6843944079e3fad8d05197029 to start an election
I20260709 13:17:20.153179 18683 raft_consensus.cc:1081] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Signalling peer 0a3115e6843944079e3fad8d05197029 to start an election
I20260709 13:17:20.154536 18429 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c338bb3d5e064f07bc473134a4090cb4"
dest_uuid: "0a3115e6843944079e3fad8d05197029"
 from {username='slave'} at 127.0.0.1:48130
I20260709 13:17:20.154975 18429 raft_consensus.cc:493] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:20.155257 18429 raft_consensus.cc:3060] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:20.159934 18429 raft_consensus.cc:515] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:20.161419 18429 leader_election.cc:290] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 3 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:20.162160 18281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c338bb3d5e064f07bc473134a4090cb4" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:20.162427 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c338bb3d5e064f07bc473134a4090cb4" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:20.162829 18281 raft_consensus.cc:3060] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:20.163000 18359 raft_consensus.cc:3055] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:20.163386 18359 raft_consensus.cc:740] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:20.164258 18359 consensus_queue.cc:260] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:20.165534 18359 raft_consensus.cc:3060] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:20.169677 18281 raft_consensus.cc:2468] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 3.
I20260709 13:17:20.170953 18398 leader_election.cc:304] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:20.171739 18729 raft_consensus.cc:2804] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:20.172420 18359 raft_consensus.cc:2468] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 3.
I20260709 13:17:20.172477 18729 raft_consensus.cc:697] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 3 LEADER]: Becoming Leader. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:20.173197 18631 raft_consensus.cc:993] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c: : Instructing follower 0a3115e6843944079e3fad8d05197029 to start an election
I20260709 13:17:20.173655 18631 raft_consensus.cc:1081] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Signalling peer 0a3115e6843944079e3fad8d05197029 to start an election
I20260709 13:17:20.173604 18729 consensus_queue.cc:237] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:20.175269 18429 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed0588646d884ab19a290e8ac8fb025c"
dest_uuid: "0a3115e6843944079e3fad8d05197029"
 from {username='slave'} at 127.0.0.1:48130
I20260709 13:17:20.175791 18429 raft_consensus.cc:493] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:20.176164 18429 raft_consensus.cc:3060] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:20.182181 18429 raft_consensus.cc:515] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:20.184191 18429 leader_election.cc:290] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 3 election: Requested vote from peers 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301), 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643)
I20260709 13:17:20.184280 18182 catalog_manager.cc:5697] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 reported cstate change: term changed from 2 to 3, leader changed from 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66) to 0a3115e6843944079e3fad8d05197029 (127.6.198.67). New cstate: current_term: 3 leader_uuid: "0a3115e6843944079e3fad8d05197029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:20.184757 18281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0588646d884ab19a290e8ac8fb025c" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "88766a58641e46449f46227d9e09bd68"
I20260709 13:17:20.185225 18359 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0588646d884ab19a290e8ac8fb025c" candidate_uuid: "0a3115e6843944079e3fad8d05197029" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "5f3f6d22d9624fc2a34acafd98a6892c"
I20260709 13:17:20.185844 18281 raft_consensus.cc:3060] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:20.185883 18359 raft_consensus.cc:3055] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:20.186376 18359 raft_consensus.cc:740] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5f3f6d22d9624fc2a34acafd98a6892c, State: Running, Role: LEADER
I20260709 13:17:20.187194 18359 consensus_queue.cc:260] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:20.188395 18359 raft_consensus.cc:3060] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:20.192965 18281 raft_consensus.cc:2468] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 3.
I20260709 13:17:20.194908 18398 leader_election.cc:304] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a3115e6843944079e3fad8d05197029, 88766a58641e46449f46227d9e09bd68; no voters: 
I20260709 13:17:20.195815 18359 raft_consensus.cc:2468] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a3115e6843944079e3fad8d05197029 in term 3.
I20260709 13:17:20.199350 18729 raft_consensus.cc:2804] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:20.200600 18729 raft_consensus.cc:697] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 3 LEADER]: Becoming Leader. State: Replica: 0a3115e6843944079e3fad8d05197029, State: Running, Role: LEADER
I20260709 13:17:20.201972 18729 consensus_queue.cc:237] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } }
I20260709 13:17:20.210429 18182 catalog_manager.cc:5697] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 reported cstate change: term changed from 2 to 3, leader changed from 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66) to 0a3115e6843944079e3fad8d05197029 (127.6.198.67). New cstate: current_term: 3 leader_uuid: "0a3115e6843944079e3fad8d05197029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a3115e6843944079e3fad8d05197029" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 41093 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:20.332113 18281 raft_consensus.cc:1275] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 3 FOLLOWER]: Refusing update from remote peer 0a3115e6843944079e3fad8d05197029: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:20.333249 18695 consensus_queue.cc:1048] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:20.343310 18359 raft_consensus.cc:1275] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 3 FOLLOWER]: Refusing update from remote peer 0a3115e6843944079e3fad8d05197029: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:20.344966 18695 consensus_queue.cc:1048] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:20.530877 18359 raft_consensus.cc:1275] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Refusing update from remote peer 0a3115e6843944079e3fad8d05197029: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:20.531950 18736 consensus_queue.cc:1048] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:20.539556 18281 raft_consensus.cc:1275] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Refusing update from remote peer 0a3115e6843944079e3fad8d05197029: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:20.540979 18695 consensus_queue.cc:1048] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:20.724283 18359 raft_consensus.cc:1275] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 3 FOLLOWER]: Refusing update from remote peer 0a3115e6843944079e3fad8d05197029: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:20.725514 18695 consensus_queue.cc:1048] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:20.728863 18356 raft_consensus.cc:1275] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 3 FOLLOWER]: Refusing update from remote peer 0a3115e6843944079e3fad8d05197029: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:20.730106 18281 raft_consensus.cc:1275] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68 [term 3 FOLLOWER]: Refusing update from remote peer 0a3115e6843944079e3fad8d05197029: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:20.730463 18695 consensus_queue.cc:1048] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f3f6d22d9624fc2a34acafd98a6892c" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 35643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:20.732331 18729 consensus_queue.cc:1048] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:20.741505 18280 raft_consensus.cc:1275] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [term 3 FOLLOWER]: Refusing update from remote peer 0a3115e6843944079e3fad8d05197029: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:20.744451 18695 consensus_queue.cc:1048] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88766a58641e46449f46227d9e09bd68" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:20.991384  6937 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:17:20.997114  6937 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:17:20.997483  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:21.002162  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:17:21.002549  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:22.029927  6937 tablet_server.cc:179] TabletServer@127.6.198.65:0 shutting down...
I20260709 13:17:22.064899  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:22.065907  6937 tablet_replica.cc:333] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.067165  6937 raft_consensus.cc:2243] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.068305  6937 raft_consensus.cc:2272] T 8cdb694dd8674ed88ba584fcf49328c9 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.070966  6937 tablet_replica.cc:333] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.071619  6937 raft_consensus.cc:2243] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.072182  6937 raft_consensus.cc:2272] T 9706990d796b40b7ad65514a7ce20f56 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.074728  6937 tablet_replica.cc:333] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.075479  6937 raft_consensus.cc:2243] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.076061  6937 raft_consensus.cc:2272] T ed0588646d884ab19a290e8ac8fb025c P 88766a58641e46449f46227d9e09bd68 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.078578  6937 tablet_replica.cc:333] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.079187  6937 raft_consensus.cc:2243] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.080066  6937 raft_consensus.cc:2272] T 87ecfb9a373b4fc49adadc26edb303a6 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.082630  6937 tablet_replica.cc:333] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.083235  6937 raft_consensus.cc:2243] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.083770  6937 raft_consensus.cc:2272] T de81f3e8dc484f52a9e49a1a8b2f8945 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.085892  6937 tablet_replica.cc:333] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.086445  6937 raft_consensus.cc:2243] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.087320  6937 raft_consensus.cc:2272] T 0d7e9ac119d140b091ea674bbe3f7a71 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.089762  6937 tablet_replica.cc:333] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.090482  6937 raft_consensus.cc:2243] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.091034  6937 raft_consensus.cc:2272] T c170e078318c43fbbaca55225e42b6b7 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.093364  6937 tablet_replica.cc:333] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.093935  6937 raft_consensus.cc:2243] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.094507  6937 raft_consensus.cc:2272] T c338bb3d5e064f07bc473134a4090cb4 P 88766a58641e46449f46227d9e09bd68 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.096676  6937 tablet_replica.cc:333] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.097220  6937 raft_consensus.cc:2243] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.098115  6937 raft_consensus.cc:2272] T 9ab85571c50949298324235b64c91bac P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.100412  6937 tablet_replica.cc:333] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.101034  6937 raft_consensus.cc:2243] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.101883  6937 raft_consensus.cc:2272] T ade415aef06a450eb50a7907a22d5849 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.104169  6937 tablet_replica.cc:333] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.104703  6937 raft_consensus.cc:2243] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.105188  6937 raft_consensus.cc:2272] T 86bd5ede009f4c36813af6db8e97baa6 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.107435  6937 tablet_replica.cc:333] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.107977  6937 raft_consensus.cc:2243] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.108464  6937 raft_consensus.cc:2272] T a84920b1dc0243028f50c02f24fb1f8b P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.110677  6937 tablet_replica.cc:333] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.111209  6937 raft_consensus.cc:2243] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.111720  6937 raft_consensus.cc:2272] T c5ac76c3dcf5452db28d5619ff1c913b P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.113991  6937 tablet_replica.cc:333] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.114545  6937 raft_consensus.cc:2243] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.115392  6937 raft_consensus.cc:2272] T 65f8d06d5edc4472ae7573fef6c1ad5a P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.117731  6937 tablet_replica.cc:333] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.118225  6937 raft_consensus.cc:2243] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.118693  6937 raft_consensus.cc:2272] T c8480c976dc449d0b38cc0e892e699c2 P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.120877  6937 tablet_replica.cc:333] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.121326  6937 raft_consensus.cc:2243] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.121783  6937 raft_consensus.cc:2272] T 1d220a74f8fc451ca4c52f0d3143131f P 88766a58641e46449f46227d9e09bd68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.123934  6937 tablet_replica.cc:333] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.124431  6937 raft_consensus.cc:2243] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.125309  6937 raft_consensus.cc:2272] T ad42bcd434774d8c8cbc5dd5188ba040 P 88766a58641e46449f46227d9e09bd68 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.127944  6937 tablet_replica.cc:333] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68: stopping tablet replica
I20260709 13:17:22.128533  6937 raft_consensus.cc:2243] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.129254  6937 raft_consensus.cc:2272] T cb850b6ebc584c56b8b97b4b6f5b0819 P 88766a58641e46449f46227d9e09bd68 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:22.141606 18324 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
W20260709 13:17:22.145130 18324 consensus_peers.cc:597] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c -> Peer 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301): Couldn't send request to peer 88766a58641e46449f46227d9e09bd68. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:22.175401  6937 tablet_server.cc:196] TabletServer@127.6.198.65:0 shutdown complete.
W20260709 13:17:22.185351 18324 consensus_peers.cc:597] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c -> Peer 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301): Couldn't send request to peer 88766a58641e46449f46227d9e09bd68. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:38301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:22.185554 18398 consensus_peers.cc:597] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 -> Peer 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301): Couldn't send request to peer 88766a58641e46449f46227d9e09bd68. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:38301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:22.195683 18398 consensus_peers.cc:597] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 -> Peer 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301): Couldn't send request to peer 88766a58641e46449f46227d9e09bd68. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:38301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:22.226397 18324 consensus_peers.cc:597] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c -> Peer 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301): Couldn't send request to peer 88766a58641e46449f46227d9e09bd68. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:38301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:22.226559  6937 tablet_server.cc:179] TabletServer@127.6.198.66:0 shutting down...
W20260709 13:17:22.233084 18398 consensus_peers.cc:597] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 -> Peer 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301): Couldn't send request to peer 88766a58641e46449f46227d9e09bd68. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:38301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:22.237546 18324 consensus_peers.cc:597] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c -> Peer 88766a58641e46449f46227d9e09bd68 (127.6.198.65:38301): Couldn't send request to peer 88766a58641e46449f46227d9e09bd68. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:38301: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:22.258877  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:22.259618  6937 tablet_replica.cc:333] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.260502  6937 raft_consensus.cc:2243] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.261507  6937 raft_consensus.cc:2272] T 8cdb694dd8674ed88ba584fcf49328c9 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.264263  6937 tablet_replica.cc:333] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.264906  6937 raft_consensus.cc:2243] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.265905  6937 raft_consensus.cc:2272] T 9706990d796b40b7ad65514a7ce20f56 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.268332  6937 tablet_replica.cc:333] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.269187  6937 raft_consensus.cc:2243] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.269670  6937 raft_consensus.cc:2272] T ed0588646d884ab19a290e8ac8fb025c P 5f3f6d22d9624fc2a34acafd98a6892c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.271770  6937 tablet_replica.cc:333] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.272471  6937 raft_consensus.cc:2243] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.273185  6937 raft_consensus.cc:2272] T 87ecfb9a373b4fc49adadc26edb303a6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.275388  6937 tablet_replica.cc:333] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.275902  6937 raft_consensus.cc:2243] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.276312  6937 raft_consensus.cc:2272] T 1d220a74f8fc451ca4c52f0d3143131f P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.278985  6937 tablet_replica.cc:333] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.279500  6937 raft_consensus.cc:2243] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.280344  6937 raft_consensus.cc:2272] T c8480c976dc449d0b38cc0e892e699c2 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.282485  6937 tablet_replica.cc:333] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.282965  6937 raft_consensus.cc:2243] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.283471  6937 raft_consensus.cc:2272] T cb850b6ebc584c56b8b97b4b6f5b0819 P 5f3f6d22d9624fc2a34acafd98a6892c [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.285642  6937 tablet_replica.cc:333] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.286116  6937 raft_consensus.cc:2243] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.286574  6937 raft_consensus.cc:2272] T 65f8d06d5edc4472ae7573fef6c1ad5a P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.288734  6937 tablet_replica.cc:333] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.289356  6937 raft_consensus.cc:2243] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.290194  6937 raft_consensus.cc:2272] T c5ac76c3dcf5452db28d5619ff1c913b P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.292395  6937 tablet_replica.cc:333] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.292955  6937 raft_consensus.cc:2243] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.293401  6937 raft_consensus.cc:2272] T ade415aef06a450eb50a7907a22d5849 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.295409  6937 tablet_replica.cc:333] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.295861  6937 raft_consensus.cc:2243] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.296283  6937 raft_consensus.cc:2272] T 86bd5ede009f4c36813af6db8e97baa6 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.298430  6937 tablet_replica.cc:333] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.298892  6937 raft_consensus.cc:2243] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.299661  6937 raft_consensus.cc:2272] T a84920b1dc0243028f50c02f24fb1f8b P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.301826  6937 tablet_replica.cc:333] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.302290  6937 raft_consensus.cc:2243] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.303104  6937 raft_consensus.cc:2272] T de81f3e8dc484f52a9e49a1a8b2f8945 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.305383  6937 tablet_replica.cc:333] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.305826  6937 raft_consensus.cc:2243] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.306377  6937 raft_consensus.cc:2272] T 0d7e9ac119d140b091ea674bbe3f7a71 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.308885  6937 tablet_replica.cc:333] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.309458  6937 raft_consensus.cc:2243] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.310148  6937 raft_consensus.cc:2272] T ad42bcd434774d8c8cbc5dd5188ba040 P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.313416  6937 tablet_replica.cc:333] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
W20260709 13:17:22.313450 18399 consensus_peers.cc:597] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 -> Peer 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643): Couldn't send request to peer 5f3f6d22d9624fc2a34acafd98a6892c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:22.314196  6937 raft_consensus.cc:2243] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.314826  6937 raft_consensus.cc:2272] T 9ab85571c50949298324235b64c91bac P 5f3f6d22d9624fc2a34acafd98a6892c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.317088  6937 tablet_replica.cc:333] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.317607  6937 raft_consensus.cc:2243] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.318169  6937 raft_consensus.cc:2272] T c170e078318c43fbbaca55225e42b6b7 P 5f3f6d22d9624fc2a34acafd98a6892c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.320432  6937 tablet_replica.cc:333] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c: stopping tablet replica
I20260709 13:17:22.321048  6937 raft_consensus.cc:2243] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.321532  6937 raft_consensus.cc:2272] T c338bb3d5e064f07bc473134a4090cb4 P 5f3f6d22d9624fc2a34acafd98a6892c [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:22.338999 18399 consensus_peers.cc:597] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 -> Peer 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643): Couldn't send request to peer 5f3f6d22d9624fc2a34acafd98a6892c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:22.370061  6937 tablet_server.cc:196] TabletServer@127.6.198.66:0 shutdown complete.
W20260709 13:17:22.381911 18399 consensus_peers.cc:597] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 -> Peer 5f3f6d22d9624fc2a34acafd98a6892c (127.6.198.66:35643): Couldn't send request to peer 5f3f6d22d9624fc2a34acafd98a6892c. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:35643: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:22.410125  6937 tablet_server.cc:179] TabletServer@127.6.198.67:0 shutting down...
I20260709 13:17:22.439114  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:22.439900  6937 tablet_replica.cc:333] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.440606  6937 raft_consensus.cc:2243] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.441089  6937 raft_consensus.cc:2272] T 8cdb694dd8674ed88ba584fcf49328c9 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.443236  6937 tablet_replica.cc:333] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.443807  6937 raft_consensus.cc:2243] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.444635  6937 raft_consensus.cc:2272] T cb850b6ebc584c56b8b97b4b6f5b0819 P 0a3115e6843944079e3fad8d05197029 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.446599  6937 tablet_replica.cc:333] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.447157  6937 raft_consensus.cc:2243] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.447646  6937 raft_consensus.cc:2272] T 9706990d796b40b7ad65514a7ce20f56 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.449558  6937 tablet_replica.cc:333] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.450102  6937 raft_consensus.cc:2243] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.450902  6937 raft_consensus.cc:2272] T ed0588646d884ab19a290e8ac8fb025c P 0a3115e6843944079e3fad8d05197029 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.452797  6937 tablet_replica.cc:333] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.453289  6937 raft_consensus.cc:2243] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.453776  6937 raft_consensus.cc:2272] T 87ecfb9a373b4fc49adadc26edb303a6 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.456533  6937 tablet_replica.cc:333] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.457111  6937 raft_consensus.cc:2243] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.457597  6937 raft_consensus.cc:2272] T ade415aef06a450eb50a7907a22d5849 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.459547  6937 tablet_replica.cc:333] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.460067  6937 raft_consensus.cc:2243] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.460845  6937 raft_consensus.cc:2272] T 86bd5ede009f4c36813af6db8e97baa6 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.463001  6937 tablet_replica.cc:333] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.463524  6937 raft_consensus.cc:2243] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.463961  6937 raft_consensus.cc:2272] T a84920b1dc0243028f50c02f24fb1f8b P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.466008  6937 tablet_replica.cc:333] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.466553  6937 raft_consensus.cc:2243] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.467085  6937 raft_consensus.cc:2272] T 65f8d06d5edc4472ae7573fef6c1ad5a P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.469048  6937 tablet_replica.cc:333] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.469574  6937 raft_consensus.cc:2243] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.470013  6937 raft_consensus.cc:2272] T c5ac76c3dcf5452db28d5619ff1c913b P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.472062  6937 tablet_replica.cc:333] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.472597  6937 raft_consensus.cc:2243] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.473412  6937 raft_consensus.cc:2272] T 1d220a74f8fc451ca4c52f0d3143131f P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.475472  6937 tablet_replica.cc:333] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.475986  6937 raft_consensus.cc:2243] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.476389  6937 raft_consensus.cc:2272] T c8480c976dc449d0b38cc0e892e699c2 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.478291  6937 tablet_replica.cc:333] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.478818  6937 raft_consensus.cc:2243] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.479283  6937 raft_consensus.cc:2272] T ad42bcd434774d8c8cbc5dd5188ba040 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.481196  6937 tablet_replica.cc:333] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.481630  6937 raft_consensus.cc:2243] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.482045  6937 raft_consensus.cc:2272] T de81f3e8dc484f52a9e49a1a8b2f8945 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.483911  6937 tablet_replica.cc:333] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.484342  6937 raft_consensus.cc:2243] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.484727  6937 raft_consensus.cc:2272] T 0d7e9ac119d140b091ea674bbe3f7a71 P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.486578  6937 tablet_replica.cc:333] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.487015  6937 raft_consensus.cc:2243] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.487736  6937 raft_consensus.cc:2272] T c170e078318c43fbbaca55225e42b6b7 P 0a3115e6843944079e3fad8d05197029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.489650  6937 tablet_replica.cc:333] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.490075  6937 raft_consensus.cc:2243] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.490775  6937 raft_consensus.cc:2272] T c338bb3d5e064f07bc473134a4090cb4 P 0a3115e6843944079e3fad8d05197029 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.492640  6937 tablet_replica.cc:333] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029: stopping tablet replica
I20260709 13:17:22.493021  6937 raft_consensus.cc:2243] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:22.493400  6937 raft_consensus.cc:2272] T 9ab85571c50949298324235b64c91bac P 0a3115e6843944079e3fad8d05197029 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.532873  6937 tablet_server.cc:196] TabletServer@127.6.198.67:0 shutdown complete.
I20260709 13:17:22.567909  6937 master.cc:562] Master@127.6.198.126:44927 shutting down...
I20260709 13:17:22.590032  6937 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.590622  6937 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.590925  6937 tablet_replica.cc:333] T 00000000000000000000000000000000 P 97ea9e0a79d04bd691f7c46dd5df9755: stopping tablet replica
I20260709 13:17:22.610801  6937 master.cc:584] Master@127.6.198.126:44927 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (12680 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:17:22.661067  6937 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.6.198.126:41745
I20260709 13:17:22.662145  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:22.667862 18752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:22.667896 18751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:22.668380 18754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:22.669013  6937 server_base.cc:1061] running on GCE node
I20260709 13:17:22.669831  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:22.669999  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:22.670112  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603042670101 us; error 0 us; skew 500 ppm
I20260709 13:17:22.670704  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:22.672879  6937 webserver.cc:533] Webserver started at http://127.6.198.126:32895/ using document root <none> and password file <none>
I20260709 13:17:22.673277  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:22.673431  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:22.673632  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:22.674773  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/master-0-root/instance:
uuid: "799f3c691b394190874bfeefbd01c18c"
format_stamp: "Formatted at 2026-07-09 13:17:22 on dist-test-slave-tqzw"
I20260709 13:17:22.678755  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:17:22.681706 18759 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:22.682449  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:17:22.682717  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/master-0-root
uuid: "799f3c691b394190874bfeefbd01c18c"
format_stamp: "Formatted at 2026-07-09 13:17:22 on dist-test-slave-tqzw"
I20260709 13:17:22.682969  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:22.702085  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:22.703269  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:22.739130  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.126:41745
I20260709 13:17:22.739228 18811 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.126:41745 every 8 connection(s)
I20260709 13:17:22.742744 18812 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.753275 18812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c: Bootstrap starting.
I20260709 13:17:22.757807 18812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.761632 18812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c: No bootstrap required, opened a new log
I20260709 13:17:22.763773 18812 raft_consensus.cc:359] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799f3c691b394190874bfeefbd01c18c" member_type: VOTER }
I20260709 13:17:22.764204 18812 raft_consensus.cc:385] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.764406 18812 raft_consensus.cc:740] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 799f3c691b394190874bfeefbd01c18c, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.764971 18812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799f3c691b394190874bfeefbd01c18c" member_type: VOTER }
I20260709 13:17:22.765434 18812 raft_consensus.cc:399] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:22.765654 18812 raft_consensus.cc:493] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:22.765919 18812 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.770143 18812 raft_consensus.cc:515] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799f3c691b394190874bfeefbd01c18c" member_type: VOTER }
I20260709 13:17:22.770668 18812 leader_election.cc:304] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 799f3c691b394190874bfeefbd01c18c; no voters: 
I20260709 13:17:22.771665 18812 leader_election.cc:290] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:22.771997 18815 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:22.773291 18815 raft_consensus.cc:697] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [term 1 LEADER]: Becoming Leader. State: Replica: 799f3c691b394190874bfeefbd01c18c, State: Running, Role: LEADER
I20260709 13:17:22.773917 18815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799f3c691b394190874bfeefbd01c18c" member_type: VOTER }
I20260709 13:17:22.774592 18812 sys_catalog.cc:565] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:22.778561 18816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "799f3c691b394190874bfeefbd01c18c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799f3c691b394190874bfeefbd01c18c" member_type: VOTER } }
I20260709 13:17:22.778885 18817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 799f3c691b394190874bfeefbd01c18c. Latest consensus state: current_term: 1 leader_uuid: "799f3c691b394190874bfeefbd01c18c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799f3c691b394190874bfeefbd01c18c" member_type: VOTER } }
I20260709 13:17:22.779392 18816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:22.779637 18817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:22.782274 18822 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:22.790256 18822 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:22.791224  6937 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:22.798575 18822 catalog_manager.cc:1383] Generated new cluster ID: f2dae76843b84c7493f9e4de5828741c
I20260709 13:17:22.798805 18822 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:22.816763 18822 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:22.817937 18822 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:22.834635 18822 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c: Generated new TSK 0
I20260709 13:17:22.835114 18822 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:22.857633  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:22.863233 18833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:22.864190 18834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:22.865553 18836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:22.866309  6937 server_base.cc:1061] running on GCE node
I20260709 13:17:22.867194  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:22.867398  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:22.867563  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603042867532 us; error 0 us; skew 500 ppm
I20260709 13:17:22.868067  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:22.870159  6937 webserver.cc:533] Webserver started at http://127.6.198.65:36845/ using document root <none> and password file <none>
I20260709 13:17:22.870620  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:22.870802  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:22.871057  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:22.872066  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-0-root/instance:
uuid: "d1c86056b295463abc9760e7158f4bce"
format_stamp: "Formatted at 2026-07-09 13:17:22 on dist-test-slave-tqzw"
I20260709 13:17:22.876222  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:17:22.879148 18841 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:22.879853  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:17:22.880116  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-0-root
uuid: "d1c86056b295463abc9760e7158f4bce"
format_stamp: "Formatted at 2026-07-09 13:17:22 on dist-test-slave-tqzw"
I20260709 13:17:22.880331  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:22.895504  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:22.896551  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:22.897969  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:22.900048  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:22.900233  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:22.900475  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:22.900640  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:22.940235  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.65:34207
I20260709 13:17:22.940346 18903 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.65:34207 every 8 connection(s)
I20260709 13:17:22.946655  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:22.954375 18908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:22.955979 18909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:22.957844 18912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:22.958834  6937 server_base.cc:1061] running on GCE node
I20260709 13:17:22.962491  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:22.962787  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:22.962984  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603042962968 us; error 0 us; skew 500 ppm
I20260709 13:17:22.963783  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:22.963805 18904 heartbeater.cc:344] Connected to a master server at 127.6.198.126:41745
I20260709 13:17:22.964322 18904 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:22.965171 18904 heartbeater.cc:507] Master 127.6.198.126:41745 requested a full tablet report, sending...
I20260709 13:17:22.966750  6937 webserver.cc:533] Webserver started at http://127.6.198.66:41717/ using document root <none> and password file <none>
I20260709 13:17:22.967315  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:22.967520  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:22.967679 18777 ts_manager.cc:194] Registered new tserver with Master: d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207)
I20260709 13:17:22.967796  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:22.969187  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-1-root/instance:
uuid: "b37c12ab1f654afc812877fd13698893"
format_stamp: "Formatted at 2026-07-09 13:17:22 on dist-test-slave-tqzw"
I20260709 13:17:22.969779 18777 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60960
I20260709 13:17:22.973893  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.005s
I20260709 13:17:22.977735 18916 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:22.978641  6937 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:17:22.978899  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-1-root
uuid: "b37c12ab1f654afc812877fd13698893"
format_stamp: "Formatted at 2026-07-09 13:17:22 on dist-test-slave-tqzw"
I20260709 13:17:22.979140  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:23.002874  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:23.004067  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:23.005457  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:23.007627  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:23.007804  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.008271  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:23.008466  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.047652  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.66:39923
I20260709 13:17:23.047767 18978 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.66:39923 every 8 connection(s)
I20260709 13:17:23.052618  6937 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:23.060827 18982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:23.063298 18983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:23.064854 18985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:23.065377  6937 server_base.cc:1061] running on GCE node
I20260709 13:17:23.065443 18979 heartbeater.cc:344] Connected to a master server at 127.6.198.126:41745
I20260709 13:17:23.066109 18979 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:23.066573  6937 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:23.066820  6937 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:23.066911 18979 heartbeater.cc:507] Master 127.6.198.126:41745 requested a full tablet report, sending...
I20260709 13:17:23.067000  6937 hybrid_clock.cc:648] HybridClock initialized: now 1783603043066970 us; error 0 us; skew 500 ppm
I20260709 13:17:23.067934  6937 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:23.069353 18777 ts_manager.cc:194] Registered new tserver with Master: b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:23.070530  6937 webserver.cc:533] Webserver started at http://127.6.198.67:40383/ using document root <none> and password file <none>
I20260709 13:17:23.071058  6937 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:23.071079 18777 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60976
I20260709 13:17:23.071298  6937 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:23.071631  6937 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:23.072777  6937 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-2-root/instance:
uuid: "4bada1aa2db9455e874b4139f9ad1a40"
format_stamp: "Formatted at 2026-07-09 13:17:23 on dist-test-slave-tqzw"
I20260709 13:17:23.077301  6937 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:17:23.080994 18990 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.081869  6937 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:17:23.082232  6937 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-2-root
uuid: "4bada1aa2db9455e874b4139f9ad1a40"
format_stamp: "Formatted at 2026-07-09 13:17:23 on dist-test-slave-tqzw"
I20260709 13:17:23.082563  6937 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskE5yAt8/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602718985547-6937-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:23.113996  6937 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:23.115233  6937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:23.116698  6937 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:23.118944  6937 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:23.119170  6937 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.119411  6937 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:23.119586  6937 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.162926  6937 rpc_server.cc:307] RPC server started. Bound to: 127.6.198.67:37349
I20260709 13:17:23.163079 19052 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.6.198.67:37349 every 8 connection(s)
I20260709 13:17:23.176908 19053 heartbeater.cc:344] Connected to a master server at 127.6.198.126:41745
I20260709 13:17:23.177307 19053 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:23.178087 19053 heartbeater.cc:507] Master 127.6.198.126:41745 requested a full tablet report, sending...
I20260709 13:17:23.180455 18777 ts_manager.cc:194] Registered new tserver with Master: 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:23.181178  6937 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014955887s
I20260709 13:17:23.182271 18777 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60978
I20260709 13:17:23.206723 18777 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60982:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:23.209574 18777 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:23.296029 18944 tablet_service.cc:1511] Processing CreateTablet for tablet f1dedc61807b4f1fb0b0df1fd7408660 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:23.297547 18944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1dedc61807b4f1fb0b0df1fd7408660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.297185 18943 tablet_service.cc:1511] Processing CreateTablet for tablet 5a8849ca00284e83968f5c4d4f7a5f59 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:23.298640 18943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a8849ca00284e83968f5c4d4f7a5f59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.303086 18868 tablet_service.cc:1511] Processing CreateTablet for tablet 5a8849ca00284e83968f5c4d4f7a5f59 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:23.304571 18868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a8849ca00284e83968f5c4d4f7a5f59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.304821 18940 tablet_service.cc:1511] Processing CreateTablet for tablet a0a28ca432c040e090e4540740f46aa6 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:23.306142 18940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0a28ca432c040e090e4540740f46aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.313256 18869 tablet_service.cc:1511] Processing CreateTablet for tablet f1dedc61807b4f1fb0b0df1fd7408660 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:23.314635 18869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1dedc61807b4f1fb0b0df1fd7408660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.307505 18939 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0a1577fa0f4741ab0ee73851eb1294 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:23.315619 18939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0a1577fa0f4741ab0ee73851eb1294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.318753 18865 tablet_service.cc:1511] Processing CreateTablet for tablet a0a28ca432c040e090e4540740f46aa6 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:23.319864 18937 tablet_service.cc:1511] Processing CreateTablet for tablet cf5def05512948249903a00f64386054 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:23.320510 18865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0a28ca432c040e090e4540740f46aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.321307 18937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf5def05512948249903a00f64386054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.323288 18936 tablet_service.cc:1511] Processing CreateTablet for tablet 1e957ba42136417fba964b3408dfec86 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:23.324774 18936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e957ba42136417fba964b3408dfec86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.324465 18863 tablet_service.cc:1511] Processing CreateTablet for tablet 8b1c41abe4964625829a184924c60d3a (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:23.325785 18863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1c41abe4964625829a184924c60d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.308673 18866 tablet_service.cc:1511] Processing CreateTablet for tablet 6d0c9c2b6ccc40ee8a9b8265ef14ad8a (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:23.327651 18866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d0c9c2b6ccc40ee8a9b8265ef14ad8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.309255 18941 tablet_service.cc:1511] Processing CreateTablet for tablet 6d0c9c2b6ccc40ee8a9b8265ef14ad8a (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:23.328799 18941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d0c9c2b6ccc40ee8a9b8265ef14ad8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.310990 18938 tablet_service.cc:1511] Processing CreateTablet for tablet 8b1c41abe4964625829a184924c60d3a (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:23.331228 18938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1c41abe4964625829a184924c60d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.331485 18862 tablet_service.cc:1511] Processing CreateTablet for tablet cf5def05512948249903a00f64386054 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:23.332844 18862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf5def05512948249903a00f64386054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.311509 18864 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0a1577fa0f4741ab0ee73851eb1294 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:23.335196 18864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0a1577fa0f4741ab0ee73851eb1294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.336048 18861 tablet_service.cc:1511] Processing CreateTablet for tablet 1e957ba42136417fba964b3408dfec86 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:23.337373 18861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e957ba42136417fba964b3408dfec86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.311805 18942 tablet_service.cc:1511] Processing CreateTablet for tablet 4799ea090d564639bc7e3998262adc88 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:23.307494 18867 tablet_service.cc:1511] Processing CreateTablet for tablet 4799ea090d564639bc7e3998262adc88 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:23.342986 18867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4799ea090d564639bc7e3998262adc88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.350208 18942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4799ea090d564639bc7e3998262adc88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.372540 19018 tablet_service.cc:1511] Processing CreateTablet for tablet f1dedc61807b4f1fb0b0df1fd7408660 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:23.374014 19018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1dedc61807b4f1fb0b0df1fd7408660. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.376322 19017 tablet_service.cc:1511] Processing CreateTablet for tablet 5a8849ca00284e83968f5c4d4f7a5f59 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:23.377735 19017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a8849ca00284e83968f5c4d4f7a5f59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.387948 19016 tablet_service.cc:1511] Processing CreateTablet for tablet 4799ea090d564639bc7e3998262adc88 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:23.388676 19015 tablet_service.cc:1511] Processing CreateTablet for tablet 6d0c9c2b6ccc40ee8a9b8265ef14ad8a (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:23.390420 19014 tablet_service.cc:1511] Processing CreateTablet for tablet a0a28ca432c040e090e4540740f46aa6 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:23.392768 19013 tablet_service.cc:1511] Processing CreateTablet for tablet 5d0a1577fa0f4741ab0ee73851eb1294 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:23.395160 19012 tablet_service.cc:1511] Processing CreateTablet for tablet 8b1c41abe4964625829a184924c60d3a (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:23.397439 19011 tablet_service.cc:1511] Processing CreateTablet for tablet cf5def05512948249903a00f64386054 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:23.399798 19010 tablet_service.cc:1511] Processing CreateTablet for tablet 1e957ba42136417fba964b3408dfec86 (DEFAULT_TABLE table=test-workload [id=3d6a04f4c7294845a88e2eef6b561ee9]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:23.415203 19010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e957ba42136417fba964b3408dfec86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.418663 19015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d0c9c2b6ccc40ee8a9b8265ef14ad8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.425994 19016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4799ea090d564639bc7e3998262adc88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.430675 19014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0a28ca432c040e090e4540740f46aa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.434767 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d0a1577fa0f4741ab0ee73851eb1294. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.438745 19012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b1c41abe4964625829a184924c60d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.452899 19011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf5def05512948249903a00f64386054. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.498803 19069 tablet_bootstrap.cc:492] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:23.498803 19068 tablet_bootstrap.cc:492] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:23.509840 19068 tablet_bootstrap.cc:654] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.513854 19069 tablet_bootstrap.cc:654] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.572661 19068 tablet_bootstrap.cc:492] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:23.573109 19068 ts_tablet_manager.cc:1403] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.075s	user 0.009s	sys 0.010s
I20260709 13:17:23.575299 19068 raft_consensus.cc:359] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.577025 19068 raft_consensus.cc:385] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.577378 19068 raft_consensus.cc:740] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.578166 19068 consensus_queue.cc:260] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.578956 19069 tablet_bootstrap.cc:492] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:23.576668 19072 tablet_bootstrap.cc:492] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:23.579533 19069 ts_tablet_manager.cc:1403] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.081s	user 0.016s	sys 0.012s
I20260709 13:17:23.580593 18979 heartbeater.cc:499] Master 127.6.198.126:41745 was elected leader, sending a full tablet report...
I20260709 13:17:23.582247 19069 raft_consensus.cc:359] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.583003 19069 raft_consensus.cc:385] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.583223 19069 raft_consensus.cc:740] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.584013 19069 consensus_queue.cc:260] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.586208 18904 heartbeater.cc:499] Master 127.6.198.126:41745 was elected leader, sending a full tablet report...
I20260709 13:17:23.587986 19068 ts_tablet_manager.cc:1434] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:17:23.589269 19068 tablet_bootstrap.cc:492] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:23.590502 19072 tablet_bootstrap.cc:654] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.595323 19069 ts_tablet_manager.cc:1434] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:17:23.595880 19068 tablet_bootstrap.cc:654] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.596392 19069 tablet_bootstrap.cc:492] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:23.605360 19069 tablet_bootstrap.cc:654] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.612304 19068 tablet_bootstrap.cc:492] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:23.612957 19068 ts_tablet_manager.cc:1403] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.008s
I20260709 13:17:23.614907 19069 tablet_bootstrap.cc:492] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:23.615512 19069 ts_tablet_manager.cc:1403] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:17:23.616386 19068 raft_consensus.cc:359] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.617277 19068 raft_consensus.cc:385] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.617654 19068 raft_consensus.cc:740] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.618709 19068 consensus_queue.cc:260] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.618829 19069 raft_consensus.cc:359] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.619745 19069 raft_consensus.cc:385] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.620065 19069 raft_consensus.cc:740] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.621160 19068 ts_tablet_manager.cc:1434] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:17:23.620934 19069 consensus_queue.cc:260] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.622006 19072 tablet_bootstrap.cc:492] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:23.622495 19068 tablet_bootstrap.cc:492] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:23.622578 19072 ts_tablet_manager.cc:1403] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.027s
I20260709 13:17:23.623113 19069 ts_tablet_manager.cc:1434] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.006s
I20260709 13:17:23.624143 19069 tablet_bootstrap.cc:492] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:23.626040 19072 raft_consensus.cc:359] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.626930 19072 raft_consensus.cc:385] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.627319 19072 raft_consensus.cc:740] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.628252 19072 consensus_queue.cc:260] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.630592 19068 tablet_bootstrap.cc:654] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.630970 19069 tablet_bootstrap.cc:654] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.634764 19074 raft_consensus.cc:493] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.635355 19074 raft_consensus.cc:515] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.638288 19072 ts_tablet_manager.cc:1434] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.007s
I20260709 13:17:23.640118 19053 heartbeater.cc:499] Master 127.6.198.126:41745 was elected leader, sending a full tablet report...
I20260709 13:17:23.640697 19072 tablet_bootstrap.cc:492] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:23.640897 19069 tablet_bootstrap.cc:492] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:23.641805 19069 ts_tablet_manager.cc:1403] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.005s
I20260709 13:17:23.643175 19074 leader_election.cc:290] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:23.645895 19069 raft_consensus.cc:359] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.647984 19069 raft_consensus.cc:385] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.648005 19072 tablet_bootstrap.cc:654] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.648418 19069 raft_consensus.cc:740] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.649006 19068 tablet_bootstrap.cc:492] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:23.649537 19068 ts_tablet_manager.cc:1403] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.027s	user 0.006s	sys 0.019s
I20260709 13:17:23.649237 19069 consensus_queue.cc:260] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.651525 19069 ts_tablet_manager.cc:1434] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260709 13:17:23.652523 19069 tablet_bootstrap.cc:492] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:23.652693 19068 raft_consensus.cc:359] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.653524 19068 raft_consensus.cc:385] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.653851 19068 raft_consensus.cc:740] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.654829 19068 consensus_queue.cc:260] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.657229 19068 ts_tablet_manager.cc:1434] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:17:23.658308 19068 tablet_bootstrap.cc:492] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:23.659966 19069 tablet_bootstrap.cc:654] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.665980 19068 tablet_bootstrap.cc:654] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.668383 19073 raft_consensus.cc:493] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.669299 19073 raft_consensus.cc:515] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.687896 19073 leader_election.cc:290] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:23.688992 19068 tablet_bootstrap.cc:492] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:23.689904 19068 ts_tablet_manager.cc:1403] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.032s	user 0.007s	sys 0.012s
I20260709 13:17:23.694176 19068 raft_consensus.cc:359] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.695254 19068 raft_consensus.cc:385] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.695662 19068 raft_consensus.cc:740] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.704166 19072 tablet_bootstrap.cc:492] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:23.705111 19072 ts_tablet_manager.cc:1403] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.065s	user 0.016s	sys 0.003s
I20260709 13:17:23.708238 19072 raft_consensus.cc:359] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.710666 19068 consensus_queue.cc:260] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.715416 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8849ca00284e83968f5c4d4f7a5f59" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:23.718616 19072 raft_consensus.cc:385] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.718986 19072 raft_consensus.cc:740] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
W20260709 13:17:23.719972 18842 leader_election.cc:343] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.720242 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8849ca00284e83968f5c4d4f7a5f59" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:23.720857 19069 tablet_bootstrap.cc:492] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:23.721464 19069 ts_tablet_manager.cc:1403] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.069s	user 0.031s	sys 0.032s
I20260709 13:17:23.720978 19072 consensus_queue.cc:260] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
W20260709 13:17:23.722249 18845 leader_election.cc:343] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.722824 19068 ts_tablet_manager.cc:1434] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.032s	user 0.011s	sys 0.002s
I20260709 13:17:23.722894 18845 leader_election.cc:304] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1c86056b295463abc9760e7158f4bce; no voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893
I20260709 13:17:23.724126 19068 tablet_bootstrap.cc:492] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:23.724293 19074 raft_consensus.cc:2749] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:23.724660 19069 raft_consensus.cc:359] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.725415 19069 raft_consensus.cc:385] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.725663 19069 raft_consensus.cc:740] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.726325 19069 consensus_queue.cc:260] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.728384 19069 ts_tablet_manager.cc:1434] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:17:23.729430 19069 tablet_bootstrap.cc:492] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:23.729872 19073 raft_consensus.cc:493] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.730434 19073 raft_consensus.cc:515] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.733053 19073 leader_election.cc:290] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:23.733861 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1c41abe4964625829a184924c60d3a" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:23.735262 19069 tablet_bootstrap.cc:654] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.736459 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4799ea090d564639bc7e3998262adc88" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
W20260709 13:17:23.738154 18917 leader_election.cc:343] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:23.738925 18917 leader_election.cc:343] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.742465 19068 tablet_bootstrap.cc:654] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.743376 19072 ts_tablet_manager.cc:1434] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.001s
I20260709 13:17:23.744783 19072 tablet_bootstrap.cc:492] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:23.750980 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1c41abe4964625829a184924c60d3a" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:23.753270 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4799ea090d564639bc7e3998262adc88" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:23.754745 19068 tablet_bootstrap.cc:492] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
W20260709 13:17:23.755091 18920 leader_election.cc:343] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.755292 19068 ts_tablet_manager.cc:1403] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.031s	user 0.006s	sys 0.012s
I20260709 13:17:23.755605 18920 leader_election.cc:304] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b37c12ab1f654afc812877fd13698893; no voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce
I20260709 13:17:23.753526 19072 tablet_bootstrap.cc:654] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.756748 19073 raft_consensus.cc:2749] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:23.758036 18920 leader_election.cc:343] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.758565 18920 leader_election.cc:304] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b37c12ab1f654afc812877fd13698893; no voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce
I20260709 13:17:23.759392 19069 tablet_bootstrap.cc:492] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:23.758669 19068 raft_consensus.cc:359] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.759831 19068 raft_consensus.cc:385] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.759893 19069 ts_tablet_manager.cc:1403] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.008s
I20260709 13:17:23.759944 19073 raft_consensus.cc:2749] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:23.760337 19068 raft_consensus.cc:740] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.761502 19068 consensus_queue.cc:260] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.764020 19068 ts_tablet_manager.cc:1434] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:23.763734 19069 raft_consensus.cc:359] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.764555 19069 raft_consensus.cc:385] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.764891 19069 raft_consensus.cc:740] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.765965 19069 consensus_queue.cc:260] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.766532 19074 raft_consensus.cc:493] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.767268 19074 raft_consensus.cc:515] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.774803 19068 tablet_bootstrap.cc:492] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:23.770020 19074 leader_election.cc:290] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:23.781467 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d0c9c2b6ccc40ee8a9b8265ef14ad8a" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:23.782100 18954 raft_consensus.cc:2468] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 0.
I20260709 13:17:23.786190 19072 tablet_bootstrap.cc:492] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:23.785580 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d0c9c2b6ccc40ee8a9b8265ef14ad8a" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
W20260709 13:17:23.786773 18825 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:23.787281 18842 leader_election.cc:304] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:23.788094 19068 tablet_bootstrap.cc:654] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.789148 19069 ts_tablet_manager.cc:1434] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.028s	user 0.017s	sys 0.008s
I20260709 13:17:23.790373 19069 tablet_bootstrap.cc:492] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
W20260709 13:17:23.792100 18845 leader_election.cc:343] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.792332 19092 raft_consensus.cc:2804] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:23.792801 19092 raft_consensus.cc:493] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.793154 19092 raft_consensus.cc:3060] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.797324 19074 raft_consensus.cc:493] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.797838 19074 raft_consensus.cc:515] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.786787 19072 ts_tablet_manager.cc:1403] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.024s
I20260709 13:17:23.800374 19074 leader_election.cc:290] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:23.801487 19092 raft_consensus.cc:515] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.801977 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1c41abe4964625829a184924c60d3a" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:23.802419 19072 raft_consensus.cc:359] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.803287 19072 raft_consensus.cc:385] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:17:23.803404 18845 leader_election.cc:343] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.803632 19072 raft_consensus.cc:740] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.804427 19092 leader_election.cc:290] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 election: Requested vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:23.804595 19072 consensus_queue.cc:260] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.806787 19068 tablet_bootstrap.cc:492] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:23.807309 19068 ts_tablet_manager.cc:1403] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.010s
I20260709 13:17:23.807268 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1c41abe4964625829a184924c60d3a" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:23.807977 19072 ts_tablet_manager.cc:1434] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:17:23.808104 18954 raft_consensus.cc:2468] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 0.
I20260709 13:17:23.809062 19072 tablet_bootstrap.cc:492] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:23.809839 18842 leader_election.cc:304] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 4bada1aa2db9455e874b4139f9ad1a40
I20260709 13:17:23.810689 19074 raft_consensus.cc:493] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.810607 19068 raft_consensus.cc:359] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.811470 19068 raft_consensus.cc:385] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.811434 19074 raft_consensus.cc:515] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.812002 19068 raft_consensus.cc:740] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.813004 19069 tablet_bootstrap.cc:654] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.813033 19068 consensus_queue.cc:260] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.814031 19074 leader_election.cc:290] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:23.815614 19068 ts_tablet_manager.cc:1434] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:23.815871 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5def05512948249903a00f64386054" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:23.816754 19074 raft_consensus.cc:2804] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:23.817941 19068 tablet_bootstrap.cc:492] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:23.819123 19072 tablet_bootstrap.cc:654] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.820439 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5def05512948249903a00f64386054" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
W20260709 13:17:23.822811 18842 leader_election.cc:343] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.817541 19074 raft_consensus.cc:493] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.823745 19074 raft_consensus.cc:3060] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.826042 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d0c9c2b6ccc40ee8a9b8265ef14ad8a" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:23.826689 18954 raft_consensus.cc:3060] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.826705 19068 tablet_bootstrap.cc:654] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.832408 19074 raft_consensus.cc:515] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
W20260709 13:17:23.834787 18845 leader_election.cc:343] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.837625 18845 leader_election.cc:304] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1c86056b295463abc9760e7158f4bce; no voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893
I20260709 13:17:23.838409 19092 raft_consensus.cc:2749] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:23.835471 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d0c9c2b6ccc40ee8a9b8265ef14ad8a" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:23.838554 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1c41abe4964625829a184924c60d3a" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:23.839463 19028 raft_consensus.cc:3060] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.839627 18953 raft_consensus.cc:3060] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.840523 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1c41abe4964625829a184924c60d3a" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
W20260709 13:17:23.841984 18845 leader_election.cc:343] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.835680 19069 tablet_bootstrap.cc:492] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:23.846866 19069 ts_tablet_manager.cc:1403] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.057s	user 0.017s	sys 0.004s
I20260709 13:17:23.846930 19028 raft_consensus.cc:2468] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 1.
I20260709 13:17:23.848546 18845 leader_election.cc:304] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:23.849339 19092 raft_consensus.cc:2804] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:23.851154 19069 raft_consensus.cc:359] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.851907 19069 raft_consensus.cc:385] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.852223 19069 raft_consensus.cc:740] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.852514 19072 tablet_bootstrap.cc:492] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:23.853392 19072 ts_tablet_manager.cc:1403] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.016s
I20260709 13:17:23.853382 18953 raft_consensus.cc:2468] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 1.
I20260709 13:17:23.854935 18842 leader_election.cc:304] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 4bada1aa2db9455e874b4139f9ad1a40
I20260709 13:17:23.857067 19072 raft_consensus.cc:359] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.858192 19072 raft_consensus.cc:385] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.858780 19072 raft_consensus.cc:740] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.860323 19092 raft_consensus.cc:697] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Becoming Leader. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:23.859941 19072 consensus_queue.cc:260] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.860827 19097 raft_consensus.cc:2804] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:23.858594 18954 raft_consensus.cc:2468] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 1.
I20260709 13:17:23.861439 19097 raft_consensus.cc:697] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Becoming Leader. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:23.862663 19072 ts_tablet_manager.cc:1434] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:17:23.853438 19069 consensus_queue.cc:260] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.861438 19092 consensus_queue.cc:237] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.863749 19072 tablet_bootstrap.cc:492] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:23.865919 19068 tablet_bootstrap.cc:492] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:23.866462 19068 ts_tablet_manager.cc:1403] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.049s	user 0.021s	sys 0.000s
I20260709 13:17:23.870081 19068 raft_consensus.cc:359] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.870898 19068 raft_consensus.cc:385] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.871286 19068 raft_consensus.cc:740] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.871443 19072 tablet_bootstrap.cc:654] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.872292 19068 consensus_queue.cc:260] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.838680 19074 leader_election.cc:290] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 election: Requested vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:23.876722 19068 ts_tablet_manager.cc:1434] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:17:23.878224 19068 tablet_bootstrap.cc:492] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:23.893610 19068 tablet_bootstrap.cc:654] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.900558 19072 tablet_bootstrap.cc:492] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:23.901098 19072 ts_tablet_manager.cc:1403] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.024s
I20260709 13:17:23.901458 19068 tablet_bootstrap.cc:492] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:23.902046 19068 ts_tablet_manager.cc:1403] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.000s
I20260709 13:17:23.862819 19097 consensus_queue.cc:237] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.904562 19072 raft_consensus.cc:359] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.906005 19072 raft_consensus.cc:385] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.906387 19072 raft_consensus.cc:740] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.907226 19072 consensus_queue.cc:260] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.905176 19068 raft_consensus.cc:359] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.910574 19068 raft_consensus.cc:385] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.910959 19068 raft_consensus.cc:740] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.911774 19068 consensus_queue.cc:260] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.913177 19069 ts_tablet_manager.cc:1434] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.066s	user 0.021s	sys 0.035s
I20260709 13:17:23.910082 19072 ts_tablet_manager.cc:1434] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:17:23.914546 19069 tablet_bootstrap.cc:492] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:23.914669 19068 ts_tablet_manager.cc:1434] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:17:23.915347 19072 tablet_bootstrap.cc:492] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:23.915738 19068 tablet_bootstrap.cc:492] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:23.922251 19069 tablet_bootstrap.cc:654] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.922729 19068 tablet_bootstrap.cc:654] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.926676 19072 tablet_bootstrap.cc:654] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.946370 19078 raft_consensus.cc:493] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.947546 19078 raft_consensus.cc:515] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.957602 19078 leader_election.cc:290] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:23.978765 19073 raft_consensus.cc:493] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.979648 19073 raft_consensus.cc:515] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.987713 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e957ba42136417fba964b3408dfec86" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
W20260709 13:17:23.989748 18917 leader_election.cc:343] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.991669 19073 leader_election.cc:290] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:23.992399 19073 raft_consensus.cc:493] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.993362 19073 raft_consensus.cc:515] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:23.997361 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5def05512948249903a00f64386054" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:23.998168 18879 raft_consensus.cc:2468] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 0.
I20260709 13:17:23.999830 18917 leader_election.cc:304] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:24.001021 19108 raft_consensus.cc:2804] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.001518 19108 raft_consensus.cc:493] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.001900 19108 raft_consensus.cc:3060] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.002559 19073 leader_election.cc:290] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:24.011551 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e957ba42136417fba964b3408dfec86" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:24.011852 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5def05512948249903a00f64386054" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:24.012274 19028 raft_consensus.cc:2468] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 0.
W20260709 13:17:24.013187 18920 leader_election.cc:343] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.014658 19108 raft_consensus.cc:515] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.015014 18920 leader_election.cc:304] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893; no voters: d1c86056b295463abc9760e7158f4bce
I20260709 13:17:24.016897 19073 raft_consensus.cc:2804] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.017436 19073 raft_consensus.cc:493] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.017937 19073 raft_consensus.cc:3060] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.018004 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5def05512948249903a00f64386054" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:24.018836 18879 raft_consensus.cc:3060] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.027257 18879 raft_consensus.cc:2468] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 1.
I20260709 13:17:24.028740 18917 leader_election.cc:304] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:24.033493 18774 catalog_manager.cc:5697] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce reported cstate change: term changed from 0 to 1, leader changed from <none> to d1c86056b295463abc9760e7158f4bce (127.6.198.65). New cstate: current_term: 1 leader_uuid: "d1c86056b295463abc9760e7158f4bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.043331 19111 raft_consensus.cc:493] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.044093 19111 raft_consensus.cc:515] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.047222 19108 leader_election.cc:290] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:24.047415 19078 raft_consensus.cc:493] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.048022 19108 raft_consensus.cc:2804] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.047951 19078 raft_consensus.cc:515] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.048653 19108 raft_consensus.cc:697] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Becoming Leader. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:24.050006 19078 leader_election.cc:290] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:24.051668 19072 tablet_bootstrap.cc:492] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:24.052105 19072 ts_tablet_manager.cc:1403] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.137s	user 0.018s	sys 0.046s
I20260709 13:17:24.054538 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0a28ca432c040e090e4540740f46aa6" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:24.054854 19072 raft_consensus.cc:359] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.055248 18878 raft_consensus.cc:2468] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 0.
I20260709 13:17:24.055617 19072 raft_consensus.cc:385] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.055936 19072 raft_consensus.cc:740] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.056636 19072 consensus_queue.cc:260] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.058843 19072 ts_tablet_manager.cc:1434] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.002s
I20260709 13:17:24.059604 18991 leader_election.cc:304] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:24.061127 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e957ba42136417fba964b3408dfec86" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:24.062665 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8849ca00284e83968f5c4d4f7a5f59" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:24.063267 18878 raft_consensus.cc:2468] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 0.
I20260709 13:17:24.064265 18917 leader_election.cc:304] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:24.064519 19078 raft_consensus.cc:2804] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.064953 19078 raft_consensus.cc:493] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.055068 19111 leader_election.cc:290] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:24.065424 19078 raft_consensus.cc:3060] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.066035 19111 raft_consensus.cc:2804] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:17:24.066860 18991 leader_election.cc:343] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.059847 19072 tablet_bootstrap.cc:492] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:24.049744 19108 consensus_queue.cc:237] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.066473 19111 raft_consensus.cc:493] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.072664 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5def05512948249903a00f64386054" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:24.073252 19028 raft_consensus.cc:3060] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.073460 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8849ca00284e83968f5c4d4f7a5f59" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:24.076756 19078 raft_consensus.cc:515] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.075294 19111 raft_consensus.cc:3060] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.064875 19073 raft_consensus.cc:515] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.081243 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0a28ca432c040e090e4540740f46aa6" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:24.081853 18878 raft_consensus.cc:3060] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.089413 18878 raft_consensus.cc:2468] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 1.
W20260709 13:17:24.090005 18920 leader_election.cc:343] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.090492 18991 leader_election.cc:304] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:24.090642 19028 raft_consensus.cc:2468] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 1.
I20260709 13:17:24.092204 19112 raft_consensus.cc:2804] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.093881 19068 tablet_bootstrap.cc:492] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:24.094614 19068 ts_tablet_manager.cc:1403] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.179s	user 0.052s	sys 0.045s
I20260709 13:17:24.096163 19069 tablet_bootstrap.cc:492] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:24.093489 19111 raft_consensus.cc:515] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.096177 19073 leader_election.cc:290] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:24.098918 19069 ts_tablet_manager.cc:1403] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.185s	user 0.058s	sys 0.091s
I20260709 13:17:24.098682 19112 raft_consensus.cc:697] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Becoming Leader. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Running, Role: LEADER
I20260709 13:17:24.101192 19112 consensus_queue.cc:237] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.101727 19068 raft_consensus.cc:359] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.101946 19069 raft_consensus.cc:359] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.102947 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e957ba42136417fba964b3408dfec86" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:24.103595 19028 raft_consensus.cc:3060] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.102985 19069 raft_consensus.cc:385] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.104344 19069 raft_consensus.cc:740] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.104821 19111 leader_election.cc:290] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:24.105506 19069 consensus_queue.cc:260] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.106297 18776 catalog_manager.cc:5697] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 reported cstate change: term changed from 0 to 1, leader changed from <none> to b37c12ab1f654afc812877fd13698893 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "b37c12ab1f654afc812877fd13698893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.103066 19068 raft_consensus.cc:385] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.109108 19068 raft_consensus.cc:740] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.106664 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8849ca00284e83968f5c4d4f7a5f59" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:24.108675 19069 ts_tablet_manager.cc:1434] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:17:24.110754 19068 consensus_queue.cc:260] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.111253 19078 leader_election.cc:290] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:24.111810 19028 raft_consensus.cc:2468] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 1.
I20260709 13:17:24.112952 19069 tablet_bootstrap.cc:492] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:24.114404 19072 tablet_bootstrap.cc:654] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.114264 19068 ts_tablet_manager.cc:1434] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:17:24.116134 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e957ba42136417fba964b3408dfec86" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:24.117602 19092 raft_consensus.cc:493] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.118069 19092 raft_consensus.cc:515] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.119637 19069 tablet_bootstrap.cc:654] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:24.112349 18920 leader_election.cc:343] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.121344 18920 leader_election.cc:304] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893; no voters: 
I20260709 13:17:24.121757 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0a1577fa0f4741ab0ee73851eb1294" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:24.121832 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0a1577fa0f4741ab0ee73851eb1294" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:24.122511 18953 raft_consensus.cc:2468] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 0.
I20260709 13:17:24.123333 19111 raft_consensus.cc:2804] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.123771 18842 leader_election.cc:304] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:24.124045 19111 raft_consensus.cc:697] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Becoming Leader. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:24.124646 19097 raft_consensus.cc:2804] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.120318 19092 leader_election.cc:290] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:24.125048 19097 raft_consensus.cc:493] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.125007 19111 consensus_queue.cc:237] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.125730 19097 raft_consensus.cc:3060] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:24.127127 18845 leader_election.cc:343] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.128391 19113 raft_consensus.cc:493] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.128885 19113 raft_consensus.cc:515] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.129657 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0a28ca432c040e090e4540740f46aa6" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:24.130306 18954 raft_consensus.cc:2468] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 0.
I20260709 13:17:24.130684 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e957ba42136417fba964b3408dfec86" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:24.130965 19113 leader_election.cc:290] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:24.130066 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8849ca00284e83968f5c4d4f7a5f59" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:24.131980 18878 raft_consensus.cc:3060] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.132030 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0a28ca432c040e090e4540740f46aa6" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:24.132642 18954 raft_consensus.cc:3060] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.133173 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1dedc61807b4f1fb0b0df1fd7408660" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:24.134006 19028 raft_consensus.cc:2468] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 0.
I20260709 13:17:24.135689 18920 leader_election.cc:304] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893; no voters: 
I20260709 13:17:24.136696 19113 raft_consensus.cc:2804] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.137063 19113 raft_consensus.cc:493] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.137450 19113 raft_consensus.cc:3060] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.139542 18954 raft_consensus.cc:2468] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 1.
W20260709 13:17:24.140558 18917 leader_election.cc:343] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.144371 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1dedc61807b4f1fb0b0df1fd7408660" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:24.145116 18879 raft_consensus.cc:2468] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 0.
I20260709 13:17:24.145557 18991 leader_election.cc:304] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40; no voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce
I20260709 13:17:24.146375 19078 raft_consensus.cc:2749] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.149721 19097 raft_consensus.cc:515] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.152832 19097 leader_election.cc:290] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 election: Requested vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:24.154036 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0a1577fa0f4741ab0ee73851eb1294" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
W20260709 13:17:24.155469 18845 leader_election.cc:343] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.150367 19072 tablet_bootstrap.cc:492] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:24.157603 19072 ts_tablet_manager.cc:1403] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.098s	user 0.014s	sys 0.007s
I20260709 13:17:24.159529 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0a1577fa0f4741ab0ee73851eb1294" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:24.160346 18954 raft_consensus.cc:3060] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.151350 18878 raft_consensus.cc:2468] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 1.
I20260709 13:17:24.163501 19069 tablet_bootstrap.cc:492] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:24.164013 19069 ts_tablet_manager.cc:1403] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.002s
I20260709 13:17:24.163600 18917 leader_election.cc:304] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 4bada1aa2db9455e874b4139f9ad1a40
I20260709 13:17:24.164716 19073 raft_consensus.cc:2804] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.165179 19073 raft_consensus.cc:697] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Becoming Leader. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:24.165967 19073 consensus_queue.cc:237] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.166647 19069 raft_consensus.cc:359] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.167376 19069 raft_consensus.cc:385] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.167656 19069 raft_consensus.cc:740] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.168612 18954 raft_consensus.cc:2468] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 1.
I20260709 13:17:24.170394 18842 leader_election.cc:304] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 4bada1aa2db9455e874b4139f9ad1a40
I20260709 13:17:24.166095 19113 raft_consensus.cc:515] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.175805 19097 raft_consensus.cc:2804] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.161482 19072 raft_consensus.cc:359] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.174827 19078 raft_consensus.cc:493] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.176362 19072 raft_consensus.cc:385] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.176600 19097 raft_consensus.cc:697] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Becoming Leader. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:24.176573 19078 raft_consensus.cc:515] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.178694 19097 consensus_queue.cc:237] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.179979 19113 leader_election.cc:290] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:24.180195 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1dedc61807b4f1fb0b0df1fd7408660" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:24.180855 19028 raft_consensus.cc:3060] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.172228 19069 consensus_queue.cc:260] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.184259 19069 ts_tablet_manager.cc:1434] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.003s
I20260709 13:17:24.185637 19069 tablet_bootstrap.cc:492] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:24.192467 19069 tablet_bootstrap.cc:654] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.193431 19078 leader_election.cc:290] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:24.176710 19072 raft_consensus.cc:740] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.195024 19072 consensus_queue.cc:260] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.195394 18777 catalog_manager.cc:5697] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce reported cstate change: term changed from 0 to 1, leader changed from <none> to d1c86056b295463abc9760e7158f4bce (127.6.198.65). New cstate: current_term: 1 leader_uuid: "d1c86056b295463abc9760e7158f4bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.197868 19072 ts_tablet_manager.cc:1434] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 13:17:24.201392 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1c41abe4964625829a184924c60d3a" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:24.201421 19072 tablet_bootstrap.cc:492] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:24.207103 19028 raft_consensus.cc:2468] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 1.
I20260709 13:17:24.210465 18920 leader_election.cc:304] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893; no voters: 
I20260709 13:17:24.211299 19113 raft_consensus.cc:2804] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.211767 19113 raft_consensus.cc:697] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Becoming Leader. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:24.212280 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1dedc61807b4f1fb0b0df1fd7408660" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:24.213099 18878 raft_consensus.cc:3060] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.212601 19113 consensus_queue.cc:237] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.218801 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1c41abe4964625829a184924c60d3a" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:24.219974 18954 raft_consensus.cc:2393] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4bada1aa2db9455e874b4139f9ad1a40 in current term 1: Already voted for candidate d1c86056b295463abc9760e7158f4bce in this term.
I20260709 13:17:24.222100 19069 tablet_bootstrap.cc:492] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:24.222708 19069 ts_tablet_manager.cc:1403] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.011s
I20260709 13:17:24.223250 18991 leader_election.cc:304] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40; no voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce
I20260709 13:17:24.224699 18878 raft_consensus.cc:2468] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 1.
I20260709 13:17:24.225504 19069 raft_consensus.cc:359] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.226363 19069 raft_consensus.cc:385] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.226663 19069 raft_consensus.cc:740] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.226840 19112 raft_consensus.cc:3060] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.227632 19069 consensus_queue.cc:260] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.229979 19069 ts_tablet_manager.cc:1434] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:17:24.231632 19072 tablet_bootstrap.cc:654] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.235780 19072 tablet_bootstrap.cc:492] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:24.236176 19072 ts_tablet_manager.cc:1403] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.000s
I20260709 13:17:24.239205 19072 raft_consensus.cc:359] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.239861 19072 raft_consensus.cc:385] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.240089 19072 raft_consensus.cc:740] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.240033 18776 catalog_manager.cc:5697] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "4bada1aa2db9455e874b4139f9ad1a40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:24.241063 19112 raft_consensus.cc:2749] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.240965 19072 consensus_queue.cc:260] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.242841 19072 ts_tablet_manager.cc:1434] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:17:24.243636 19072 tablet_bootstrap.cc:492] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:24.247844 19072 tablet_bootstrap.cc:654] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.249629 18775 catalog_manager.cc:5697] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 reported cstate change: term changed from 0 to 1, leader changed from <none> to b37c12ab1f654afc812877fd13698893 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "b37c12ab1f654afc812877fd13698893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.252566 19072 tablet_bootstrap.cc:492] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:24.252996 19072 ts_tablet_manager.cc:1403] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:17:24.254861 19072 raft_consensus.cc:359] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.255487 19072 raft_consensus.cc:385] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.255738 19072 raft_consensus.cc:740] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.256302 19072 consensus_queue.cc:260] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.258435 19072 ts_tablet_manager.cc:1434] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:17:24.260875 18777 catalog_manager.cc:5697] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce reported cstate change: term changed from 0 to 1, leader changed from <none> to d1c86056b295463abc9760e7158f4bce (127.6.198.65). New cstate: current_term: 1 leader_uuid: "d1c86056b295463abc9760e7158f4bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.265458 18776 catalog_manager.cc:5697] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 reported cstate change: term changed from 0 to 1, leader changed from <none> to b37c12ab1f654afc812877fd13698893 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "b37c12ab1f654afc812877fd13698893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.266747 18776 catalog_manager.cc:5697] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 reported cstate change: term changed from 0 to 1, leader changed from <none> to b37c12ab1f654afc812877fd13698893 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "b37c12ab1f654afc812877fd13698893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.333009 19097 raft_consensus.cc:493] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.333588 19097 raft_consensus.cc:515] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.335436 19097 leader_election.cc:290] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:24.336256 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e957ba42136417fba964b3408dfec86" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:24.336571 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e957ba42136417fba964b3408dfec86" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:24.337304 19028 raft_consensus.cc:2393] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1c86056b295463abc9760e7158f4bce in current term 1: Already voted for candidate b37c12ab1f654afc812877fd13698893 in this term.
I20260709 13:17:24.338196 18845 leader_election.cc:304] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1c86056b295463abc9760e7158f4bce; no voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893
I20260709 13:17:24.338965 19097 raft_consensus.cc:3060] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.343726 19097 raft_consensus.cc:2749] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.418583 19097 consensus_queue.cc:1048] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.433358 19092 consensus_queue.cc:1048] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.451267 19097 consensus_queue.cc:1048] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.462235 19097 consensus_queue.cc:1048] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.476296 19112 raft_consensus.cc:493] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.476816 19112 raft_consensus.cc:515] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.479022 19112 leader_election.cc:290] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:24.479809 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0a1577fa0f4741ab0ee73851eb1294" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:24.479924 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0a1577fa0f4741ab0ee73851eb1294" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:24.480616 18954 raft_consensus.cc:2393] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4bada1aa2db9455e874b4139f9ad1a40 in current term 1: Already voted for candidate d1c86056b295463abc9760e7158f4bce in this term.
I20260709 13:17:24.481590 18991 leader_election.cc:304] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40; no voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce
I20260709 13:17:24.482295 19112 raft_consensus.cc:3060] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.487668 19112 raft_consensus.cc:2749] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.516330 19112 consensus_queue.cc:1048] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.527271 19078 consensus_queue.cc:1048] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.539299 19113 consensus_queue.cc:1048] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.551414 19113 consensus_queue.cc:1048] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.551788 19078 raft_consensus.cc:493] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.552407 19078 raft_consensus.cc:515] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.554492 19078 leader_election.cc:290] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:24.555159 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8849ca00284e83968f5c4d4f7a5f59" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:24.555969 18878 raft_consensus.cc:2393] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4bada1aa2db9455e874b4139f9ad1a40 in current term 1: Already voted for candidate b37c12ab1f654afc812877fd13698893 in this term.
I20260709 13:17:24.558310 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8849ca00284e83968f5c4d4f7a5f59" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:24.560101 18991 leader_election.cc:304] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40; no voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce
I20260709 13:17:24.560949 19078 raft_consensus.cc:3060] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.564942 19111 consensus_queue.cc:1048] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.567867 19078 raft_consensus.cc:2749] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.575029 19108 consensus_queue.cc:1048] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.578363 19073 consensus_queue.cc:1048] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.592672 19073 consensus_queue.cc:1048] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.654822 19073 consensus_queue.cc:1048] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.665310 19111 consensus_queue.cc:1048] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.668732 19078 raft_consensus.cc:493] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.669236 19078 raft_consensus.cc:515] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.671751 19078 leader_election.cc:290] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:24.672946 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4799ea090d564639bc7e3998262adc88" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:24.673332 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4799ea090d564639bc7e3998262adc88" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:24.673689 18878 raft_consensus.cc:2468] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 0.
I20260709 13:17:24.674046 18954 raft_consensus.cc:2468] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 0.
I20260709 13:17:24.675040 18991 leader_election.cc:304] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:24.675829 19078 raft_consensus.cc:2804] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.676214 19078 raft_consensus.cc:493] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.676548 19078 raft_consensus.cc:3060] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.683027 19078 raft_consensus.cc:515] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.684887 19078 leader_election.cc:290] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:24.685540 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4799ea090d564639bc7e3998262adc88" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:24.685865 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4799ea090d564639bc7e3998262adc88" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:24.686124 18878 raft_consensus.cc:3060] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.686573 18954 raft_consensus.cc:3060] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.691277 18878 raft_consensus.cc:2468] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 1.
I20260709 13:17:24.691274 18954 raft_consensus.cc:2468] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 1.
I20260709 13:17:24.692438 18991 leader_election.cc:304] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:24.693109 19078 raft_consensus.cc:2804] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.693543 19078 raft_consensus.cc:697] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Becoming Leader. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Running, Role: LEADER
I20260709 13:17:24.694258 19078 consensus_queue.cc:237] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.700692 18777 catalog_manager.cc:5697] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "4bada1aa2db9455e874b4139f9ad1a40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:24.710399 19097 consensus_queue.cc:1048] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.720512 19133 consensus_queue.cc:1048] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.872689 18954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1dedc61807b4f1fb0b0df1fd7408660"
dest_uuid: "b37c12ab1f654afc812877fd13698893"
mode: GRACEFUL
new_leader_uuid: "d1c86056b295463abc9760e7158f4bce"
 from {username='slave'} at 127.0.0.1:54088
I20260709 13:17:24.873282 18954 raft_consensus.cc:606] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Received request to transfer leadership to TS d1c86056b295463abc9760e7158f4bce
I20260709 13:17:24.878139 18954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5a8849ca00284e83968f5c4d4f7a5f59"
dest_uuid: "b37c12ab1f654afc812877fd13698893"
mode: GRACEFUL
new_leader_uuid: "d1c86056b295463abc9760e7158f4bce"
 from {username='slave'} at 127.0.0.1:54088
I20260709 13:17:24.878629 18954 raft_consensus.cc:606] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Received request to transfer leadership to TS d1c86056b295463abc9760e7158f4bce
I20260709 13:17:24.891919 19028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4799ea090d564639bc7e3998262adc88"
dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
mode: GRACEFUL
new_leader_uuid: "d1c86056b295463abc9760e7158f4bce"
 from {username='slave'} at 127.0.0.1:60712
I20260709 13:17:24.892442 19028 raft_consensus.cc:606] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Received request to transfer leadership to TS d1c86056b295463abc9760e7158f4bce
I20260709 13:17:24.899783 19028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0a28ca432c040e090e4540740f46aa6"
dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
mode: GRACEFUL
new_leader_uuid: "b37c12ab1f654afc812877fd13698893"
 from {username='slave'} at 127.0.0.1:60712
I20260709 13:17:24.900270 19028 raft_consensus.cc:606] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Received request to transfer leadership to TS b37c12ab1f654afc812877fd13698893
I20260709 13:17:24.912185 18878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d0a1577fa0f4741ab0ee73851eb1294"
dest_uuid: "d1c86056b295463abc9760e7158f4bce"
mode: GRACEFUL
new_leader_uuid: "b37c12ab1f654afc812877fd13698893"
 from {username='slave'} at 127.0.0.1:46190
I20260709 13:17:24.912674 18878 raft_consensus.cc:606] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Received request to transfer leadership to TS b37c12ab1f654afc812877fd13698893
I20260709 13:17:24.917447 18878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b1c41abe4964625829a184924c60d3a"
dest_uuid: "d1c86056b295463abc9760e7158f4bce"
mode: GRACEFUL
new_leader_uuid: "b37c12ab1f654afc812877fd13698893"
 from {username='slave'} at 127.0.0.1:46190
I20260709 13:17:24.917897 18878 raft_consensus.cc:606] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Received request to transfer leadership to TS b37c12ab1f654afc812877fd13698893
I20260709 13:17:24.925058 18954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e957ba42136417fba964b3408dfec86"
dest_uuid: "b37c12ab1f654afc812877fd13698893"
mode: GRACEFUL
new_leader_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
 from {username='slave'} at 127.0.0.1:54088
I20260709 13:17:24.925554 18954 raft_consensus.cc:606] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Received request to transfer leadership to TS 4bada1aa2db9455e874b4139f9ad1a40
I20260709 13:17:24.927207 19116 raft_consensus.cc:993] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40: : Instructing follower b37c12ab1f654afc812877fd13698893 to start an election
I20260709 13:17:24.927637 19078 raft_consensus.cc:1081] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Signalling peer b37c12ab1f654afc812877fd13698893 to start an election
I20260709 13:17:24.928990 18953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0a28ca432c040e090e4540740f46aa6"
dest_uuid: "b37c12ab1f654afc812877fd13698893"
 from {username='slave'} at 127.0.0.1:54078
I20260709 13:17:24.929471 18953 raft_consensus.cc:493] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:24.929764 18953 raft_consensus.cc:3060] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.935928 18953 raft_consensus.cc:515] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.937618 18953 leader_election.cc:290] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 2 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:24.938299 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0a28ca432c040e090e4540740f46aa6" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:24.938657 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0a28ca432c040e090e4540740f46aa6" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:24.938984 18878 raft_consensus.cc:3060] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.939226 19028 raft_consensus.cc:3055] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:24.939587 19028 raft_consensus.cc:740] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Running, Role: LEADER
I20260709 13:17:24.940224 19028 consensus_queue.cc:260] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.941154 19028 raft_consensus.cc:3060] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.944216 18878 raft_consensus.cc:2468] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 2.
I20260709 13:17:24.945317 18917 leader_election.cc:304] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:24.945871 19028 raft_consensus.cc:2468] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 2.
I20260709 13:17:24.945989 19111 raft_consensus.cc:2804] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:24.946624 19111 raft_consensus.cc:697] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [term 2 LEADER]: Becoming Leader. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:24.947365 19111 consensus_queue.cc:237] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:24.953271 18775 catalog_manager.cc:5697] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 reported cstate change: term changed from 1 to 2, leader changed from 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67) to b37c12ab1f654afc812877fd13698893 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "b37c12ab1f654afc812877fd13698893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.008535 19073 raft_consensus.cc:993] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893: : Instructing follower 4bada1aa2db9455e874b4139f9ad1a40 to start an election
I20260709 13:17:25.008962 19111 raft_consensus.cc:1081] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Signalling peer 4bada1aa2db9455e874b4139f9ad1a40 to start an election
I20260709 13:17:25.010517 19028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e957ba42136417fba964b3408dfec86"
dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
 from {username='slave'} at 127.0.0.1:60696
I20260709 13:17:25.010993 19028 raft_consensus.cc:493] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:25.011279 19028 raft_consensus.cc:3060] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.016461 19028 raft_consensus.cc:515] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.018092 19028 leader_election.cc:290] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 2 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:25.018922 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e957ba42136417fba964b3408dfec86" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:25.019156 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e957ba42136417fba964b3408dfec86" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:25.019577 18878 raft_consensus.cc:3060] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.019778 18953 raft_consensus.cc:3055] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:25.020145 18953 raft_consensus.cc:740] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:25.020759 18953 consensus_queue.cc:260] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.021672 18953 raft_consensus.cc:3060] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.024528 18878 raft_consensus.cc:2468] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 2.
I20260709 13:17:25.025552 18991 leader_election.cc:304] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:25.026219 19078 raft_consensus.cc:2804] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:25.026715 19078 raft_consensus.cc:697] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 LEADER]: Becoming Leader. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Running, Role: LEADER
I20260709 13:17:25.027315 18953 raft_consensus.cc:2468] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 2.
I20260709 13:17:25.027535 19078 consensus_queue.cc:237] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.033843 18775 catalog_manager.cc:5697] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 reported cstate change: term changed from 1 to 2, leader changed from b37c12ab1f654afc812877fd13698893 (127.6.198.66) to 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67). New cstate: current_term: 2 leader_uuid: "4bada1aa2db9455e874b4139f9ad1a40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.084400 19110 raft_consensus.cc:993] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893: : Instructing follower d1c86056b295463abc9760e7158f4bce to start an election
I20260709 13:17:25.084883 19111 raft_consensus.cc:1081] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Signalling peer d1c86056b295463abc9760e7158f4bce to start an election
I20260709 13:17:25.085177 19078 consensus_queue.cc:1048] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.086722 18879 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5a8849ca00284e83968f5c4d4f7a5f59"
dest_uuid: "d1c86056b295463abc9760e7158f4bce"
 from {username='slave'} at 127.0.0.1:46178
I20260709 13:17:25.087330 18879 raft_consensus.cc:493] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:25.087703 18879 raft_consensus.cc:3060] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.094885 19116 raft_consensus.cc:993] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40: : Instructing follower d1c86056b295463abc9760e7158f4bce to start an election
I20260709 13:17:25.095005 18879 raft_consensus.cc:515] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.095642 19078 raft_consensus.cc:1081] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Signalling peer d1c86056b295463abc9760e7158f4bce to start an election
I20260709 13:17:25.097290 18879 leader_election.cc:290] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 2 election: Requested vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:25.100123 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8849ca00284e83968f5c4d4f7a5f59" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:25.100718 18953 raft_consensus.cc:3055] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:25.101023 18953 raft_consensus.cc:740] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:25.101817 18878 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4799ea090d564639bc7e3998262adc88"
dest_uuid: "d1c86056b295463abc9760e7158f4bce"
 from {username='slave'} at 127.0.0.1:46180
I20260709 13:17:25.101797 18953 consensus_queue.cc:260] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.102528 18878 raft_consensus.cc:493] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:25.102917 18878 raft_consensus.cc:3060] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.103205 18953 raft_consensus.cc:3060] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.107227 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a8849ca00284e83968f5c4d4f7a5f59" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:25.107849 19028 raft_consensus.cc:3060] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.107710 19078 consensus_queue.cc:1048] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:25.117792 19110 raft_consensus.cc:993] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893: : Instructing follower d1c86056b295463abc9760e7158f4bce to start an election
I20260709 13:17:25.118280 19110 raft_consensus.cc:1081] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Signalling peer d1c86056b295463abc9760e7158f4bce to start an election
I20260709 13:17:25.120150 18879 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1dedc61807b4f1fb0b0df1fd7408660"
dest_uuid: "d1c86056b295463abc9760e7158f4bce"
 from {username='slave'} at 127.0.0.1:46178
I20260709 13:17:25.120277 18953 raft_consensus.cc:2468] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 2.
I20260709 13:17:25.120712 18879 raft_consensus.cc:493] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:25.121165 18879 raft_consensus.cc:3060] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.121596 18842 leader_election.cc:304] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:25.122438 19133 raft_consensus.cc:2804] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:25.123044 19133 raft_consensus.cc:697] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 2 LEADER]: Becoming Leader. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:25.124161 19028 raft_consensus.cc:2468] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 2.
I20260709 13:17:25.124130 19133 consensus_queue.cc:237] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.127220 18878 raft_consensus.cc:515] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.128221 18879 raft_consensus.cc:515] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.130431 18878 leader_election.cc:290] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 2 election: Requested vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:25.131337 18879 leader_election.cc:290] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 2 election: Requested vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:25.132647 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4799ea090d564639bc7e3998262adc88" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:25.133255 19028 raft_consensus.cc:3055] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:25.133589 19028 raft_consensus.cc:740] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Running, Role: LEADER
I20260709 13:17:25.133730 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1dedc61807b4f1fb0b0df1fd7408660" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:25.134320 19027 raft_consensus.cc:3060] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.134392 19028 consensus_queue.cc:260] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.135844 19028 raft_consensus.cc:3060] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.143530 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1dedc61807b4f1fb0b0df1fd7408660" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:25.144202 18953 raft_consensus.cc:3055] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:25.144513 19028 raft_consensus.cc:2468] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 2.
I20260709 13:17:25.144990 19027 raft_consensus.cc:2468] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 2.
I20260709 13:17:25.145668 18845 leader_election.cc:304] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:25.146095 19026 raft_consensus.cc:1240] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Rejecting Update request from peer b37c12ab1f654afc812877fd13698893 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:25.146620 18777 catalog_manager.cc:5697] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce reported cstate change: term changed from 1 to 2, leader changed from b37c12ab1f654afc812877fd13698893 (127.6.198.66) to d1c86056b295463abc9760e7158f4bce (127.6.198.65). New cstate: current_term: 2 leader_uuid: "d1c86056b295463abc9760e7158f4bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.148072 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4799ea090d564639bc7e3998262adc88" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:25.148715 18954 raft_consensus.cc:3060] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.149636 19073 consensus_queue.cc:1059] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:25.149896 19133 raft_consensus.cc:2804] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:25.150418 19133 raft_consensus.cc:697] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [term 2 LEADER]: Becoming Leader. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:25.151261 19133 consensus_queue.cc:237] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.153518 18845 leader_election.cc:304] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:25.154330 19150 raft_consensus.cc:2804] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:25.154806 19150 raft_consensus.cc:697] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [term 2 LEADER]: Becoming Leader. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:25.155678 18954 raft_consensus.cc:2468] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 2.
I20260709 13:17:25.155618 19150 consensus_queue.cc:237] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.144516 18953 raft_consensus.cc:740] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:25.159510 18953 consensus_queue.cc:260] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.160465 18953 raft_consensus.cc:3060] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.166543 18953 raft_consensus.cc:2468] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 2.
W20260709 13:17:25.166932 19073 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:17:25.167306 18777 catalog_manager.cc:5697] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce reported cstate change: term changed from 1 to 2, leader changed from 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67) to d1c86056b295463abc9760e7158f4bce (127.6.198.65). New cstate: current_term: 2 leader_uuid: "d1c86056b295463abc9760e7158f4bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.168951 18777 catalog_manager.cc:5697] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce reported cstate change: term changed from 1 to 2, leader changed from b37c12ab1f654afc812877fd13698893 (127.6.198.66) to d1c86056b295463abc9760e7158f4bce (127.6.198.65). New cstate: current_term: 2 leader_uuid: "d1c86056b295463abc9760e7158f4bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.171479 19092 raft_consensus.cc:993] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce: : Instructing follower b37c12ab1f654afc812877fd13698893 to start an election
I20260709 13:17:25.171949 19092 raft_consensus.cc:1081] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Signalling peer b37c12ab1f654afc812877fd13698893 to start an election
I20260709 13:17:25.173702 18953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d0a1577fa0f4741ab0ee73851eb1294"
dest_uuid: "b37c12ab1f654afc812877fd13698893"
 from {username='slave'} at 127.0.0.1:54062
I20260709 13:17:25.175568 18953 raft_consensus.cc:493] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:25.175920 18953 raft_consensus.cc:3060] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.182747 18953 raft_consensus.cc:515] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.185635 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0a1577fa0f4741ab0ee73851eb1294" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:25.186262 18879 raft_consensus.cc:3055] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:25.186597 18879 raft_consensus.cc:740] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:25.187361 18879 consensus_queue.cc:260] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.188407 18953 leader_election.cc:290] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 2 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:25.188488 18879 raft_consensus.cc:3060] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.189383 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d0a1577fa0f4741ab0ee73851eb1294" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:25.193730 19027 raft_consensus.cc:3060] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.196882 18879 raft_consensus.cc:2468] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 2.
I20260709 13:17:25.198089 18917 leader_election.cc:304] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:25.198912 19073 raft_consensus.cc:2804] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:25.199330 19073 raft_consensus.cc:697] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [term 2 LEADER]: Becoming Leader. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:25.200047 19073 consensus_queue.cc:237] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.200300 19027 raft_consensus.cc:2468] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 2.
I20260709 13:17:25.206317 18775 catalog_manager.cc:5697] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 reported cstate change: term changed from 1 to 2, leader changed from d1c86056b295463abc9760e7158f4bce (127.6.198.65) to b37c12ab1f654afc812877fd13698893 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "b37c12ab1f654afc812877fd13698893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.404284 19092 raft_consensus.cc:993] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce: : Instructing follower b37c12ab1f654afc812877fd13698893 to start an election
I20260709 13:17:25.404788 19150 raft_consensus.cc:1081] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Signalling peer b37c12ab1f654afc812877fd13698893 to start an election
I20260709 13:17:25.406090 18953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b1c41abe4964625829a184924c60d3a"
dest_uuid: "b37c12ab1f654afc812877fd13698893"
 from {username='slave'} at 127.0.0.1:54062
I20260709 13:17:25.406591 18953 raft_consensus.cc:493] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:25.406884 18953 raft_consensus.cc:3060] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.411882 18953 raft_consensus.cc:515] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.415037 18953 leader_election.cc:290] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 2 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:25.415870 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1c41abe4964625829a184924c60d3a" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:25.416049 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b1c41abe4964625829a184924c60d3a" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:25.416474 18879 raft_consensus.cc:3055] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:25.416774 19027 raft_consensus.cc:3060] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.416862 18879 raft_consensus.cc:740] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:25.417954 18879 consensus_queue.cc:260] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.419302 18879 raft_consensus.cc:3060] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:25.422753 19027 raft_consensus.cc:2468] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 2.
I20260709 13:17:25.423804 18920 leader_election.cc:304] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893; no voters: 
I20260709 13:17:25.424211 18879 raft_consensus.cc:2468] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 2.
I20260709 13:17:25.424494 19073 raft_consensus.cc:2804] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:25.425350 19073 raft_consensus.cc:697] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 2 LEADER]: Becoming Leader. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:25.426034 19073 consensus_queue.cc:237] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:25.432117 18775 catalog_manager.cc:5697] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 reported cstate change: term changed from 1 to 2, leader changed from d1c86056b295463abc9760e7158f4bce (127.6.198.65) to b37c12ab1f654afc812877fd13698893 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "b37c12ab1f654afc812877fd13698893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.479475 19027 raft_consensus.cc:1275] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Refusing update from remote peer b37c12ab1f654afc812877fd13698893: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:25.480702 19073 consensus_queue.cc:1048] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:25.487674 18879 raft_consensus.cc:1275] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Refusing update from remote peer b37c12ab1f654afc812877fd13698893: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:25.489362 19111 consensus_queue.cc:1048] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:25.539690 18953 raft_consensus.cc:1275] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Refusing update from remote peer d1c86056b295463abc9760e7158f4bce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:25.540632 18954 raft_consensus.cc:1275] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Refusing update from remote peer d1c86056b295463abc9760e7158f4bce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:25.540977 19150 consensus_queue.cc:1048] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:25.541832 19092 consensus_queue.cc:1048] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:25.553467 19027 raft_consensus.cc:1275] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Refusing update from remote peer d1c86056b295463abc9760e7158f4bce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:25.555684 19026 raft_consensus.cc:1275] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Refusing update from remote peer d1c86056b295463abc9760e7158f4bce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:25.557641 19150 consensus_queue.cc:1048] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:25.564810 18879 raft_consensus.cc:1275] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Refusing update from remote peer 4bada1aa2db9455e874b4139f9ad1a40: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:25.565793 19160 consensus_queue.cc:1048] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:25.569531 19157 consensus_queue.cc:1048] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:25.573493 18953 raft_consensus.cc:1275] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Refusing update from remote peer 4bada1aa2db9455e874b4139f9ad1a40: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:25.577683 19078 consensus_queue.cc:1048] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:25.598001 18953 raft_consensus.cc:1275] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Refusing update from remote peer d1c86056b295463abc9760e7158f4bce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:25.599237 19160 consensus_queue.cc:1048] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:25.605928 19027 raft_consensus.cc:1275] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Refusing update from remote peer d1c86056b295463abc9760e7158f4bce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:25.607272 19159 consensus_queue.cc:1048] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:25.760329 19027 raft_consensus.cc:1275] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Refusing update from remote peer b37c12ab1f654afc812877fd13698893: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:25.760329 18879 raft_consensus.cc:1275] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Refusing update from remote peer b37c12ab1f654afc812877fd13698893: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:25.761674 19111 consensus_queue.cc:1048] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:25.762229 19110 consensus_queue.cc:1048] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:25.940611 18879 raft_consensus.cc:1275] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Refusing update from remote peer b37c12ab1f654afc812877fd13698893: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:25.941762 19111 consensus_queue.cc:1048] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:25.949848 19027 raft_consensus.cc:1275] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Refusing update from remote peer b37c12ab1f654afc812877fd13698893: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:25.951220 19110 consensus_queue.cc:1048] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:27.951159 18777 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60998:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:27.953123 18777 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:28.002681 18867 tablet_service.cc:1511] Processing CreateTablet for tablet b8c7d4ef242b4910820fea2c73a795b3 (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:28.003877 18940 tablet_service.cc:1511] Processing CreateTablet for tablet b8c7d4ef242b4910820fea2c73a795b3 (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:28.004758 19013 tablet_service.cc:1511] Processing CreateTablet for tablet b8c7d4ef242b4910820fea2c73a795b3 (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:28.005118 18940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8c7d4ef242b4910820fea2c73a795b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.005755 19013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8c7d4ef242b4910820fea2c73a795b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.008292 19017 tablet_service.cc:1511] Processing CreateTablet for tablet 1aa4e1c6824341d0b7db3922adf767f7 (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:28.009493 19017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aa4e1c6824341d0b7db3922adf767f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.010171 18861 tablet_service.cc:1511] Processing CreateTablet for tablet 1aa4e1c6824341d0b7db3922adf767f7 (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:28.012182 18937 tablet_service.cc:1511] Processing CreateTablet for tablet 1aa4e1c6824341d0b7db3922adf767f7 (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:28.012763 18936 tablet_service.cc:1511] Processing CreateTablet for tablet 8b337eb0188147028dd46c7cda6597ba (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:28.013440 18937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aa4e1c6824341d0b7db3922adf767f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.003932 18867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8c7d4ef242b4910820fea2c73a795b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.015097 18865 tablet_service.cc:1511] Processing CreateTablet for tablet 00a17ef3085648e69f49a68f06005e60 (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:28.015296 18864 tablet_service.cc:1511] Processing CreateTablet for tablet 8b337eb0188147028dd46c7cda6597ba (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:28.016269 18865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00a17ef3085648e69f49a68f06005e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.016829 18864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b337eb0188147028dd46c7cda6597ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.017376 18936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b337eb0188147028dd46c7cda6597ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.022632 18861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1aa4e1c6824341d0b7db3922adf767f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.023145 18943 tablet_service.cc:1511] Processing CreateTablet for tablet 01a469d775f84c398d3030e2e0adcdae (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:28.023455 18939 tablet_service.cc:1511] Processing CreateTablet for tablet 00a17ef3085648e69f49a68f06005e60 (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:28.024365 18943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01a469d775f84c398d3030e2e0adcdae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.026433 18863 tablet_service.cc:1511] Processing CreateTablet for tablet 01a469d775f84c398d3030e2e0adcdae (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:28.027591 18863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01a469d775f84c398d3030e2e0adcdae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.027295 18942 tablet_service.cc:1511] Processing CreateTablet for tablet c6ae37ec42794c0a927016a80d2a6d2f (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:28.028529 18942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6ae37ec42794c0a927016a80d2a6d2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.030742 18862 tablet_service.cc:1511] Processing CreateTablet for tablet 4b873aaeb80544cd8c78969a9c265c2c (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:28.031930 18862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b873aaeb80544cd8c78969a9c265c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.035287 18869 tablet_service.cc:1511] Processing CreateTablet for tablet c6ae37ec42794c0a927016a80d2a6d2f (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:28.036429 18869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6ae37ec42794c0a927016a80d2a6d2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.038582 18939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00a17ef3085648e69f49a68f06005e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.043045 19016 tablet_service.cc:1511] Processing CreateTablet for tablet 8b337eb0188147028dd46c7cda6597ba (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:28.044314 19016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b337eb0188147028dd46c7cda6597ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.047338 19011 tablet_service.cc:1511] Processing CreateTablet for tablet 00a17ef3085648e69f49a68f06005e60 (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:28.048548 19011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00a17ef3085648e69f49a68f06005e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.051282 19018 tablet_service.cc:1511] Processing CreateTablet for tablet 01a469d775f84c398d3030e2e0adcdae (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:28.052534 19018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01a469d775f84c398d3030e2e0adcdae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.054889 19010 tablet_service.cc:1511] Processing CreateTablet for tablet c6ae37ec42794c0a927016a80d2a6d2f (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:28.056506 19010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6ae37ec42794c0a927016a80d2a6d2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.059443 19012 tablet_service.cc:1511] Processing CreateTablet for tablet 4b873aaeb80544cd8c78969a9c265c2c (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:28.061069 19012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b873aaeb80544cd8c78969a9c265c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.063632 19014 tablet_service.cc:1511] Processing CreateTablet for tablet db427c7f60374393bc721fed01136cdf (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:28.065274 19014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db427c7f60374393bc721fed01136cdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.067629 19015 tablet_service.cc:1511] Processing CreateTablet for tablet 07373452952c4549acb3d36b47e24e3a (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:28.069190 19015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07373452952c4549acb3d36b47e24e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.072358 18941 tablet_service.cc:1511] Processing CreateTablet for tablet 4b873aaeb80544cd8c78969a9c265c2c (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:28.072916 18938 tablet_service.cc:1511] Processing CreateTablet for tablet 07373452952c4549acb3d36b47e24e3a (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:28.073957 18941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b873aaeb80544cd8c78969a9c265c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.073983 18944 tablet_service.cc:1511] Processing CreateTablet for tablet db427c7f60374393bc721fed01136cdf (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:28.074965 18938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07373452952c4549acb3d36b47e24e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.075578 18944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db427c7f60374393bc721fed01136cdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.078364 18866 tablet_service.cc:1511] Processing CreateTablet for tablet db427c7f60374393bc721fed01136cdf (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:28.078688 18868 tablet_service.cc:1511] Processing CreateTablet for tablet 07373452952c4549acb3d36b47e24e3a (DEFAULT_TABLE table=soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:28.079617 18866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db427c7f60374393bc721fed01136cdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.080885 18868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07373452952c4549acb3d36b47e24e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:28.126358 19179 tablet_bootstrap.cc:492] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:28.133761 19179 tablet_bootstrap.cc:654] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.143522 19182 tablet_bootstrap.cc:492] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:28.151413 19181 tablet_bootstrap.cc:492] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:28.155539 19182 tablet_bootstrap.cc:654] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.167691 19181 tablet_bootstrap.cc:654] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.179903 19181 tablet_bootstrap.cc:492] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:28.180619 19181 ts_tablet_manager.cc:1403] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.006s
I20260709 13:17:28.184954 19181 raft_consensus.cc:359] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.186159 19181 raft_consensus.cc:385] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.186571 19181 raft_consensus.cc:740] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.187626 19181 consensus_queue.cc:260] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.196830 19181 ts_tablet_manager.cc:1434] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:17:28.198175 19181 tablet_bootstrap.cc:492] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:28.212160 19181 tablet_bootstrap.cc:654] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.225469 19181 tablet_bootstrap.cc:492] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:28.226117 19181 ts_tablet_manager.cc:1403] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.007s
I20260709 13:17:28.229628 19181 raft_consensus.cc:359] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.230314 19181 raft_consensus.cc:385] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.230612 19181 raft_consensus.cc:740] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.231470 19181 consensus_queue.cc:260] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.236970 19181 ts_tablet_manager.cc:1434] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.005s
I20260709 13:17:28.238632 19181 tablet_bootstrap.cc:492] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:28.246928 19181 tablet_bootstrap.cc:654] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.271447 19181 tablet_bootstrap.cc:492] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:28.272323 19181 ts_tablet_manager.cc:1403] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.003s
I20260709 13:17:28.276343 19181 raft_consensus.cc:359] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.277033 19181 raft_consensus.cc:385] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.277369 19181 raft_consensus.cc:740] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.278169 19181 consensus_queue.cc:260] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.281674 19181 ts_tablet_manager.cc:1434] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:17:28.283039 19181 tablet_bootstrap.cc:492] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:28.292187 19181 tablet_bootstrap.cc:654] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.306559 19181 tablet_bootstrap.cc:492] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:28.307178 19181 ts_tablet_manager.cc:1403] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.007s
I20260709 13:17:28.310812 19181 raft_consensus.cc:359] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.311725 19181 raft_consensus.cc:385] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.312099 19181 raft_consensus.cc:740] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.313221 19181 consensus_queue.cc:260] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.324361 19181 ts_tablet_manager.cc:1434] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:17:28.325620 19181 tablet_bootstrap.cc:492] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:28.335525 19181 tablet_bootstrap.cc:654] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:28.383865 18980 tablet.cc:2406] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:17:28.403869 19110 raft_consensus.cc:493] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.404693 19110 raft_consensus.cc:515] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.408473 19179 tablet_bootstrap.cc:492] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
W20260709 13:17:28.409065 18917 leader_election.cc:343] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Not found: Tablet not found: 07373452952c4549acb3d36b47e24e3a
I20260709 13:17:28.409047 19179 ts_tablet_manager.cc:1403] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.283s	user 0.102s	sys 0.170s
I20260709 13:17:28.412788 19179 raft_consensus.cc:359] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.413508 19179 raft_consensus.cc:385] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.413825 19179 raft_consensus.cc:740] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.414651 19179 consensus_queue.cc:260] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.416059 19110 leader_election.cc:290] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:28.416921 19185 raft_consensus.cc:493] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:28.417227 18920 leader_election.cc:343] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Not found: Tablet not found: 07373452952c4549acb3d36b47e24e3a
I20260709 13:17:28.417390 19185 raft_consensus.cc:515] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.417932 18920 leader_election.cc:304] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b37c12ab1f654afc812877fd13698893; no voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce
I20260709 13:17:28.419440 19110 raft_consensus.cc:2749] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.419602 19185 leader_election.cc:290] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:28.421967 19026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aa4e1c6824341d0b7db3922adf767f7" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:28.422166 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aa4e1c6824341d0b7db3922adf767f7" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:28.423115 19179 ts_tablet_manager.cc:1434] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
W20260709 13:17:28.423614 18917 leader_election.cc:343] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.424211 19179 tablet_bootstrap.cc:492] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
W20260709 13:17:28.424171 18920 leader_election.cc:343] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.424906 18920 leader_election.cc:304] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b37c12ab1f654afc812877fd13698893; no voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce
I20260709 13:17:28.427937 19182 tablet_bootstrap.cc:492] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:28.428449 19182 ts_tablet_manager.cc:1403] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.285s	user 0.086s	sys 0.153s
I20260709 13:17:28.432111 19179 tablet_bootstrap.cc:654] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.432015 19182 raft_consensus.cc:359] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.433041 19182 raft_consensus.cc:385] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.433434 19182 raft_consensus.cc:740] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.442551 19191 raft_consensus.cc:2749] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.444057 19179 tablet_bootstrap.cc:492] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:28.444926 19179 ts_tablet_manager.cc:1403] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.000s
I20260709 13:17:28.448287 19179 raft_consensus.cc:359] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.449118 19179 raft_consensus.cc:385] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.449465 19179 raft_consensus.cc:740] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.450223 19179 consensus_queue.cc:260] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.443676 19182 consensus_queue.cc:260] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.452778 19182 ts_tablet_manager.cc:1434] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.003s
I20260709 13:17:28.453806 19182 tablet_bootstrap.cc:492] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:28.459749 19182 tablet_bootstrap.cc:654] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.461764 19179 ts_tablet_manager.cc:1434] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 13:17:28.461930 19150 raft_consensus.cc:493] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.462458 19150 raft_consensus.cc:515] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.462910 19179 tablet_bootstrap.cc:492] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:28.467502 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db427c7f60374393bc721fed01136cdf" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
W20260709 13:17:28.469055 18842 leader_election.cc:343] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.469969 19179 tablet_bootstrap.cc:654] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.471788 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db427c7f60374393bc721fed01136cdf" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
W20260709 13:17:28.474612 18845 leader_election.cc:343] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.475049 18845 leader_election.cc:304] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1c86056b295463abc9760e7158f4bce; no voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893
I20260709 13:17:28.479815 19190 raft_consensus.cc:2749] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.480000 19179 tablet_bootstrap.cc:492] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:28.480609 19179 ts_tablet_manager.cc:1403] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.004s
I20260709 13:17:28.484822 19150 leader_election.cc:290] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:28.487897 19179 raft_consensus.cc:359] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.488804 19179 raft_consensus.cc:385] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.489097 19179 raft_consensus.cc:740] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.489571 19182 tablet_bootstrap.cc:492] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:28.490057 19182 ts_tablet_manager.cc:1403] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.003s
I20260709 13:17:28.492267 19179 consensus_queue.cc:260] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.494038 19182 raft_consensus.cc:359] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.494787 19179 ts_tablet_manager.cc:1434] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.004s
I20260709 13:17:28.495245 19182 raft_consensus.cc:385] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.495666 19182 raft_consensus.cc:740] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.495800 19179 tablet_bootstrap.cc:492] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:28.497363 19182 consensus_queue.cc:260] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.498454 19181 tablet_bootstrap.cc:492] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:28.498957 19181 ts_tablet_manager.cc:1403] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.174s	user 0.052s	sys 0.073s
I20260709 13:17:28.499058 19185 raft_consensus.cc:493] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.499557 19185 raft_consensus.cc:515] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.501616 19185 leader_election.cc:290] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:28.503391 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01a469d775f84c398d3030e2e0adcdae" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:28.504837 19181 raft_consensus.cc:359] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.505551 19181 raft_consensus.cc:385] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.505820 19181 raft_consensus.cc:740] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.506563 19181 consensus_queue.cc:260] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.508857 19181 ts_tablet_manager.cc:1434] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
W20260709 13:17:28.506904 18920 leader_election.cc:343] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.510937 19182 ts_tablet_manager.cc:1434] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 13:17:28.511281 19181 tablet_bootstrap.cc:492] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:28.510991 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01a469d775f84c398d3030e2e0adcdae" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
W20260709 13:17:28.513453 18917 leader_election.cc:343] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.513880 18917 leader_election.cc:304] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b37c12ab1f654afc812877fd13698893; no voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce
I20260709 13:17:28.511922 19182 tablet_bootstrap.cc:492] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:28.514904 19179 tablet_bootstrap.cc:654] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.515000 19191 raft_consensus.cc:2749] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.521050 19181 tablet_bootstrap.cc:654] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.528822 19182 tablet_bootstrap.cc:654] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.532524 19181 tablet_bootstrap.cc:492] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:28.533215 19181 ts_tablet_manager.cc:1403] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.002s
I20260709 13:17:28.536430 19181 raft_consensus.cc:359] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.537465 19181 raft_consensus.cc:385] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.537901 19181 raft_consensus.cc:740] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.539074 19181 consensus_queue.cc:260] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.542198 19181 ts_tablet_manager.cc:1434] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:17:28.543121 19150 raft_consensus.cc:493] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.543763 19150 raft_consensus.cc:515] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.549119 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b873aaeb80544cd8c78969a9c265c2c" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:28.550868 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b873aaeb80544cd8c78969a9c265c2c" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:28.551754 18953 raft_consensus.cc:2468] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 0.
I20260709 13:17:28.553400 18842 leader_election.cc:304] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:28.554404 19190 raft_consensus.cc:2804] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.554752 19190 raft_consensus.cc:493] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:28.554970 18845 leader_election.cc:343] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.555078 19190 raft_consensus.cc:3060] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.555606 19150 leader_election.cc:290] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:28.556972 19181 tablet_bootstrap.cc:492] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:28.562669 19190 raft_consensus.cc:515] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.565423 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b873aaeb80544cd8c78969a9c265c2c" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:28.565521 19190 leader_election.cc:290] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 election: Requested vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:28.566025 18953 raft_consensus.cc:3060] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.566540 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b873aaeb80544cd8c78969a9c265c2c" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
W20260709 13:17:28.568094 18845 leader_election.cc:343] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.570670 19182 tablet_bootstrap.cc:492] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:28.570757 19179 tablet_bootstrap.cc:492] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:28.571677 19182 ts_tablet_manager.cc:1403] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.060s	user 0.014s	sys 0.019s
I20260709 13:17:28.571780 19179 ts_tablet_manager.cc:1403] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.076s	user 0.047s	sys 0.013s
I20260709 13:17:28.573220 18953 raft_consensus.cc:2468] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 1.
I20260709 13:17:28.571094 19181 tablet_bootstrap.cc:654] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.574401 18842 leader_election.cc:304] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 4bada1aa2db9455e874b4139f9ad1a40
I20260709 13:17:28.575167 19190 raft_consensus.cc:2804] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.575731 19190 raft_consensus.cc:697] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Becoming Leader. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:28.576613 19190 consensus_queue.cc:237] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.579538 19182 raft_consensus.cc:359] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.579736 19179 raft_consensus.cc:359] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.581511 19179 raft_consensus.cc:385] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.581935 19179 raft_consensus.cc:740] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.582963 19179 consensus_queue.cc:260] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.585158 19179 ts_tablet_manager.cc:1434] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 13:17:28.586290 19179 tablet_bootstrap.cc:492] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:28.586484 19182 raft_consensus.cc:385] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.586937 19182 raft_consensus.cc:740] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.592140 19182 consensus_queue.cc:260] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.593205 19179 tablet_bootstrap.cc:654] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.594652 19182 ts_tablet_manager.cc:1434] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.004s
I20260709 13:17:28.594012 18776 catalog_manager.cc:5697] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce reported cstate change: term changed from 0 to 1, leader changed from <none> to d1c86056b295463abc9760e7158f4bce (127.6.198.65). New cstate: current_term: 1 leader_uuid: "d1c86056b295463abc9760e7158f4bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.595825 19182 tablet_bootstrap.cc:492] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:28.604615 19182 tablet_bootstrap.cc:654] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.608395 19181 tablet_bootstrap.cc:492] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:28.608934 19181 ts_tablet_manager.cc:1403] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.052s	user 0.019s	sys 0.016s
I20260709 13:17:28.612556 19116 raft_consensus.cc:493] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.612928 19179 tablet_bootstrap.cc:492] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:28.613018 19116 raft_consensus.cc:515] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.613772 19179 ts_tablet_manager.cc:1403] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.012s
I20260709 13:17:28.615483 19116 leader_election.cc:290] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:28.616137 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aa4e1c6824341d0b7db3922adf767f7" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:28.616772 18879 raft_consensus.cc:2468] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 0.
I20260709 13:17:28.617914 19179 raft_consensus.cc:359] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.618480 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aa4e1c6824341d0b7db3922adf767f7" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:28.618939 19179 raft_consensus.cc:385] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.619130 18953 raft_consensus.cc:2468] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 0.
I20260709 13:17:28.619375 19179 raft_consensus.cc:740] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.620342 19179 consensus_queue.cc:260] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.621094 18991 leader_election.cc:304] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:28.622108 19116 raft_consensus.cc:2804] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.622476 19116 raft_consensus.cc:493] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.622818 19116 raft_consensus.cc:3060] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.623679 19179 ts_tablet_manager.cc:1434] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.003s
I20260709 13:17:28.625026 19179 tablet_bootstrap.cc:492] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:28.612020 19181 raft_consensus.cc:359] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.633235 19179 tablet_bootstrap.cc:654] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.633464 19181 raft_consensus.cc:385] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.634681 19181 raft_consensus.cc:740] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.636612 19116 raft_consensus.cc:515] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.642709 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aa4e1c6824341d0b7db3922adf767f7" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:28.643344 18879 raft_consensus.cc:3060] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.650776 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aa4e1c6824341d0b7db3922adf767f7" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:28.651567 18953 raft_consensus.cc:3060] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.658504 19185 raft_consensus.cc:493] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.659070 19185 raft_consensus.cc:515] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.637847 19181 consensus_queue.cc:260] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.661151 19185 leader_election.cc:290] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:28.663964 19181 ts_tablet_manager.cc:1434] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.000s
I20260709 13:17:28.665256 19181 tablet_bootstrap.cc:492] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:28.666503 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ae37ec42794c0a927016a80d2a6d2f" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:28.641192 19116 leader_election.cc:290] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:28.667160 18878 raft_consensus.cc:2468] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 0.
I20260709 13:17:28.670071 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ae37ec42794c0a927016a80d2a6d2f" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:28.670413 18917 leader_election.cc:304] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:28.671317 19191 raft_consensus.cc:2804] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.671672 19191 raft_consensus.cc:493] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.671981 19191 raft_consensus.cc:3060] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.675333 19182 tablet_bootstrap.cc:492] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:28.675940 19182 ts_tablet_manager.cc:1403] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.080s	user 0.041s	sys 0.034s
I20260709 13:17:28.677119 18879 raft_consensus.cc:2468] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 1.
W20260709 13:17:28.675945 18920 leader_election.cc:343] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.680722 18991 leader_election.cc:304] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:28.681551 19116 raft_consensus.cc:2804] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.682049 19116 raft_consensus.cc:697] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Becoming Leader. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Running, Role: LEADER
I20260709 13:17:28.683456 19116 consensus_queue.cc:237] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.686280 18953 raft_consensus.cc:2468] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 1.
I20260709 13:17:28.692528 19182 raft_consensus.cc:359] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.693030 19191 raft_consensus.cc:515] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.697479 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ae37ec42794c0a927016a80d2a6d2f" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:28.698019 18878 raft_consensus.cc:3060] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.693552 19182 raft_consensus.cc:385] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.700079 19181 tablet_bootstrap.cc:654] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.709601 19191 leader_election.cc:290] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:28.709555 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ae37ec42794c0a927016a80d2a6d2f" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
W20260709 13:17:28.711205 18920 leader_election.cc:343] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.708488 19182 raft_consensus.cc:740] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.715449 19182 consensus_queue.cc:260] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.718969 19179 tablet_bootstrap.cc:492] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:28.719473 19179 ts_tablet_manager.cc:1403] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.095s	user 0.024s	sys 0.053s
I20260709 13:17:28.720273 18776 catalog_manager.cc:5697] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "4bada1aa2db9455e874b4139f9ad1a40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:28.722322 19179 raft_consensus.cc:359] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.723055 19179 raft_consensus.cc:385] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.723345 19179 raft_consensus.cc:740] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.724179 19179 consensus_queue.cc:260] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.725610 18878 raft_consensus.cc:2468] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 1.
I20260709 13:17:28.726794 18917 leader_election.cc:304] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 4bada1aa2db9455e874b4139f9ad1a40
I20260709 13:17:28.727905 19191 raft_consensus.cc:2804] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.728369 19191 raft_consensus.cc:697] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Becoming Leader. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:28.729174 19191 consensus_queue.cc:237] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.730480 19150 raft_consensus.cc:493] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.730957 19150 raft_consensus.cc:515] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.732959 19150 leader_election.cc:290] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:28.737203 19182 ts_tablet_manager.cc:1434] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.061s	user 0.002s	sys 0.005s
I20260709 13:17:28.739486 19182 tablet_bootstrap.cc:492] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:28.740124 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c7d4ef242b4910820fea2c73a795b3" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:28.741281 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c7d4ef242b4910820fea2c73a795b3" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:28.741876 19027 raft_consensus.cc:2468] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 0.
W20260709 13:17:28.743134 18842 leader_election.cc:343] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.746622 19179 ts_tablet_manager.cc:1434] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:17:28.747634 19179 tablet_bootstrap.cc:492] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:28.750200 18845 leader_election.cc:304] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: b37c12ab1f654afc812877fd13698893
I20260709 13:17:28.758883 19190 raft_consensus.cc:2804] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.759315 19190 raft_consensus.cc:493] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.759641 19190 raft_consensus.cc:3060] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.754487 19179 tablet_bootstrap.cc:654] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.760632 18775 catalog_manager.cc:5697] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 reported cstate change: term changed from 0 to 1, leader changed from <none> to b37c12ab1f654afc812877fd13698893 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "b37c12ab1f654afc812877fd13698893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.763844 19182 tablet_bootstrap.cc:654] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.770556 19116 raft_consensus.cc:493] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.771050 19116 raft_consensus.cc:515] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.772969 19116 leader_election.cc:290] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:28.775614 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01a469d775f84c398d3030e2e0adcdae" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:28.775787 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01a469d775f84c398d3030e2e0adcdae" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:28.776726 18953 raft_consensus.cc:2468] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 0.
W20260709 13:17:28.777212 18991 leader_election.cc:343] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.778139 18991 leader_election.cc:304] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893; no voters: d1c86056b295463abc9760e7158f4bce
I20260709 13:17:28.779079 19116 raft_consensus.cc:2804] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.779449 19116 raft_consensus.cc:493] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.779770 19116 raft_consensus.cc:3060] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.786113 19182 tablet_bootstrap.cc:492] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:28.785125 19190 raft_consensus.cc:515] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.786875 19116 raft_consensus.cc:515] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.788257 19182 ts_tablet_manager.cc:1403] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.008s
I20260709 13:17:28.789244 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c7d4ef242b4910820fea2c73a795b3" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893"
W20260709 13:17:28.790655 18842 leader_election.cc:343] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.790899 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01a469d775f84c398d3030e2e0adcdae" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce"
W20260709 13:17:28.792562 18991 leader_election.cc:343] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.793619 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c7d4ef242b4910820fea2c73a795b3" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:28.794170 19027 raft_consensus.cc:3060] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.794567 19182 raft_consensus.cc:359] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.795528 19116 leader_election.cc:290] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:28.795562 19182 raft_consensus.cc:385] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.796205 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01a469d775f84c398d3030e2e0adcdae" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:28.798712 18953 raft_consensus.cc:3060] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.796221 19182 raft_consensus.cc:740] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.804170 19027 raft_consensus.cc:2468] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 1.
I20260709 13:17:28.805312 19181 tablet_bootstrap.cc:492] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:28.805399 18845 leader_election.cc:304] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: b37c12ab1f654afc812877fd13698893
I20260709 13:17:28.806138 19181 ts_tablet_manager.cc:1403] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.141s	user 0.013s	sys 0.020s
I20260709 13:17:28.806591 19150 raft_consensus.cc:2804] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.814694 19182 consensus_queue.cc:260] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.817926 19182 ts_tablet_manager.cc:1434] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.029s	user 0.002s	sys 0.005s
I20260709 13:17:28.818449 19190 leader_election.cc:290] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 election: Requested vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:28.822326 19212 raft_consensus.cc:493] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.823431 19212 raft_consensus.cc:515] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.824143 19182 tablet_bootstrap.cc:492] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:28.822703 18953 raft_consensus.cc:2468] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 1.
I20260709 13:17:28.822763 19150 raft_consensus.cc:697] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Becoming Leader. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:28.826290 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b337eb0188147028dd46c7cda6597ba" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:28.827212 19150 consensus_queue.cc:237] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
W20260709 13:17:28.827884 18842 leader_election.cc:343] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.830327 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b337eb0188147028dd46c7cda6597ba" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:28.831718 19179 tablet_bootstrap.cc:492] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
W20260709 13:17:28.831782 18845 leader_election.cc:343] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.832369 19179 ts_tablet_manager.cc:1403] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.085s	user 0.026s	sys 0.031s
I20260709 13:17:28.832428 18845 leader_election.cc:304] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1c86056b295463abc9760e7158f4bce; no voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893
I20260709 13:17:28.833480 19190 raft_consensus.cc:2749] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.809737 19181 raft_consensus.cc:359] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.834333 19181 raft_consensus.cc:385] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.834649 19181 raft_consensus.cc:740] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.834674 19212 leader_election.cc:290] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
W20260709 13:17:28.835216 18825 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:28.835881 18991 leader_election.cc:304] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893; no voters: d1c86056b295463abc9760e7158f4bce
I20260709 13:17:28.836683 19116 raft_consensus.cc:2804] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.837157 19116 raft_consensus.cc:697] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Becoming Leader. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Running, Role: LEADER
I20260709 13:17:28.837978 19116 consensus_queue.cc:237] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.839337 19179 raft_consensus.cc:359] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.839826 19182 tablet_bootstrap.cc:654] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.840070 19179 raft_consensus.cc:385] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.840368 19179 raft_consensus.cc:740] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.841235 19179 consensus_queue.cc:260] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.843547 19179 ts_tablet_manager.cc:1434] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:17:28.844455 19181 consensus_queue.cc:260] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.848834 19181 ts_tablet_manager.cc:1434] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 13:17:28.849740 19181 tablet_bootstrap.cc:492] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893: Bootstrap starting.
I20260709 13:17:28.849327 18776 catalog_manager.cc:5697] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce reported cstate change: term changed from 0 to 1, leader changed from <none> to d1c86056b295463abc9760e7158f4bce (127.6.198.65). New cstate: current_term: 1 leader_uuid: "d1c86056b295463abc9760e7158f4bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.852294 19179 tablet_bootstrap.cc:492] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:28.856117 19181 tablet_bootstrap.cc:654] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.864791 18777 catalog_manager.cc:5697] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "4bada1aa2db9455e874b4139f9ad1a40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:28.869748 19179 tablet_bootstrap.cc:654] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.882810 19182 tablet_bootstrap.cc:492] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:28.884565 19182 ts_tablet_manager.cc:1403] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.061s	user 0.011s	sys 0.008s
I20260709 13:17:28.887979 19179 tablet_bootstrap.cc:492] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:28.888407 19179 ts_tablet_manager.cc:1403] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.012s
I20260709 13:17:28.890782 19179 raft_consensus.cc:359] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.891332 19179 raft_consensus.cc:385] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.891546 19179 raft_consensus.cc:740] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.892122 19179 consensus_queue.cc:260] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.895130 19181 tablet_bootstrap.cc:492] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893: No bootstrap required, opened a new log
I20260709 13:17:28.895617 19181 ts_tablet_manager.cc:1403] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.000s
I20260709 13:17:28.896003 19182 raft_consensus.cc:359] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.896733 19182 raft_consensus.cc:385] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.897006 19182 raft_consensus.cc:740] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.897714 19182 consensus_queue.cc:260] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.899967 19182 ts_tablet_manager.cc:1434] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:17:28.900947 19182 tablet_bootstrap.cc:492] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:28.898412 19181 raft_consensus.cc:359] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.901404 19181 raft_consensus.cc:385] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.901713 19181 raft_consensus.cc:740] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.902422 19181 consensus_queue.cc:260] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.903972 19181 ts_tablet_manager.cc:1434] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:28.907503 19179 ts_tablet_manager.cc:1434] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:17:28.908456 19179 tablet_bootstrap.cc:492] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce: Bootstrap starting.
I20260709 13:17:28.913704 19179 tablet_bootstrap.cc:654] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.916705 19182 tablet_bootstrap.cc:654] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.922705 19179 tablet_bootstrap.cc:492] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce: No bootstrap required, opened a new log
I20260709 13:17:28.923149 19179 ts_tablet_manager.cc:1403] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 13:17:28.925858 19179 raft_consensus.cc:359] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.926823 19179 raft_consensus.cc:385] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.927222 19179 raft_consensus.cc:740] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.928119 19179 consensus_queue.cc:260] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.931221 19179 ts_tablet_manager.cc:1434] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:28.936587 19182 tablet_bootstrap.cc:492] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:28.937042 19182 ts_tablet_manager.cc:1403] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.005s
I20260709 13:17:28.939162 19182 raft_consensus.cc:359] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.939669 19182 raft_consensus.cc:385] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.939913 19182 raft_consensus.cc:740] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.940544 19182 consensus_queue.cc:260] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.942509 19182 ts_tablet_manager.cc:1434] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:17:28.943360 19182 tablet_bootstrap.cc:492] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:28.949349 19182 tablet_bootstrap.cc:654] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.957458 19182 tablet_bootstrap.cc:492] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:28.957846 19182 ts_tablet_manager.cc:1403] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.015s	user 0.006s	sys 0.005s
I20260709 13:17:28.960115 19182 raft_consensus.cc:359] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.960703 19182 raft_consensus.cc:385] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.960947 19182 raft_consensus.cc:740] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.961532 19182 consensus_queue.cc:260] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.963056 19182 ts_tablet_manager.cc:1434] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.001s
I20260709 13:17:28.963928 19182 tablet_bootstrap.cc:492] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40: Bootstrap starting.
I20260709 13:17:28.968662 19182 tablet_bootstrap.cc:654] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.974097 19182 tablet_bootstrap.cc:492] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40: No bootstrap required, opened a new log
I20260709 13:17:28.974512 19182 ts_tablet_manager.cc:1403] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40: Time spent bootstrapping tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:17:28.976827 19182 raft_consensus.cc:359] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.977288 19182 raft_consensus.cc:385] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.977504 19182 raft_consensus.cc:740] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.978024 19182 consensus_queue.cc:260] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.979681 19182 ts_tablet_manager.cc:1434] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:17:28.997133 19191 raft_consensus.cc:493] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.997574 19191 raft_consensus.cc:515] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:28.999276 19191 leader_election.cc:290] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:28.999994 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00a17ef3085648e69f49a68f06005e60" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:29.000303 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00a17ef3085648e69f49a68f06005e60" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:29.000730 18878 raft_consensus.cc:2468] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 0.
I20260709 13:17:29.001050 19027 raft_consensus.cc:2468] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 0.
I20260709 13:17:29.001734 19191 raft_consensus.cc:493] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.001996 18917 leader_election.cc:304] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:29.002233 19191 raft_consensus.cc:515] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.002907 19185 raft_consensus.cc:2804] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:29.003261 19185 raft_consensus.cc:493] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.003571 19185 raft_consensus.cc:3060] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.004302 19191 leader_election.cc:290] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:29.004980 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db427c7f60374393bc721fed01136cdf" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:29.005653 18878 raft_consensus.cc:2468] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 0.
I20260709 13:17:29.006717 18917 leader_election.cc:304] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:29.006922 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db427c7f60374393bc721fed01136cdf" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:29.007578 19191 raft_consensus.cc:2804] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:29.007689 19027 raft_consensus.cc:2468] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 0.
I20260709 13:17:29.008024 19191 raft_consensus.cc:493] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.008543 19191 raft_consensus.cc:3060] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.010921 19185 raft_consensus.cc:515] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.012969 19185 leader_election.cc:290] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:29.013278 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00a17ef3085648e69f49a68f06005e60" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:29.013662 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00a17ef3085648e69f49a68f06005e60" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:29.013870 18878 raft_consensus.cc:3060] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.013971 19116 raft_consensus.cc:493] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.014308 19027 raft_consensus.cc:3060] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.014576 19116 raft_consensus.cc:515] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.016569 19116 leader_election.cc:290] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:29.017524 19191 raft_consensus.cc:515] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.018113 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07373452952c4549acb3d36b47e24e3a" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:29.018744 18953 raft_consensus.cc:2468] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 0.
I20260709 13:17:29.019570 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07373452952c4549acb3d36b47e24e3a" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:29.019709 19191 leader_election.cc:290] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:29.020227 18879 raft_consensus.cc:2468] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 0.
I20260709 13:17:29.021158 19027 raft_consensus.cc:2468] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 1.
I20260709 13:17:29.021917 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db427c7f60374393bc721fed01136cdf" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:29.022449 18879 raft_consensus.cc:3060] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.022617 18920 leader_election.cc:304] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893; no voters: 
I20260709 13:17:29.022960 18878 raft_consensus.cc:2468] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 1.
I20260709 13:17:29.023026 18991 leader_election.cc:304] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893; no voters: 
I20260709 13:17:29.023623 19191 raft_consensus.cc:2804] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:29.023866 19116 raft_consensus.cc:2804] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:29.024170 19191 raft_consensus.cc:697] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Becoming Leader. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:29.024507 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db427c7f60374393bc721fed01136cdf" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:29.024340 19116 raft_consensus.cc:493] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.025120 19116 raft_consensus.cc:3060] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.025130 19027 raft_consensus.cc:3060] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.028935 18879 raft_consensus.cc:2468] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 1.
I20260709 13:17:29.030479 18917 leader_election.cc:304] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:29.031921 19027 raft_consensus.cc:2468] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 1.
I20260709 13:17:29.031751 19116 raft_consensus.cc:515] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.033152 19185 raft_consensus.cc:2804] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:29.033651 19185 raft_consensus.cc:697] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Becoming Leader. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:29.034475 19116 leader_election.cc:290] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:29.034454 19185 consensus_queue.cc:237] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.025313 19191 consensus_queue.cc:237] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.034416 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07373452952c4549acb3d36b47e24e3a" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:29.041541 18879 raft_consensus.cc:3060] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.045790 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07373452952c4549acb3d36b47e24e3a" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:29.046574 18953 raft_consensus.cc:3060] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.050697 19116 raft_consensus.cc:493] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.051368 19116 raft_consensus.cc:515] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.053923 19116 leader_election.cc:290] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:29.054874 18953 raft_consensus.cc:2468] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 1.
I20260709 13:17:29.055318 19150 raft_consensus.cc:493] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.055522 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b873aaeb80544cd8c78969a9c265c2c" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:29.056046 19150 raft_consensus.cc:515] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.056923 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b873aaeb80544cd8c78969a9c265c2c" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:29.057885 18953 raft_consensus.cc:2393] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4bada1aa2db9455e874b4139f9ad1a40 in current term 1: Already voted for candidate d1c86056b295463abc9760e7158f4bce in this term.
I20260709 13:17:29.059474 18991 leader_election.cc:304] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893; no voters: 
I20260709 13:17:29.060611 18991 leader_election.cc:304] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40; no voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce
I20260709 13:17:29.061523 19205 raft_consensus.cc:2804] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:29.062570 19150 leader_election.cc:290] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:29.063910 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01a469d775f84c398d3030e2e0adcdae" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:29.067225 18775 catalog_manager.cc:5697] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 reported cstate change: term changed from 0 to 1, leader changed from <none> to b37c12ab1f654afc812877fd13698893 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "b37c12ab1f654afc812877fd13698893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.072362 18879 raft_consensus.cc:2468] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 1.
I20260709 13:17:29.064682 19116 raft_consensus.cc:3060] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.081283 19205 raft_consensus.cc:697] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Becoming Leader. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Running, Role: LEADER
I20260709 13:17:29.081686 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01a469d775f84c398d3030e2e0adcdae" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:29.082433 18953 raft_consensus.cc:2393] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1c86056b295463abc9760e7158f4bce in current term 1: Already voted for candidate 4bada1aa2db9455e874b4139f9ad1a40 in this term.
I20260709 13:17:29.082221 19205 consensus_queue.cc:237] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.086154 19226 consensus_queue.cc:1048] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.093245 18775 catalog_manager.cc:5697] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 reported cstate change: term changed from 0 to 1, leader changed from <none> to b37c12ab1f654afc812877fd13698893 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "b37c12ab1f654afc812877fd13698893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.097146 18842 leader_election.cc:304] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1c86056b295463abc9760e7158f4bce; no voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893
I20260709 13:17:29.098695 19150 raft_consensus.cc:3060] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.093261 19116 raft_consensus.cc:2749] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.106913 19226 consensus_queue.cc:1048] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:17:29.119278 19226 raft_consensus.cc:493] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.118788 18774 catalog_manager.cc:5697] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67). New cstate: current_term: 1 leader_uuid: "4bada1aa2db9455e874b4139f9ad1a40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:29.119753 19226 raft_consensus.cc:515] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.121799 19226 leader_election.cc:290] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:29.123046 19150 raft_consensus.cc:2749] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.125494 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ae37ec42794c0a927016a80d2a6d2f" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b37c12ab1f654afc812877fd13698893" is_pre_election: true
I20260709 13:17:29.130129 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6ae37ec42794c0a927016a80d2a6d2f" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:29.131035 18878 raft_consensus.cc:2393] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4bada1aa2db9455e874b4139f9ad1a40 in current term 1: Already voted for candidate b37c12ab1f654afc812877fd13698893 in this term.
I20260709 13:17:29.132997 18991 leader_election.cc:304] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40; no voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce
I20260709 13:17:29.134035 19227 raft_consensus.cc:3060] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.146925 19227 raft_consensus.cc:2749] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.153101 19185 consensus_queue.cc:1048] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.162108 19150 consensus_queue.cc:1048] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.166462 19185 consensus_queue.cc:1048] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.190294 19150 consensus_queue.cc:1048] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.280331 19116 consensus_queue.cc:1048] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.284479 19185 raft_consensus.cc:493] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.285085 19185 raft_consensus.cc:515] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.288604 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b337eb0188147028dd46c7cda6597ba" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:29.288779 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b337eb0188147028dd46c7cda6597ba" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:29.289204 19028 raft_consensus.cc:2468] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 0.
I20260709 13:17:29.289479 18878 raft_consensus.cc:2468] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 0.
I20260709 13:17:29.290741 18917 leader_election.cc:304] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:29.291620 19223 raft_consensus.cc:2804] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:29.291960 19223 raft_consensus.cc:493] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.292327 19223 raft_consensus.cc:3060] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.296710 19185 leader_election.cc:290] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:29.296823 19116 consensus_queue.cc:1048] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:29.299072 19223 raft_consensus.cc:515] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.301263 19223 leader_election.cc:290] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:29.301765 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b337eb0188147028dd46c7cda6597ba" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:29.302372 18878 raft_consensus.cc:3060] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.306010 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b337eb0188147028dd46c7cda6597ba" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:29.306995 19028 raft_consensus.cc:3060] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.309675 18878 raft_consensus.cc:2468] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 1.
I20260709 13:17:29.311012 18917 leader_election.cc:304] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:29.312098 19223 raft_consensus.cc:2804] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:29.312600 19223 raft_consensus.cc:697] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Becoming Leader. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:29.313298 19223 consensus_queue.cc:237] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.314132 19185 raft_consensus.cc:493] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.314321 19028 raft_consensus.cc:2468] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 1.
I20260709 13:17:29.314698 19185 raft_consensus.cc:515] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.316701 19185 leader_election.cc:290] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:29.317898 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c7d4ef242b4910820fea2c73a795b3" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40" is_pre_election: true
I20260709 13:17:29.318634 19028 raft_consensus.cc:2393] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b37c12ab1f654afc812877fd13698893 in current term 1: Already voted for candidate d1c86056b295463abc9760e7158f4bce in this term.
I20260709 13:17:29.318449 18878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8c7d4ef242b4910820fea2c73a795b3" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c86056b295463abc9760e7158f4bce" is_pre_election: true
I20260709 13:17:29.320210 18917 leader_election.cc:304] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b37c12ab1f654afc812877fd13698893; no voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce
I20260709 13:17:29.320963 19223 raft_consensus.cc:3060] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.326751 19223 raft_consensus.cc:2749] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.326686 18774 catalog_manager.cc:5697] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 reported cstate change: term changed from 0 to 1, leader changed from <none> to b37c12ab1f654afc812877fd13698893 (127.6.198.66). New cstate: current_term: 1 leader_uuid: "b37c12ab1f654afc812877fd13698893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.409898 19150 consensus_queue.cc:1048] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.420699 19212 consensus_queue.cc:1048] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.424480 19028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1aa4e1c6824341d0b7db3922adf767f7"
dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
mode: GRACEFUL
new_leader_uuid: "d1c86056b295463abc9760e7158f4bce"
 from {username='slave'} at 127.0.0.1:60712
I20260709 13:17:29.425240 19028 raft_consensus.cc:606] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Received request to transfer leadership to TS d1c86056b295463abc9760e7158f4bce
I20260709 13:17:29.431444 18954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b337eb0188147028dd46c7cda6597ba"
dest_uuid: "b37c12ab1f654afc812877fd13698893"
mode: GRACEFUL
new_leader_uuid: "d1c86056b295463abc9760e7158f4bce"
 from {username='slave'} at 127.0.0.1:54088
I20260709 13:17:29.432011 18954 raft_consensus.cc:606] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Received request to transfer leadership to TS d1c86056b295463abc9760e7158f4bce
I20260709 13:17:29.437879 18953 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "00a17ef3085648e69f49a68f06005e60"
dest_uuid: "b37c12ab1f654afc812877fd13698893"
mode: GRACEFUL
new_leader_uuid: "d1c86056b295463abc9760e7158f4bce"
 from {username='slave'} at 127.0.0.1:54088
I20260709 13:17:29.438480 18953 raft_consensus.cc:606] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Received request to transfer leadership to TS d1c86056b295463abc9760e7158f4bce
I20260709 13:17:29.443647 19028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01a469d775f84c398d3030e2e0adcdae"
dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
mode: GRACEFUL
new_leader_uuid: "b37c12ab1f654afc812877fd13698893"
 from {username='slave'} at 127.0.0.1:60712
I20260709 13:17:29.444125 19028 raft_consensus.cc:606] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Received request to transfer leadership to TS b37c12ab1f654afc812877fd13698893
I20260709 13:17:29.450850 18878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b873aaeb80544cd8c78969a9c265c2c"
dest_uuid: "d1c86056b295463abc9760e7158f4bce"
mode: GRACEFUL
new_leader_uuid: "b37c12ab1f654afc812877fd13698893"
 from {username='slave'} at 127.0.0.1:46190
I20260709 13:17:29.451324 18878 raft_consensus.cc:606] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Received request to transfer leadership to TS b37c12ab1f654afc812877fd13698893
I20260709 13:17:29.452541 19223 consensus_queue.cc:1048] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.463053 19191 consensus_queue.cc:1048] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.605275 19212 raft_consensus.cc:993] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce: : Instructing follower b37c12ab1f654afc812877fd13698893 to start an election
I20260709 13:17:29.605767 19150 raft_consensus.cc:1081] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Signalling peer b37c12ab1f654afc812877fd13698893 to start an election
I20260709 13:17:29.607194 18953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b873aaeb80544cd8c78969a9c265c2c"
dest_uuid: "b37c12ab1f654afc812877fd13698893"
 from {username='slave'} at 127.0.0.1:54062
I20260709 13:17:29.607559 18953 raft_consensus.cc:493] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:29.607779 18953 raft_consensus.cc:3060] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.612484 18953 raft_consensus.cc:515] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.613977 18953 leader_election.cc:290] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 2 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:29.614658 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b873aaeb80544cd8c78969a9c265c2c" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:29.615015 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b873aaeb80544cd8c78969a9c265c2c" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:29.615342 18879 raft_consensus.cc:3055] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.615561 19028 raft_consensus.cc:3060] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.615686 18879 raft_consensus.cc:740] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:29.616460 18879 consensus_queue.cc:260] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.617612 18879 raft_consensus.cc:3060] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.621692 19028 raft_consensus.cc:2468] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 2.
I20260709 13:17:29.622843 18920 leader_election.cc:304] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, b37c12ab1f654afc812877fd13698893; no voters: 
I20260709 13:17:29.623493 18879 raft_consensus.cc:2468] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 2.
I20260709 13:17:29.623584 19223 raft_consensus.cc:2804] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:29.624192 19223 raft_consensus.cc:697] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [term 2 LEADER]: Becoming Leader. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:29.624958 19223 consensus_queue.cc:237] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.630923 18775 catalog_manager.cc:5697] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 reported cstate change: term changed from 1 to 2, leader changed from d1c86056b295463abc9760e7158f4bce (127.6.198.65) to b37c12ab1f654afc812877fd13698893 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "b37c12ab1f654afc812877fd13698893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.639089 19116 consensus_queue.cc:1048] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.648516 19226 consensus_queue.cc:1048] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.659423 19191 consensus_queue.cc:1048] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.662169 19191 consensus_queue.cc:1048] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.668457 19223 raft_consensus.cc:993] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893: : Instructing follower d1c86056b295463abc9760e7158f4bce to start an election
I20260709 13:17:29.669178 19191 raft_consensus.cc:1081] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Signalling peer d1c86056b295463abc9760e7158f4bce to start an election
I20260709 13:17:29.670732 18879 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "00a17ef3085648e69f49a68f06005e60"
dest_uuid: "d1c86056b295463abc9760e7158f4bce"
 from {username='slave'} at 127.0.0.1:46178
I20260709 13:17:29.672861 18879 raft_consensus.cc:493] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:29.673208 18879 raft_consensus.cc:3060] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.686371 18879 raft_consensus.cc:515] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.688364 18879 leader_election.cc:290] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 2 election: Requested vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:29.688678 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00a17ef3085648e69f49a68f06005e60" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:29.689266 18953 raft_consensus.cc:3055] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.689541 18953 raft_consensus.cc:740] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:29.690272 18953 consensus_queue.cc:260] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.691414 18953 raft_consensus.cc:3060] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.691954 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00a17ef3085648e69f49a68f06005e60" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:29.692507 19028 raft_consensus.cc:3060] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.694499 19205 raft_consensus.cc:993] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40: : Instructing follower d1c86056b295463abc9760e7158f4bce to start an election
I20260709 13:17:29.694963 19205 raft_consensus.cc:1081] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Signalling peer d1c86056b295463abc9760e7158f4bce to start an election
I20260709 13:17:29.697664 18953 raft_consensus.cc:2468] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 2.
I20260709 13:17:29.697701 18879 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1aa4e1c6824341d0b7db3922adf767f7"
dest_uuid: "d1c86056b295463abc9760e7158f4bce"
 from {username='slave'} at 127.0.0.1:46180
I20260709 13:17:29.697835 19028 raft_consensus.cc:2468] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 2.
I20260709 13:17:29.698536 18879 raft_consensus.cc:493] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:29.698928 18879 raft_consensus.cc:3060] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.699152 18845 leader_election.cc:304] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:29.699951 19150 raft_consensus.cc:2804] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:29.700352 19150 raft_consensus.cc:697] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [term 2 LEADER]: Becoming Leader. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:29.701067 19150 consensus_queue.cc:237] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.703693 18879 raft_consensus.cc:515] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.705344 18879 leader_election.cc:290] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 2 election: Requested vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:29.706110 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aa4e1c6824341d0b7db3922adf767f7" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:29.706460 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1aa4e1c6824341d0b7db3922adf767f7" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:29.706753 18953 raft_consensus.cc:3060] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.707005 19028 raft_consensus.cc:3055] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.707347 19028 raft_consensus.cc:740] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Running, Role: LEADER
I20260709 13:17:29.708027 19028 consensus_queue.cc:260] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.709134 19028 raft_consensus.cc:3060] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.708654 18776 catalog_manager.cc:5697] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce reported cstate change: term changed from 1 to 2, leader changed from b37c12ab1f654afc812877fd13698893 (127.6.198.66) to d1c86056b295463abc9760e7158f4bce (127.6.198.65). New cstate: current_term: 2 leader_uuid: "d1c86056b295463abc9760e7158f4bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.711591 18953 raft_consensus.cc:2468] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 2.
I20260709 13:17:29.712781 18842 leader_election.cc:304] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:29.714305 19150 raft_consensus.cc:2804] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:29.714799 19150 raft_consensus.cc:697] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [term 2 LEADER]: Becoming Leader. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:29.715396 19150 consensus_queue.cc:237] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.716579 19028 raft_consensus.cc:2468] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 2.
I20260709 13:17:29.721481 18776 catalog_manager.cc:5697] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce reported cstate change: term changed from 1 to 2, leader changed from 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67) to d1c86056b295463abc9760e7158f4bce (127.6.198.65). New cstate: current_term: 2 leader_uuid: "d1c86056b295463abc9760e7158f4bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.787029 19223 consensus_queue.cc:1048] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.796356 19222 consensus_queue.cc:1048] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.804400 19222 raft_consensus.cc:993] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893: : Instructing follower d1c86056b295463abc9760e7158f4bce to start an election
I20260709 13:17:29.804826 19185 raft_consensus.cc:1081] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Signalling peer d1c86056b295463abc9760e7158f4bce to start an election
I20260709 13:17:29.806259 18879 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b337eb0188147028dd46c7cda6597ba"
dest_uuid: "d1c86056b295463abc9760e7158f4bce"
 from {username='slave'} at 127.0.0.1:46178
I20260709 13:17:29.806734 18879 raft_consensus.cc:493] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:29.807039 18879 raft_consensus.cc:3060] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.811316 18879 raft_consensus.cc:515] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.812877 18879 leader_election.cc:290] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 2 election: Requested vote from peers b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:29.813661 18954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b337eb0188147028dd46c7cda6597ba" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:29.813854 19028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b337eb0188147028dd46c7cda6597ba" candidate_uuid: "d1c86056b295463abc9760e7158f4bce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:29.814312 18954 raft_consensus.cc:3055] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.814448 19028 raft_consensus.cc:3060] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.814718 18954 raft_consensus.cc:740] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:29.815577 18954 consensus_queue.cc:260] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.816630 18954 raft_consensus.cc:3060] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.820007 19028 raft_consensus.cc:2468] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 2.
I20260709 13:17:29.820962 18845 leader_election.cc:304] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:29.821938 19150 raft_consensus.cc:2804] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:29.822422 19150 raft_consensus.cc:697] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 2 LEADER]: Becoming Leader. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:29.823230 19150 consensus_queue.cc:237] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.823417 18954 raft_consensus.cc:2468] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1c86056b295463abc9760e7158f4bce in term 2.
I20260709 13:17:29.829345 18775 catalog_manager.cc:5697] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce reported cstate change: term changed from 1 to 2, leader changed from b37c12ab1f654afc812877fd13698893 (127.6.198.66) to d1c86056b295463abc9760e7158f4bce (127.6.198.65). New cstate: current_term: 2 leader_uuid: "d1c86056b295463abc9760e7158f4bce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.851648 19226 raft_consensus.cc:993] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40: : Instructing follower b37c12ab1f654afc812877fd13698893 to start an election
I20260709 13:17:29.852079 19226 raft_consensus.cc:1081] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Signalling peer b37c12ab1f654afc812877fd13698893 to start an election
I20260709 13:17:29.853477 18954 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "01a469d775f84c398d3030e2e0adcdae"
dest_uuid: "b37c12ab1f654afc812877fd13698893"
 from {username='slave'} at 127.0.0.1:54078
I20260709 13:17:29.853996 18954 raft_consensus.cc:493] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:29.854380 18954 raft_consensus.cc:3060] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.859709 18954 raft_consensus.cc:515] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.861472 18954 leader_election.cc:290] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 2 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67:37349)
I20260709 13:17:29.861943 18879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01a469d775f84c398d3030e2e0adcdae" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:29.862545 18879 raft_consensus.cc:3060] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.862612 19027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01a469d775f84c398d3030e2e0adcdae" candidate_uuid: "b37c12ab1f654afc812877fd13698893" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
I20260709 13:17:29.863269 19027 raft_consensus.cc:3055] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:29.863636 19027 raft_consensus.cc:740] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Running, Role: LEADER
I20260709 13:17:29.864459 19027 consensus_queue.cc:260] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.865382 19027 raft_consensus.cc:3060] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:29.867630 18879 raft_consensus.cc:2468] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 2.
I20260709 13:17:29.868507 18917 leader_election.cc:304] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b37c12ab1f654afc812877fd13698893, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:29.869093 19185 raft_consensus.cc:2804] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:29.869549 19185 raft_consensus.cc:697] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 2 LEADER]: Becoming Leader. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:29.870398 19027 raft_consensus.cc:2468] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b37c12ab1f654afc812877fd13698893 in term 2.
I20260709 13:17:29.870308 19185 consensus_queue.cc:237] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:29.876753 18775 catalog_manager.cc:5697] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 reported cstate change: term changed from 1 to 2, leader changed from 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67) to b37c12ab1f654afc812877fd13698893 (127.6.198.66). New cstate: current_term: 2 leader_uuid: "b37c12ab1f654afc812877fd13698893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:30.031168 19027 raft_consensus.cc:1275] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Refusing update from remote peer b37c12ab1f654afc812877fd13698893: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.032224 19185 consensus_queue.cc:1048] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:30.039016 18879 raft_consensus.cc:1275] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Refusing update from remote peer b37c12ab1f654afc812877fd13698893: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.040450 19185 consensus_queue.cc:1048] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:30.221192 19027 raft_consensus.cc:1275] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Refusing update from remote peer d1c86056b295463abc9760e7158f4bce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.222451 19252 consensus_queue.cc:1048] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:30.229701 18954 raft_consensus.cc:1275] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Refusing update from remote peer d1c86056b295463abc9760e7158f4bce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.231429 19252 consensus_queue.cc:1048] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:30.239233 18954 raft_consensus.cc:1275] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Refusing update from remote peer d1c86056b295463abc9760e7158f4bce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.240342 19150 consensus_queue.cc:1048] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:30.251756 19027 raft_consensus.cc:1275] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Refusing update from remote peer d1c86056b295463abc9760e7158f4bce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.253333 19252 consensus_queue.cc:1048] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:30.279534 18954 raft_consensus.cc:1275] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Refusing update from remote peer d1c86056b295463abc9760e7158f4bce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.280654 19252 consensus_queue.cc:1048] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:30.290414 19027 raft_consensus.cc:1275] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Refusing update from remote peer d1c86056b295463abc9760e7158f4bce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.292243 19255 consensus_queue.cc:1048] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:30.327419 18879 raft_consensus.cc:1275] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Refusing update from remote peer b37c12ab1f654afc812877fd13698893: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.328536 19185 consensus_queue.cc:1048] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:30.336247 19027 raft_consensus.cc:1275] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Refusing update from remote peer b37c12ab1f654afc812877fd13698893: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:30.337540 19222 consensus_queue.cc:1048] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:32.460628 18775 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60998:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:17:32.488567 18775 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60998:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:17:32.543985  6937 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:32.741699 19285 log.cc:927] Time spent T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40: Append to log took a long time: real 0.061s	user 0.004s	sys 0.000s
I20260709 13:17:32.861924 19306 tablet.cc:1724] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.897473 19318 tablet.cc:1724] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.877465 19314 tablet.cc:1724] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.968528 19315 tablet.cc:1724] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:17:32.971277 19313 log.cc:927] Time spent T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:17:32.976313 19317 tablet.cc:1724] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.988297 19320 tablet.cc:1724] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.002873 19322 tablet.cc:1724] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.978693 19316 tablet.cc:1724] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.015725 19329 tablet.cc:1724] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.006371 19323 tablet.cc:1724] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.019104 19328 tablet.cc:1724] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:32.998085 19324 tablet.cc:1724] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.032238 18879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d0c9c2b6ccc40ee8a9b8265ef14ad8a"
dest_uuid: "d1c86056b295463abc9760e7158f4bce"
mode: GRACEFUL
new_leader_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
 from {username='slave'} at 127.0.0.1:34294
I20260709 13:17:33.033314 18879 raft_consensus.cc:606] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Received request to transfer leadership to TS 4bada1aa2db9455e874b4139f9ad1a40
I20260709 13:17:33.064111 19323 tablet.cc:1724] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:17:33.081722 18920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.67:37349, user_credentials={real_user=slave}} blocked reactor thread for 62186.3us
I20260709 13:17:33.083829 19306 tablet.cc:1724] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.095817 19320 tablet.cc:1724] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:17:33.098433 18991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.6.198.65:34207, user_credentials={real_user=slave}} blocked reactor thread for 61381.2us
I20260709 13:17:33.102962 19328 tablet.cc:1724] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.106029 19314 tablet.cc:1724] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.109408 19321 raft_consensus.cc:993] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce: : Instructing follower 4bada1aa2db9455e874b4139f9ad1a40 to start an election
I20260709 13:17:33.109947 19321 raft_consensus.cc:1081] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Signalling peer 4bada1aa2db9455e874b4139f9ad1a40 to start an election
I20260709 13:17:33.117415 19317 tablet.cc:1724] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.120383 19323 tablet.cc:1724] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.122265 19320 tablet.cc:1724] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.132232 19315 tablet.cc:1724] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.134277 19320 tablet.cc:1724] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.139281 19329 tablet.cc:1724] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.143668 19318 tablet.cc:1724] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.160665 19317 tablet.cc:1724] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.170075 19324 tablet.cc:1724] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.173273 19328 tablet.cc:1724] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:33.179956 18953 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf5def05512948249903a00f64386054"
dest_uuid: "b37c12ab1f654afc812877fd13698893"
mode: GRACEFUL
new_leader_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
 from {username='slave'} at 127.0.0.1:55670
I20260709 13:17:33.181496 18953 raft_consensus.cc:606] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Received request to transfer leadership to TS 4bada1aa2db9455e874b4139f9ad1a40
I20260709 13:17:33.182451 19023 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d0c9c2b6ccc40ee8a9b8265ef14ad8a"
dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
 from {username='slave'} at 127.0.0.1:60684
I20260709 13:17:33.183169 19023 raft_consensus.cc:493] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:33.183631 19023 raft_consensus.cc:3060] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:33.183828  6937 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:33.184712  6937 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:33.202929 19023 raft_consensus.cc:515] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:33.206946 18876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d0c9c2b6ccc40ee8a9b8265ef14ad8a" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:33.207010 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d0c9c2b6ccc40ee8a9b8265ef14ad8a" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:33.207641 18876 raft_consensus.cc:3055] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:33.207895 18953 raft_consensus.cc:3060] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:33.208030 18876 raft_consensus.cc:740] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d1c86056b295463abc9760e7158f4bce, State: Running, Role: LEADER
I20260709 13:17:33.209023 18876 consensus_queue.cc:260] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:33.210253 18876 raft_consensus.cc:3060] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:33.214428 19023 leader_election.cc:290] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 2 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:33.220737 18876 raft_consensus.cc:2468] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 2.
I20260709 13:17:33.221906 18991 leader_election.cc:304] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:33.222178 18953 raft_consensus.cc:2468] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 2.
I20260709 13:17:33.222676 19116 raft_consensus.cc:2804] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:33.223176 19116 raft_consensus.cc:697] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 LEADER]: Becoming Leader. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Running, Role: LEADER
I20260709 13:17:33.224072 19116 consensus_queue.cc:237] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:33.226193 18775 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c: soft_deleted_table [id=5833794a36e24cae88370a9b222d6f1c] alter complete (version 1)
I20260709 13:17:33.239349 18776 catalog_manager.cc:5697] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 reported cstate change: term changed from 1 to 2, leader changed from d1c86056b295463abc9760e7158f4bce (127.6.198.65) to 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67). New cstate: current_term: 2 leader_uuid: "4bada1aa2db9455e874b4139f9ad1a40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:33.356976 19297 raft_consensus.cc:993] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893: : Instructing follower 4bada1aa2db9455e874b4139f9ad1a40 to start an election
I20260709 13:17:33.357611 19262 raft_consensus.cc:1081] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Signalling peer 4bada1aa2db9455e874b4139f9ad1a40 to start an election
I20260709 13:17:33.358919 19023 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf5def05512948249903a00f64386054"
dest_uuid: "4bada1aa2db9455e874b4139f9ad1a40"
 from {username='slave'} at 127.0.0.1:60696
I20260709 13:17:33.359316 19023 raft_consensus.cc:493] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:33.359551 19023 raft_consensus.cc:3060] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:33.364365 19023 raft_consensus.cc:515] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:33.365866 19023 leader_election.cc:290] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 2 election: Requested vote from peers d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207), b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923)
I20260709 13:17:33.366539 18876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5def05512948249903a00f64386054" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d1c86056b295463abc9760e7158f4bce"
I20260709 13:17:33.366750 18953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf5def05512948249903a00f64386054" candidate_uuid: "4bada1aa2db9455e874b4139f9ad1a40" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b37c12ab1f654afc812877fd13698893"
I20260709 13:17:33.367228 18876 raft_consensus.cc:3060] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:33.367347 18953 raft_consensus.cc:3055] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:33.367707 18953 raft_consensus.cc:740] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b37c12ab1f654afc812877fd13698893, State: Running, Role: LEADER
I20260709 13:17:33.368270 18953 consensus_queue.cc:260] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:33.369127 18953 raft_consensus.cc:3060] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:33.372105 18876 raft_consensus.cc:2468] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 2.
I20260709 13:17:33.373261 18953 raft_consensus.cc:2468] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4bada1aa2db9455e874b4139f9ad1a40 in term 2.
I20260709 13:17:33.373230 18991 leader_election.cc:304] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bada1aa2db9455e874b4139f9ad1a40, d1c86056b295463abc9760e7158f4bce; no voters: 
I20260709 13:17:33.374198 19116 raft_consensus.cc:2804] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:33.374680 19116 raft_consensus.cc:697] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 LEADER]: Becoming Leader. State: Replica: 4bada1aa2db9455e874b4139f9ad1a40, State: Running, Role: LEADER
I20260709 13:17:33.375428 19116 consensus_queue.cc:237] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } }
I20260709 13:17:33.384238 18776 catalog_manager.cc:5697] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 reported cstate change: term changed from 1 to 2, leader changed from b37c12ab1f654afc812877fd13698893 (127.6.198.66) to 4bada1aa2db9455e874b4139f9ad1a40 (127.6.198.67). New cstate: current_term: 2 leader_uuid: "4bada1aa2db9455e874b4139f9ad1a40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4bada1aa2db9455e874b4139f9ad1a40" member_type: VOTER last_known_addr { host: "127.6.198.67" port: 37349 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:33.725391 18953 raft_consensus.cc:1275] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Refusing update from remote peer 4bada1aa2db9455e874b4139f9ad1a40: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:33.726471 19116 consensus_queue.cc:1048] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:33.739516 18876 raft_consensus.cc:1275] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Refusing update from remote peer 4bada1aa2db9455e874b4139f9ad1a40: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:33.740800 19116 consensus_queue.cc:1048] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:33.770509 18876 raft_consensus.cc:1275] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Refusing update from remote peer 4bada1aa2db9455e874b4139f9ad1a40: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:33.771606 19116 consensus_queue.cc:1048] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c86056b295463abc9760e7158f4bce" member_type: VOTER last_known_addr { host: "127.6.198.65" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:33.781734 18952 raft_consensus.cc:1275] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Refusing update from remote peer 4bada1aa2db9455e874b4139f9ad1a40: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:33.783529 19348 consensus_queue.cc:1048] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b37c12ab1f654afc812877fd13698893" member_type: VOTER last_known_addr { host: "127.6.198.66" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:34.237308  6937 tablet_server.cc:179] TabletServer@127.6.198.65:0 shutting down...
I20260709 13:17:34.266314  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:34.267036  6937 tablet_replica.cc:333] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.267658  6937 raft_consensus.cc:2243] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.268052  6937 raft_consensus.cc:2272] T 07373452952c4549acb3d36b47e24e3a P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.270066  6937 tablet_replica.cc:333] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.270663  6937 raft_consensus.cc:2243] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.271360  6937 raft_consensus.cc:2272] T 00a17ef3085648e69f49a68f06005e60 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:34.274019 18991 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 9 similar messages]
I20260709 13:17:34.274070  6937 tablet_replica.cc:333] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.275182  6937 raft_consensus.cc:2243] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.276228  6937 raft_consensus.cc:2272] T b8c7d4ef242b4910820fea2c73a795b3 P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:34.277180 18991 consensus_peers.cc:597] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 -> Peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Couldn't send request to peer d1c86056b295463abc9760e7158f4bce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:34.279403  6937 tablet_replica.cc:333] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.280241  6937 raft_consensus.cc:2243] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.280725  6937 raft_consensus.cc:2272] T c6ae37ec42794c0a927016a80d2a6d2f P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.283337  6937 tablet_replica.cc:333] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.283874  6937 raft_consensus.cc:2243] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.284375  6937 raft_consensus.cc:2272] T 01a469d775f84c398d3030e2e0adcdae P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.286559  6937 tablet_replica.cc:333] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.287139  6937 raft_consensus.cc:2243] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.287560  6937 raft_consensus.cc:2272] T cf5def05512948249903a00f64386054 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.290292  6937 tablet_replica.cc:333] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.290997  6937 raft_consensus.cc:2243] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.291510  6937 raft_consensus.cc:2272] T 4b873aaeb80544cd8c78969a9c265c2c P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.293763  6937 tablet_replica.cc:333] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.294399  6937 raft_consensus.cc:2243] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.294884  6937 raft_consensus.cc:2272] T 1e957ba42136417fba964b3408dfec86 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.296902  6937 tablet_replica.cc:333] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.297469  6937 raft_consensus.cc:2243] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.297909  6937 raft_consensus.cc:2272] T a0a28ca432c040e090e4540740f46aa6 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.300004  6937 tablet_replica.cc:333] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.300542  6937 raft_consensus.cc:2243] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.301330  6937 raft_consensus.cc:2272] T f1dedc61807b4f1fb0b0df1fd7408660 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.303457  6937 tablet_replica.cc:333] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.303954  6937 raft_consensus.cc:2243] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.304664  6937 raft_consensus.cc:2272] T 1aa4e1c6824341d0b7db3922adf767f7 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.306689  6937 tablet_replica.cc:333] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.307209  6937 raft_consensus.cc:2243] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.307955  6937 raft_consensus.cc:2272] T 4799ea090d564639bc7e3998262adc88 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.310104  6937 tablet_replica.cc:333] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.310654  6937 raft_consensus.cc:2243] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.311100  6937 raft_consensus.cc:2272] T 5d0a1577fa0f4741ab0ee73851eb1294 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.312983  6937 tablet_replica.cc:333] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.313580  6937 raft_consensus.cc:2243] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.314313  6937 raft_consensus.cc:2272] T 8b337eb0188147028dd46c7cda6597ba P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.316242  6937 tablet_replica.cc:333] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.316733  6937 raft_consensus.cc:2243] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.317531  6937 raft_consensus.cc:2272] T 5a8849ca00284e83968f5c4d4f7a5f59 P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.319696  6937 tablet_replica.cc:333] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.320247  6937 raft_consensus.cc:2243] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.320713  6937 raft_consensus.cc:2272] T 8b1c41abe4964625829a184924c60d3a P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.324867  6937 tablet_replica.cc:333] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.325412  6937 raft_consensus.cc:2243] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.325865  6937 raft_consensus.cc:2272] T db427c7f60374393bc721fed01136cdf P d1c86056b295463abc9760e7158f4bce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.327831  6937 tablet_replica.cc:333] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce: stopping tablet replica
I20260709 13:17:34.328333  6937 raft_consensus.cc:2243] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.328730  6937 raft_consensus.cc:2272] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P d1c86056b295463abc9760e7158f4bce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.368376  6937 tablet_server.cc:196] TabletServer@127.6.198.65:0 shutdown complete.
W20260709 13:17:34.403331 18991 consensus_peers.cc:597] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 -> Peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Couldn't send request to peer d1c86056b295463abc9760e7158f4bce. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:34207: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:34.406778  6937 tablet_server.cc:179] TabletServer@127.6.198.66:0 shutting down...
W20260709 13:17:34.436374 18917 consensus_peers.cc:597] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 -> Peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Couldn't send request to peer d1c86056b295463abc9760e7158f4bce. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:34207: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:34.440248  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:34.441061  6937 tablet_replica.cc:333] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.442180  6937 raft_consensus.cc:2243] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.442837  6937 raft_consensus.cc:2272] T b8c7d4ef242b4910820fea2c73a795b3 P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.444769  6937 tablet_replica.cc:333] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.445317  6937 raft_consensus.cc:2243] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.445986  6937 raft_consensus.cc:2272] T db427c7f60374393bc721fed01136cdf P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.447979  6937 tablet_replica.cc:333] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.448858  6937 raft_consensus.cc:2243] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.449424  6937 raft_consensus.cc:2272] T 00a17ef3085648e69f49a68f06005e60 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.451787  6937 tablet_replica.cc:333] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.452343  6937 raft_consensus.cc:2243] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.452759  6937 raft_consensus.cc:2272] T 07373452952c4549acb3d36b47e24e3a P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.454571  6937 tablet_replica.cc:333] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.455165  6937 raft_consensus.cc:2243] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.455968  6937 raft_consensus.cc:2272] T 4b873aaeb80544cd8c78969a9c265c2c P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.457914  6937 tablet_replica.cc:333] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.458525  6937 raft_consensus.cc:2243] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.459249  6937 raft_consensus.cc:2272] T 01a469d775f84c398d3030e2e0adcdae P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.461117  6937 tablet_replica.cc:333] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.461649  6937 raft_consensus.cc:2243] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.462029  6937 raft_consensus.cc:2272] T cf5def05512948249903a00f64386054 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.464612  6937 tablet_replica.cc:333] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.465348  6937 raft_consensus.cc:2243] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.466297  6937 raft_consensus.cc:2272] T a0a28ca432c040e090e4540740f46aa6 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.468643  6937 tablet_replica.cc:333] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893: stopping tablet replica
W20260709 13:17:34.468804 18991 consensus_peers.cc:597] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 -> Peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Couldn't send request to peer d1c86056b295463abc9760e7158f4bce. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:34207: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:34.469233  6937 raft_consensus.cc:2243] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.469913  6937 raft_consensus.cc:2272] T 1e957ba42136417fba964b3408dfec86 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.472788  6937 tablet_replica.cc:333] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.473558  6937 raft_consensus.cc:2243] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:17:34.473886 18917 consensus_peers.cc:597] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 -> Peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Couldn't send request to peer d1c86056b295463abc9760e7158f4bce. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:34207: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:34.474776  6937 raft_consensus.cc:2272] T 5d0a1577fa0f4741ab0ee73851eb1294 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.476851  6937 tablet_replica.cc:333] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.477391  6937 raft_consensus.cc:2243] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.478122  6937 raft_consensus.cc:2272] T 8b1c41abe4964625829a184924c60d3a P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.480059  6937 tablet_replica.cc:333] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.480571  6937 raft_consensus.cc:2243] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.480994  6937 raft_consensus.cc:2272] T 8b337eb0188147028dd46c7cda6597ba P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.482976  6937 tablet_replica.cc:333] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.483495  6937 raft_consensus.cc:2243] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.483986  6937 raft_consensus.cc:2272] T 5a8849ca00284e83968f5c4d4f7a5f59 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.486249  6937 tablet_replica.cc:333] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.486758  6937 raft_consensus.cc:2243] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.487118  6937 raft_consensus.cc:2272] T 1aa4e1c6824341d0b7db3922adf767f7 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.488960  6937 tablet_replica.cc:333] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.489408  6937 raft_consensus.cc:2243] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.489751  6937 raft_consensus.cc:2272] T 4799ea090d564639bc7e3998262adc88 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.491715  6937 tablet_replica.cc:333] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.492161  6937 raft_consensus.cc:2243] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.492487  6937 raft_consensus.cc:2272] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.494625  6937 tablet_replica.cc:333] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.495045  6937 raft_consensus.cc:2243] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.495692  6937 raft_consensus.cc:2272] T c6ae37ec42794c0a927016a80d2a6d2f P b37c12ab1f654afc812877fd13698893 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.497397  6937 tablet_replica.cc:333] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893: stopping tablet replica
I20260709 13:17:34.497797  6937 raft_consensus.cc:2243] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.498178  6937 raft_consensus.cc:2272] T f1dedc61807b4f1fb0b0df1fd7408660 P b37c12ab1f654afc812877fd13698893 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.535660  6937 tablet_server.cc:196] TabletServer@127.6.198.66:0 shutdown complete.
W20260709 13:17:34.558841 18991 consensus_peers.cc:597] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 -> Peer d1c86056b295463abc9760e7158f4bce (127.6.198.65:34207): Couldn't send request to peer d1c86056b295463abc9760e7158f4bce. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.65:34207: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:34.570178  6937 tablet_server.cc:179] TabletServer@127.6.198.67:0 shutting down...
I20260709 13:17:34.596344  6937 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:34.597329  6937 tablet_replica.cc:333] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.598060  6937 raft_consensus.cc:2243] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:34.598384 18991 consensus_peers.cc:597] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 -> Peer b37c12ab1f654afc812877fd13698893 (127.6.198.66:39923): Couldn't send request to peer b37c12ab1f654afc812877fd13698893. Status: Network error: Client connection negotiation failed: client connection to 127.6.198.66:39923: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:34.598590  6937 raft_consensus.cc:2272] T c6ae37ec42794c0a927016a80d2a6d2f P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.600701  6937 tablet_replica.cc:333] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.601251  6937 raft_consensus.cc:2243] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.601655  6937 raft_consensus.cc:2272] T db427c7f60374393bc721fed01136cdf P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.603718  6937 tablet_replica.cc:333] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.604288  6937 raft_consensus.cc:2243] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.604990  6937 raft_consensus.cc:2272] T 07373452952c4549acb3d36b47e24e3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.606909  6937 tablet_replica.cc:333] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.607437  6937 raft_consensus.cc:2243] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.607852  6937 raft_consensus.cc:2272] T 00a17ef3085648e69f49a68f06005e60 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.609613  6937 tablet_replica.cc:333] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.610165  6937 raft_consensus.cc:2243] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.610623  6937 raft_consensus.cc:2272] T 4b873aaeb80544cd8c78969a9c265c2c P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.612444  6937 tablet_replica.cc:333] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.612951  6937 raft_consensus.cc:2243] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.613379  6937 raft_consensus.cc:2272] T 01a469d775f84c398d3030e2e0adcdae P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.615257  6937 tablet_replica.cc:333] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.615784  6937 raft_consensus.cc:2243] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.616497  6937 raft_consensus.cc:2272] T cf5def05512948249903a00f64386054 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.618845  6937 tablet_replica.cc:333] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.619375  6937 raft_consensus.cc:2243] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.619809  6937 raft_consensus.cc:2272] T f1dedc61807b4f1fb0b0df1fd7408660 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.621757  6937 tablet_replica.cc:333] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.622282  6937 raft_consensus.cc:2243] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.622774  6937 raft_consensus.cc:2272] T 4799ea090d564639bc7e3998262adc88 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.624603  6937 tablet_replica.cc:333] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.625087  6937 raft_consensus.cc:2243] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.625523  6937 raft_consensus.cc:2272] T 1aa4e1c6824341d0b7db3922adf767f7 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.627364  6937 tablet_replica.cc:333] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.627835  6937 raft_consensus.cc:2243] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.628291  6937 raft_consensus.cc:2272] T 5d0a1577fa0f4741ab0ee73851eb1294 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.630328  6937 tablet_replica.cc:333] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.630883  6937 raft_consensus.cc:2243] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.631290  6937 raft_consensus.cc:2272] T 8b337eb0188147028dd46c7cda6597ba P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.633093  6937 tablet_replica.cc:333] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.633605  6937 raft_consensus.cc:2243] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.634021  6937 raft_consensus.cc:2272] T 5a8849ca00284e83968f5c4d4f7a5f59 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.635910  6937 tablet_replica.cc:333] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.636391  6937 raft_consensus.cc:2243] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.636829  6937 raft_consensus.cc:2272] T 8b1c41abe4964625829a184924c60d3a P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.638682  6937 tablet_replica.cc:333] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.639163  6937 raft_consensus.cc:2243] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.639585  6937 raft_consensus.cc:2272] T b8c7d4ef242b4910820fea2c73a795b3 P 4bada1aa2db9455e874b4139f9ad1a40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.641348  6937 tablet_replica.cc:333] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.641817  6937 raft_consensus.cc:2243] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.642584  6937 raft_consensus.cc:2272] T 6d0c9c2b6ccc40ee8a9b8265ef14ad8a P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.644800  6937 tablet_replica.cc:333] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.645296  6937 raft_consensus.cc:2243] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:34.645709  6937 raft_consensus.cc:2272] T a0a28ca432c040e090e4540740f46aa6 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.647624  6937 tablet_replica.cc:333] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40: stopping tablet replica
I20260709 13:17:34.648159  6937 raft_consensus.cc:2243] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.648871  6937 raft_consensus.cc:2272] T 1e957ba42136417fba964b3408dfec86 P 4bada1aa2db9455e874b4139f9ad1a40 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.687288  6937 tablet_server.cc:196] TabletServer@127.6.198.67:0 shutdown complete.
I20260709 13:17:34.717712  6937 master.cc:562] Master@127.6.198.126:41745 shutting down...
I20260709 13:17:34.735386  6937 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:34.735853  6937 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:34.736123  6937 tablet_replica.cc:333] T 00000000000000000000000000000000 P 799f3c691b394190874bfeefbd01c18c: stopping tablet replica
I20260709 13:17:34.755967  6937 master.cc:584] Master@127.6.198.126:41745 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (12146 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (24827 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (335657 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:17:34.810935  6937 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 395 messages since previous log ~15 seconds ago
I20260709 13:17:34.811132  6937 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 400 messages since previous log ~15 seconds ago
I20260709 13:17:34.811281  6937 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 400 messages since previous log ~15 seconds ago
I20260709 13:17:34.811429  6937 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:17:34.811571  6937 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:17:34.811712  6937 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 5 messages since previous log ~160 seconds ago
I20260709 13:17:34.811851  6937 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 3 messages since previous log ~269 seconds ago
I20260709 13:17:34.811980  6937 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 3 messages since previous log ~270 seconds ago
I20260709 13:17:34.812186  6937 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 6 messages since previous log ~0 seconds ago
I20260709 13:17:34.812417  6937 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 26 messages since previous log ~6 seconds ago
